UPDATE: There is now an accepted answer that "works". You should never, ever, ever, ever use it. Ever.
First let me preface my question by stating that I'm a game developer. There's a legitimate - if highly unusual - performance-related reason for wanting to do this.
Say I have a C# class like this:
class Foo
{
public int a, b, c;
public void MyMethod(int d) { a = d; b = d; c = a + b; }
}
Nothing fancy. Note that it is a reference type that contains only value types.
In managed code I'd like to have something like this:
Foo foo;
foo = Voodoo.NewInUnmanagedMemory<Foo>(); // <- ???
foo.MyMethod(1);
What would the function NewInUnmanagedMemory look like? If it can't be done in C#, could it be done in IL? (Or maybe C++/CLI?)
Basically: Is there a way - no matter how hacky - to turn some totally arbitrary pointer into an object reference. And - short of making the CLR explode - damn the consequences.
(Another way to put my question is: "I want to implement a custom allocator for C#")
This leads to the follow-up question: What does the garbage collector do (implementation-specific, if need be) when faced with a reference that points outside of managed memory?
And, related to that, what would happen if Foo had a reference as a member field? What if it pointed at managed memory? What if it only ever pointed at other objects allocated in unmanaged memory?
Finally, if this is impossible: Why?
Update: Here are the "missing pieces" so far:
#1: How to convert an IntPtr to an object reference? It might be possible though unverifiable IL (see comments). So far I've had no luck with this. The framework seems to be extremely careful to prevent this from happening.
(It would also be nice to be able to get the size and layout information for non-blittable managed types at runtime. Again, the framework tries to make this impossible.)
#2: Assuming problem one can be solved - what does the GC do when it encounters an object reference that points outside of the GC heap? Does it crash? Anton Tykhyy, in his answer, guesses that it will. Given how careful the framework is to prevent #1, it does seem likely. Something that confirms this would be nice.
(Alternatively the object reference could point to pinned memory inside the GC heap. Would that make a difference?)
Based on this, I'm inclined to think that this idea for a hack is impossible - or at least not worth the effort. But I'd be interested to get an answer that goes into the technical details of #1 or #2 or both.
I have been experimenting creating classes in unmanaged memory. It is possible but has a problem I am currently unable to solve - you can't assign objects to reference-type fields -see edit at the bottom-, so you can have only structure fields in your custom class.
This is evil:
using System;
using System.Reflection.Emit;
using System.Runtime.InteropServices;
public class Voodoo<T> where T : class
{
static readonly IntPtr tptr;
static readonly int tsize;
static readonly byte[] zero;
public static T NewInUnmanagedMemory()
{
IntPtr handle = Marshal.AllocHGlobal(tsize);
Marshal.Copy(zero, 0, handle, tsize);
IntPtr ptr = handle+4;
Marshal.WriteIntPtr(ptr, tptr);
return GetO(ptr);
}
public static void FreeUnmanagedInstance(T obj)
{
IntPtr ptr = GetPtr(obj);
IntPtr handle = ptr-4;
Marshal.FreeHGlobal(handle);
}
delegate T GetO_d(IntPtr ptr);
static readonly GetO_d GetO;
delegate IntPtr GetPtr_d(T obj);
static readonly GetPtr_d GetPtr;
static Voodoo()
{
Type t = typeof(T);
tptr = t.TypeHandle.Value;
tsize = Marshal.ReadInt32(tptr, 4);
zero = new byte[tsize];
DynamicMethod m = new DynamicMethod("GetO", typeof(T), new[]{typeof(IntPtr)}, typeof(Voodoo<T>), true);
var il = m.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ret);
GetO = m.CreateDelegate(typeof(GetO_d)) as GetO_d;
m = new DynamicMethod("GetPtr", typeof(IntPtr), new[]{typeof(T)}, typeof(Voodoo<T>), true);
il = m.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ret);
GetPtr = m.CreateDelegate(typeof(GetPtr_d)) as GetPtr_d;
}
}
If you care about memory leak, you should always call FreeUnmanagedInstance when you are done with your class.
If you want more complex solution, you can try this:
using System;
using System.Reflection.Emit;
using System.Runtime.InteropServices;
public class ObjectHandle<T> : IDisposable where T : class
{
bool freed;
readonly IntPtr handle;
readonly T value;
readonly IntPtr tptr;
public ObjectHandle() : this(typeof(T))
{
}
public ObjectHandle(Type t)
{
tptr = t.TypeHandle.Value;
int size = Marshal.ReadInt32(tptr, 4);//base instance size
handle = Marshal.AllocHGlobal(size);
byte[] zero = new byte[size];
Marshal.Copy(zero, 0, handle, size);//zero memory
IntPtr ptr = handle+4;
Marshal.WriteIntPtr(ptr, tptr);//write type ptr
value = GetO(ptr);//convert to reference
}
public T Value{
get{
return value;
}
}
public bool Valid{
get{
return Marshal.ReadIntPtr(handle, 4) == tptr;
}
}
public void Dispose()
{
if(!freed)
{
Marshal.FreeHGlobal(handle);
freed = true;
GC.SuppressFinalize(this);
}
}
~ObjectHandle()
{
Dispose();
}
delegate T GetO_d(IntPtr ptr);
static readonly GetO_d GetO;
static ObjectHandle()
{
DynamicMethod m = new DynamicMethod("GetO", typeof(T), new[]{typeof(IntPtr)}, typeof(ObjectHandle<T>), true);
var il = m.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ret);
GetO = m.CreateDelegate(typeof(GetO_d)) as GetO_d;
}
}
/*Usage*/
using(var handle = new ObjectHandle<MyClass>())
{
//do some work
}
I hope it will help you on your path.
Edit: Found a solution to reference-type fields:
class MyClass
{
private IntPtr a_ptr;
public object a{
get{
return Voodoo<object>.GetO(a_ptr);
}
set{
a_ptr = Voodoo<object>.GetPtr(value);
}
}
public int b;
public int c;
}
Edit: Even better solution. Just use ObjectContainer<object> instead of object and so on.
public struct ObjectContainer<T> where T : class
{
private readonly T val;
public ObjectContainer(T obj)
{
val = obj;
}
public T Value{
get{
return val;
}
}
public static implicit operator T(ObjectContainer<T> #ref)
{
return #ref.val;
}
public static implicit operator ObjectContainer<T>(T obj)
{
return new ObjectContainer<T>(obj);
}
public override string ToString()
{
return val.ToString();
}
public override int GetHashCode()
{
return val.GetHashCode();
}
public override bool Equals(object obj)
{
return val.Equals(obj);
}
}
"I want to implement a custom allocator for C#"
GC is at the core of the CLR. Only Microsoft (or the Mono team in case of Mono) can replace it, at a great cost in development effort. GC being at the core of the CLR, messing around with the GC or the managed heap will crash the CLR — quickly if you're very-very lucky.
What does the garbage collector do (implementation-specific, if need be) when faced with a reference that points outside of managed memory?
It crashes in an implementation-specific way ;)
Purely C# Approach
So, there are a few options. The easiest is to use new/delete in an unsafe context for structs. The second is to use built-in Marshalling services to deal with unmanaged memory (code for this is visible below). However, both of these deal with structs (though I think the latter method is very close to what you want). My code has a limitation in that you must stick to structures throughout and use IntPtrs for references (using ChunkAllocator.ConvertPointerToStructure to get the data and ChunkAllocator.StoreStructure to store the changed data). This is obviously cumbersome, so you'd better really want the performance if you use my approach. However, if you are dealing with only value-types, this approach is sufficient.
Detour: Classes in the CLR
Classes have a 8 byte "prefix" in their allocated memory. Four bytes are for the sync index for multithreading, and four bytes are for identifying their type (basically, virtual method table and run-time reflection). This makes it hard to deal with unamanaged memory since these are CLR specific and since the sync index can change during run-time. See here for details on run-time object creation and here for an overview of memory layout for a reference type. Also check out CLR via C# for a more in-depth explanation.
A Caveat
As usual, things are rarely so simple as yes/no. The real complexity of reference types has to do with how the garbage collector compacts allocated memory during a garbage collection. If you can somehow ensure that a garbage collection doesn't happen or that it won't affect the data in question (see the fixed keyword) then you can turn an arbitrary pointer into an object reference (just offset the pointer by 8 bytes, then interpret that data as a struct with the same fields and memory layout; perhaps use StructLayoutAttribute to be sure). I would experiment with non-virtual methods to see if they work; they should (especially if you put them on the struct) but virtual methods are no-go due to the virtual method table that you'd have to discard.
One Does Not Simply Walk Into Mordor
Simply put, this means that managed reference types (classes) cannot be allocated in unmanaged memory. You could use managed reference types in C++, but those would be subject to garbage collection... and the process and code is more painful than the struct-based approach. Where does that leave us? Back where we started, of course.
There is a Secret Way
We could brave Shelob's Lair memory allocation ourselves. Unfortunately, this is where our paths must part, because I am not that knowledgeable about it. I will provide you with a link or two - perhaps three or four in actuality. This is rather complicated and begs the question: Are there other optimizations you could try? Cache coherency and superior algorithms is one approach, as is judicious application of P/Invoke for performance-critical code. You could also apply the aforementioned structures-only memory allocation for key methods/classes.
Good luck, and let us know if you find a superior alternative.
Appendix: Source Code
ChunkAllocator.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
namespace MemAllocLib
{
public sealed class ChunkAllocator : IDisposable
{
IntPtr m_chunkStart;
int m_offset;//offset from already allocated memory
readonly int m_size;
public ChunkAllocator(int memorySize = 1024)
{
if (memorySize < 1)
throw new ArgumentOutOfRangeException("memorySize must be positive");
m_size = memorySize;
m_chunkStart = Marshal.AllocHGlobal(memorySize);
}
~ChunkAllocator()
{
Dispose();
}
public IntPtr Allocate<T>() where T : struct
{
int reqBytes = Marshal.SizeOf(typeof(T));//not highly performant
return Allocate<T>(reqBytes);
}
public IntPtr Allocate<T>(int reqBytes) where T : struct
{
if (m_chunkStart == IntPtr.Zero)
throw new ObjectDisposedException("ChunkAllocator");
if (m_offset + reqBytes > m_size)
throw new OutOfMemoryException("Too many bytes allocated: " + reqBytes + " needed, but only " + (m_size - m_offset) + " bytes available");
T created = default(T);
Marshal.StructureToPtr(created, m_chunkStart + m_offset, false);
m_offset += reqBytes;
return m_chunkStart + (m_offset - reqBytes);
}
public void Dispose()
{
if (m_chunkStart != IntPtr.Zero)
{
Marshal.FreeHGlobal(m_chunkStart);
m_offset = 0;
m_chunkStart = IntPtr.Zero;
}
}
public void ReleaseAllMemory()
{
m_offset = 0;
}
public int AllocatedMemory
{
get { return m_offset; }
}
public int AvailableMemory
{
get { return m_size - m_offset; }
}
public int TotalMemory
{
get { return m_size; }
}
public static T ConvertPointerToStruct<T>(IntPtr ptr) where T : struct
{
return (T)Marshal.PtrToStructure(ptr, typeof(T));
}
public static void StoreStructure<T>(IntPtr ptr, T data) where T : struct
{
Marshal.StructureToPtr(data, ptr, false);
}
}
}
Program.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace MemoryAllocation
{
class Program
{
static void Main(string[] args)
{
using (MemAllocLib.ChunkAllocator chunk = new MemAllocLib.ChunkAllocator())
{
Console.WriteLine(">> Simple data test");
SimpleDataTest(chunk);
Console.WriteLine();
Console.WriteLine(">> Complex data test");
ComplexDataTest(chunk);
}
Console.ReadLine();
}
private static void SimpleDataTest(MemAllocLib.ChunkAllocator chunk)
{
IntPtr ptr = chunk.Allocate<System.Int32>();
Console.WriteLine(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Int32>(ptr));
System.Diagnostics.Debug.Assert(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Int32>(ptr) == 0, "Data not initialized properly");
System.Diagnostics.Debug.Assert(chunk.AllocatedMemory == sizeof(Int32), "Data not allocated properly");
int data = MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Int32>(ptr);
data = 10;
MemAllocLib.ChunkAllocator.StoreStructure(ptr, data);
Console.WriteLine(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Int32>(ptr));
System.Diagnostics.Debug.Assert(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Int32>(ptr) == 10, "Data not set properly");
Console.WriteLine("All tests passed");
}
private static void ComplexDataTest(MemAllocLib.ChunkAllocator chunk)
{
IntPtr ptr = chunk.Allocate<Person>();
Console.WriteLine(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Person>(ptr));
System.Diagnostics.Debug.Assert(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Person>(ptr).Age == 0, "Data age not initialized properly");
System.Diagnostics.Debug.Assert(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Person>(ptr).Name == null, "Data name not initialized properly");
System.Diagnostics.Debug.Assert(chunk.AllocatedMemory == System.Runtime.InteropServices.Marshal.SizeOf(typeof(Person)) + sizeof(Int32), "Data not allocated properly");
Person data = MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Person>(ptr);
data.Name = "Bob";
data.Age = 20;
MemAllocLib.ChunkAllocator.StoreStructure(ptr, data);
Console.WriteLine(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Person>(ptr));
System.Diagnostics.Debug.Assert(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Person>(ptr).Age == 20, "Data age not set properly");
System.Diagnostics.Debug.Assert(MemAllocLib.ChunkAllocator.ConvertPointerToStruct<Person>(ptr).Name == "Bob", "Data name not set properly");
Console.WriteLine("All tests passed");
}
struct Person
{
public string Name;
public int Age;
public Person(string name, int age)
{
Name = name;
Age = age;
}
public override string ToString()
{
if (string.IsNullOrWhiteSpace(Name))
return "Age is " + Age;
return Name + " is " + Age + " years old";
}
}
}
}
You can write code in C++ and call it from .NET using P/Invoke or you can you can write code in managed C++ that gives you full access to the native API from inside a .NET language. However, on the managed side you can only work with managed types so you will have to encapsulate your unmanaged objects.
To give a simple example: Marshal.AllocHGlobal allows you to allocate memory on the Windows heap. The handle returned is not of much use in .NET but can be required when calling a native Windows API requiring a buffer.
This is not possible.
However you can use a managed struct and create a pointer of this struct type. This pointer can point anywhere (including to unmanaged memory).
The question is, why would you want to have a class in unmanaged memory? You wouldn't get GC features anyway. You can just use a pointer-to-struct.
Nothing like that is possible. You can access managed memory in unsafe context, but said memory is still managed and subject to GC.
Why?
Simplicity and security.
But now that I think about it, I think you can mix managed and unmanaged with C++/CLI. But I'm not sure about that, because I never worked with C++/CLI.
I don't know a way to hold a C# class instance in the unmanaged heap, not even in C++/CLI.
It's possible to design a value-type allocator entirely within .net, without using any unmanaged code, which can allocate and free an arbitrary number of value-type instances without any significant GC pressure. The trick is to create a relatively small number of arrays (possibly one for each type) to hold the instances, and then pass around "instance reference" structs which hold the array indices of the index in question.
Suppose, for example, that I want to have a "creature" class which holds XYZ positions (float), XYZ velocity (also float), roll/pitch/yaw (ditto), damage (float), and kind (enumeration). An interface "ICreatureReference" would define getters and setters for all those properties. A typical implementation would be a struct CreatureReference with a single private field int _index, and property accessors like:
float Position {
get {return Creatures[_index].Position;}
set {Creatures[_index].Position = value;}
};
The system would keep a list of which array slots are used and vacant (it could, if desired, use one of the fields within Creatures to form a linked list of vacant slots). The CreatureReference.Create method would allocate an item from the vacant-items list; the Dispose method of a CreatureReference instance would add its array slot to the vacant-items list.
This approach ends up requiring an annoying amount of boilerplate code, but it can be reasonably efficient and avoid GC pressure. The biggest problems with are probably that (1) it makes structs behave more like reference types than structs, and (2) it requires absolute discipline with calling IDispose, since non-disposed array slots will never get reclaimed. Another irksome quirk is that one will be unable to use property setters for read-only values of type CreatureReference, even though the property setters would not try to mutate any fields of the CreatureReference instance to which they are applied. Using an interface ICreatureReference may avoid this difficulty, but one must be careful to only declare storage locations of generic types constrained to ICreatureReference, rather than declaring storage locations of ICreatureReference.
Related
I apologize in advance. My domain is mostly C (and C++). I'm trying to write something similar in C#. Let me explain with code.
In C++, I can use large static arrays that are processed during compile-time and stored in a read-only section of the PE file. For instance:
typedef struct _MY_ASSOC{
const char* name;
unsigned int value;
}MY_ASSOC, *LPMY_ASSOC;
bool GetValueForName(const char* pName, unsigned int* pnOutValue = nullptr)
{
bool bResult = false;
unsigned int nValue = 0;
static const MY_ASSOC all_assoc[] = {
{"name1", 123},
{"name2", 213},
{"name3", 1433},
//... more to follow
{"nameN", 12837},
};
for(size_t i = 0; i < _countof(all_assoc); i++)
{
if(strcmp(all_assoc[i].name, pName) == 0)
{
nValue = all_assoc[i].value;
bResult = true;
break;
}
}
if(pnOutValue)
*pnOutValue = nValue;
return bResult;
}
In the example above, the initialization of static const MY_ASSOC all_assoc is never called at run-time. It is entirely processed during the compile-time.
Now if I write something similar in C#:
public struct NameValue
{
public string name;
public uint value;
}
private static readonly NameValue[] g_arrNV_Assoc = new NameValue[] {
new NameValue() { name = "name1", value = 123 },
new NameValue() { name = "name2", value = 213 },
new NameValue() { name = "name3", value = 1433 },
// ... more to follow
new NameValue() { name = "nameN", value = 12837 },
};
public static bool GetValueForName(string name, out uint nOutValue)
{
foreach (NameValue nv in g_arrNV_Assoc)
{
if (name == nv.name)
{
nOutValue = nv.value;
return true;
}
}
nOutValue = 0;
return false;
}
The line private static readonly NameValue[] g_arrNV_Assoc has to be called once during the host class initialization, and it is done for every single element in that array!
So my question -- can I somehow optimize it so that the data stored in g_arrNV_Assoc array is stored in the PE section and not initialized at run-time?
PS. I hope I'm clear for the .NET folks with my terminology.
Indeed the terminology is sufficient enough, large static array is fine.
There is nothing you can really do to make it more efficient out of the box.
It will load initially once (at different times depending on which version of .net and if you have a static constructor). However, it will load before you call it.
Even if you created it empty with just the predetermined size, the CLR is still going to initialize each element to default, then you would have to buffer copy over your data somehow which in turn will have to be loaded from file.
The question are though
How much overhead does loading the default static array of struct actually have compared to what you are doing in C
Does it matter when in the lifecycle of the application when its loaded
And if this is way too much over-head (which i have already assumed you have determined), what other options are possibly available outside the box?
You could possibly pre-allocate a chunk of unmanaged memory, then read and copy the bytes in from somewhere, then inturn access using pointers.
You could also create this in a standard Dll, Pinvoke just like an other un-managed DLL. However i'm not really sure you will get much of a free-lunch here anyway, as there is overhead to marshal these sorts of calls to load your dll.
If your question is only academic, these are really your only options. However if this is actually a performance problem you have, you will need to try and benchmark this for micro-optimization and try to figure out what is suitable to you.
Anyway, i don't profess to know everything, maybe someone else has a better idea or more information. Good luck
I am trying to create a struct that contains a function and an object to be handled by that function. For example:
public delegate void MyFunc(object o);
public struct MyData
{
public object o;
public MyFunc func;
public MyData(object o, MyFunc func)
{
this.o = o;
this.func = func;
}
public HandleData()
{
func(o);
}
}
The purpose of this struct is to use any function to handle any data.
I wrap this struct into a IntPtr data type and send to another to handle this struct
private void PrepareData(object o, MyFunc func)
{
MyData md = new MyData(o, func);
int size = Marshal.SizeOf(md);
IntPtr wParam = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(md, wParam, false);
DoJob(wParam);
}
private void DoJob(IntPtr wParam)
{
if (wParam != IntPtr.Zero)
{
Type type = typeof(MyData);
MyData md = (MyData)Marshal.PtrToStructure(p,type);
md.HandleData();
}
}
Sometimes, I receive error like:
1) "[System.RuntimeType] = {Name = Cannot evaluate expression because the code of the current method is optimized. FullName = Cannot evaluate expression because the code of the current method is optimized.}"
2) in HanldeData function, the func instance variable has been Garbage Collected, and is not able to work properly.
like:
{Method = Cannot evaluate expression because a thread is stopped at a point where garbage collection is impossible, possibly because the code is optimized.}
Managed Debugging Assistant 'CallbackOnCollectedDelegate' has detected a problem
Note: I haven't check the "optimise code" in the property of the project.
Without a complete code example and a clear description of what you are actually trying to achieve here, it's impossible to know for sure what the best answer. That said, frankly, the whole scheme seems nuts to me. IntPtr? Seriously?
I don't see anything in your question that describes a problem that can't be accomplished more easily simply by wrapping the delegate and object in a new delegate object.
E.g.:
private void PrepareData(object o, MyFunc func)
{
DoJob(() => func(o));
}
private void DoJob(Action wParam)
{
if (wParam != null)
{
wParam();
}
}
Note that in your original code, if the only references left to your object and delegate are in the unmanaged block you allocate, they may in fact be GC'ed, as they would then be unreachable via any managed reference (which is the only thing the GC cares about).
Note also that even if the objects are still reachable via a managed reference, that the GC may move the objects in memory (e.g. to compact the heap), rendering the values you've copied into your unmanaged block of memory useless.
If you stick to the use of managed code and objects in your program, you will avoid these problems.
(I also note that your code example doesn't even seem valid, as your DoJob() parameter name is wParam, but the variable you check and marshal back to a managed struct is named p).
I am not sure that the Unmanaged Code will be GC'ed. However, I follow the tutorial: link
and what I am missing in my code is just to forget free the allocated memory for the next use.
private void DoJob(IntPtr wParam)
{
if (wParam != IntPtr.Zero)
{
Type type = typeof(MyData);
MyData md = (MyData)Marshal.PtrToStructure(p,type);
md.HandleData();
// Free the unmanaged representation of MyData struct.
Marshal.FreeHGlobal(wParam);
}
}
It works very well until now after many tests.
Most of the code I have seen deletes the pointer in the finalizer/destructor:
public ref class CPPObjectWrapper
{
private:
CPPObject *_cppObject;
public:
CPPObjectWrapper()
{
_cppObject = new CPPObject();
}
CPPObjectWrapper(IntPtr ^ptr)
{
_cppObject = ptr->ToPointer();
}
~CPPObjectWrapper()
{
delete _cppObject;
_cppObject = 0;
}
!CPPObjectWrapper()
{
if (_cppObject != 0) delete _cppObject;
}
IntPtr^ GetPointer()
{
return gcnew IntPtr(_cppObject);
}
}
My question is what would be standard practice if the library your wrapping does something like this:
void AddObject(CPPObject *cppObject)
{
// adds to a std::list
}
CPPObject* FindObject(/* criteria */)
{
// return reference to std::list item based on criteria
}
If your c# wrapper does this:
void AddObject(CPPObjectWrapper ^cppObject)
{
_internal->addObject(cppObject->GetPointer()->ToPointer());
}
CPPObjectWrapper^ FindObject(/* criteria */)
{
CPPObject *cppObject = _internal->findObject(/* criteria */);
return gcnew CPPObjectWrapper(gcnew IntPtr(cppObjet));
}
You run into a memory issue because your managed object should not delete the pointer because its referenced in another object. The same is true when returning. Would you simply add functionality to tell your managed wrapper not to free the memory when ownership is transferred?
A classic situation when dealing with mixed mode projects, and your suggestion is OK!
It would make sense to have a bool in the constructor that tells it not to destroy the pointer if the same object is used in another non-wrapped object. The ideal case is that every object was wrapped, and the destruction would be done by the CLR.
You can make a generic base class out of this (using the code you already have there), setting the bool by default by the subclass. You are guaranteed to have this functionality many times over. Another tip is to have a virtual OnFinalize() method that is called from the CLR destructor (!) that can do special operations in the subclass, like calling some special free function provided by the native library.
I have 2 interop data structure as private member in a class,
public class RunInterop
{
private AlphaShapeCg _alphaHandler;
private DoubleCgList alphaLevels;
private FaceCgList faceCgList;
public RunInterop()
{
faceCgList =new FaceCgList();
alphaLevels = new DoubleCgList();
Interop_Init(ref _alphaHandler, ref faceCgList, ref alphaLevels);
Interop_Run(ref _alphaHandler);
}
}
The problem now, is that I will get an System.AccessViolationException at Interop_Run line.
However, if I rewrite my code in the following manner:
public class RunInterop
{
private AlphaShapeCg _alphaHandler;
public RunInterop()
{
var faceCgList =new FaceCgList();
var alphaLevels = new DoubleCgList();
Interop_Init(ref _alphaHandler, ref faceCgList, ref alphaLevels);
Interop_Run(ref _alphaHandler);
}
}
Then I wouldn't have any problem. Any idea why this is the case?
Edit: What is really puzzling is that, why, if I declare both the faceCgList and alphaLevels as local variables, the problem would just go away?
What happens in Interop_Init and Interop_Run? You are passing the two members alphaLevels and faceCgList into Interop_Init - maybe it is keeping some reference to them that are used again when calling Interop_Run, at which point it might appear to be accessing the private member of a different class?
Edit: just an idea :)
public class RunInterop
{
private AlphaShapeCg _alphaHandler;
private DoubleCgList _alphaLevels;
private FaceCgList _faceCgList;
public RunInterop()
{
AlphaShapeCg faceCgList = new FaceCgList();
DoubleCgList alphaLevels = new DoubleCgList();
Interop_Init(ref _alphaHandler, ref faceCgList, ref alphaLevels);
Interop_Run(ref _alphaHandler);
_alphaLevels = alphaLevels;
_faceCgList = faceCgList;
}
}
Edit: I found this link explaining how the managed / unmanaged marshaling works in Mono - I'm struggling to find a similar informative article for Microsoft's dotNET Framework, but my guess would be that it should work in a similar way. Here's one quote from the article:
Additionally, if (a) the structure is located on the stack, and (b)
the structure contains only blittable types, then if you pass a
structure to an unmanaged function by-reference, the structure will be
passed directly to the unmanaged function, without an intermediate
unmanaged memory copy.
The following code compile without errors. Basically, the C#2005 Console application calls VC++2005 class library which in turn calls native VC++6 code. I get the following error when I run the C#2005 application:
"Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt."
What is the cause of this error? And how to go about correcting it?
Edit1: It crashes at the line StdStringWrapper ssw = w.GetNext();
Edit2: I followed the advice of Naveen and used an integer index instead of iterators and there is no more errors now. A big thanks to all who commented as well!
Code Written in C#2005 as Console Application:
class Program
{
static void Main(string[] args)
{
Class1 test= new Class1();
test.PerformAction();
test.PerformAction();
test.PerformAction();
test.PerformAction();
}
}
Code Written in VC++2005 as Class Library:
public ref class Class1
{
public:
void PerformAction();
};
void Class1::PerformAction()
{
DoSomethingClass d;
StdStringContainer w;
d.PerformAction(w);
for(int i=0; i<w.GetSize(); i++)
{
StdStringWrapper ssw = w.GetNext();
std::cout << ssw.CStr() << std::endl;
}
}
Code Written in VC++6 as Dynamic Link Library:
#ifdef NATIVECODE_EXPORTS
#define NATIVECODE_API __declspec(dllexport)
#else
#define NATIVECODE_API __declspec(dllimport)
#endif
class NATIVECODE_API StdStringWrapper
{
private:
std::string _s;
public:
StdStringWrapper();
StdStringWrapper(const char *s);
void Append(const char *s);
const char* CStr() const;
};
StdStringWrapper::StdStringWrapper()
{
}
StdStringWrapper::StdStringWrapper(const char *s)
{
_s.append(s);
}
void StdStringWrapper::Append(const char *s)
{
_s.append(s);
}
const char* StdStringWrapper::CStr() const
{
return _s.c_str();
}
//
class NATIVECODE_API StdStringContainer
{
private:
std::vector<StdStringWrapper> _items;
std::vector<StdStringWrapper>::iterator _it;
public:
void Add(const StdStringWrapper& item);
int GetSize() const;
StdStringWrapper& GetNext();
};
void StdStringContainer::Add(const StdStringWrapper &item)
{
_items.insert(_items.end(),item);
}
int StdStringContainer::GetSize() const
{
return _items.size();
}
StdStringWrapper& StdStringContainer::GetNext()
{
std::vector<StdStringWrapper>::iterator it = _it;
_it++;
return *it;
}
//
class NATIVECODE_API DoSomethingClass
{
public:
void PerformAction(StdStringContainer &s);
};
void DoSomethingClass::PerformAction(StdStringContainer &s)
{
StdStringWrapper w1;
w1.Append("This is string one");
s.Add(w1);
StdStringWrapper w2;
w2.Append("This is string two");
s.Add(w2);
}
The member _it in StdStringContainer is never initialized to point into the _items vector. This means it's an invalid iterator. When you assign _it to it in GetNext(), you've given it the invalid, uninitialized value that existed in _it. You then increment the uninitialized _it via _it++, which is what's triggering your fault.
As Stroustrup says in 19.2, an uninitialized iterator is an invalid iterator. This means that your uninitialized _it is invalid and that operations performed with it are undefined, and likely to cause dramatic failure.
Your problem is deeper, however. Iterators have a fundamentally different lifetime from the containers that they enumerate. There aren't really any "good" ways to do what you're trying to do with a single iterator member like this unless the container is immutable and initialized in the constructor.
If you can't expose the std:: namespace names, have you considered aliasing them via typedef's, e.g.? What about your organization or project makes it impossible to expose the template classes?
The main problem from my point of view is you are storing an iterator to a vector in your stdStringContainer class. Remember that whenever vector resizes all the existing iterators are invalidated. So whenever you do insert operation into the vector it may be possible that it resizes and your existing iterator becomes invalid. If you try to to dereference it in GetNext() then it will access invalid memory location. For checking whether this really the case try to reserve the initial vector size to some relatively big number so that the resizing doesn't happen. You can reserve the size using reserve() method, in which case it is guaranteed that the capacity() of the vector is greater than or equal to the reserved value.
Sounds like you have a memory leak. I would suggest looking anywhere where there is pointer arithmetic, writing to memory, or array usage. Check for the bounds conditions in the array accessing.
Another issue: The leak many not even be in your code. If this is the case you'll have to exclude the library from your project.
My guess is, that you have the crash because std::string and std::vector in the interface between two C++ modules were compiled with different compilers and runtime libraries.
The memory layout of vector and string maybe changed between VC6 and 2005.
When the 2005 DLL allocates objects of type StdStringContainer and StdStringWrapper, it does so based on the declarations of string and vector in the 2005 headers.
When member functions are called on these objects (which have been compiled with the VC6 compiler and libraries), they assume a different memory layout and fail with access violations.