I have Dll that reads file from disk and returns it's content:
mydll.h:
extern "C" __declspec(dllexport) void InitFile3dPoints(wchar_t* i_file);
extern "C" __declspec(dllexport) int GetNumPointsForSurface(int i_surf_index);
extern "C" __declspec(dllexport) void GetPointsForSurface
(double* o_result, int i_resultLength, int i_surf_index);
mydll.cpp:
File3dPoints file_3dPoints;
void InitFile3dPoints(wchar_t* i_file)
{ file_3dPoints = readFile3dObjectFromDisk(i_file) }
int GetNumPointsForSurface(int i_surf_index)
{ return file_3dPoints[i_surf_index].getNumPoints(); }
void GetPointsForSurface(double* o_result, int i_resultLength, int i_surf_index);
{
const int num_points = file_3dPoints[i_surf_index].getNumPoints();
if (num_points < i_resultLength)
return;
for (int i = 0; i < num_points; ++i)
o_result[i] = file_3dPoints[i_surf_index].getPoint(i);
}
client.cs:
IntPtr inst = LoadLibrary("mydll.dll");
InitFile3dPoints(filename);
for (int i = 0; i < n; ++i)
{
int num_points_for_surface = GetNumPointsForSurface(i);
double[] points = new double[num_points_for_surface];
GetPointsForSurface(points, points.Length, i);
// some code
}
FreeLibrary(inst);
My dll is not thread-safe. One thread can call InitFile3dPoints. And before calling GetPointsForSurface another thread can call InitFile3dPoints.
Could you plase advise me how do make it thread-safe? Creating mutex for accessing file_3dPoints will not solve the problem, I need every thread that comes in mydll.cpp to have its copy of file_3dPoints.
Thanks
There are many options to do that.
First and most important do not use global variables, they're a nightmare (for this and other reasons). Let's start to change InitFile3dPoints signature to allocate the required memory and to return it to the caller (so the address can be used as "handle"):
File3dPoints* InitFile3dPoints(const wchar_t* i_file)
{
return readFile3dObjectFromDisk(i_file);
}
Please note that readFile3dObjectFromDisk must return a heap allocated object of type File3dPoints.
Then change each function to accept that pointer:
int GetNumPointsForSurface(const File3dPoints* data, int i_surf_index)
{
return *data[i_surf_index].getNumPoints();
}
File3dPoints* can be marshaled in C# with IntPtr:
IntPtr inst = LoadLibrary("mydll.dll");
IntPtr data = InitFile3dPoints(filename);
Finally do not forget to add a DisposeFile3dPoints function where you deallocate data:
void DisposeFile3dPoints(File3dPoints* data)
{
if (data != NULL)
delete data;
}
In general to make a DLL "thread-safe" (according to the context of your question) you should make each function self-contained (all the data it needs comes from its parameters, it doesn't have any local static variable nor global ones). Please note that this doesn't make it really thread-safe in any wider sense (if, for example, you'll expose a write function to that data then you'll still need to protect access but it can be done more easily in the C# side).
You can do even better wrapping all these functions inside a C# class, users won't even see that:
public class File3D : IDisposable
{
public File3D(string path)
{
// Initialize. Call InitFile3dPoints
}
~File3D()
{
// Call Dispose(false)
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
private IntPtr _data;
private void Dispose(bool disposing)
{
// Unmanaged resource, ignore disposing parameter
// and call DisposeFile3dPoints
}
}
Protect access to global variables using a mutex. Better yet, don't use global variables.
Reorganize your code so that InitFile3dPoint() returns a File3dPoints that you then pass to the other functions as context. The typical approach is to return a pointer to a dynamically allocated instance. You then have to take care of proper ownership and deallocation management, but you make the DLL itself thread-safe even without any mutex.
Related
I have a c++ dll from a third party, I can't modify.
I've made a c++ wrapper to call the functions inside that c++ dll.
That wrapper is called from a C# class with IJW.
It works well with natives types, but I ran into a struct array problem.
This is the c++ wrapper class :
int Helper::GetCameras(cameraStruct * cameraArray, size_t * size)
{
return originalC++Dll_getCameraList(cameraArray, size);
}
And the C# call :
var ret = m_Helper.GetCameras(pointer, size);
Do I need to redefine the "cameraStruct" in c# like this ?
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public class aivusdk_cameraInfo
{
public int blabla;
}
To use it like with a pointer in the C# function, or is it another way of doing it (mashalling), or am I completly wrong ?
Forgive my bad english and thanks for your response.
The primary reason to use IJW is to avoid all the hassle of trying to define parallel structures in your C# code using StructLayout and calling all that stuff in the Marshal class. If you are going to go to that trouble, you may as well do everything using P/Invoke.
One approach would be to define, in your C# assembly, an interface that will be implemented by the c++/clr assembly and a managed class containing properties corresponding to the interesting fields of the cameraStruct structure.
public interface IHelper
{
public void GetCameras(IList<Camera> cameras);
public Picture TakePicture(Camera camera);
public void LoseCamera(Camera camera);
}
public class Camera
{
public string Name { get; set; }
public int BlaBla { get; set; }
}
The Helper class in your c++/clr assembly would implement the IHelper interface. The implementation of the GetCameras method would call originalDLL_getCameraList with a locally allocated array of cameraStruct. It would then iterate over the returned cameras and gcnew a Camera object for each one. It would copy the information from the cameraStruct to the Camera object and then add the Camera object to the cameras list. The locally allocated cameraStruct array can then be freed.
One issue is that the c++/clr project must have a build dependency (reference) to the C# project. The C# project cannot have a reference to the c++/clr project. For your C# code to get a Helper object, you could dynamically load the c++/clr assembly from the C# assembly and use reflection to create a Helper object that you cast to an IHelper.
You can turn that relationship around, but that means defining more in c++/clr and less in C#. (For example, you would need to define the Camera class in the c++/clr assembly.) I prefer to define things using C# when practical.
You need to allocate unmanaged memory for the array that you can pass to the API.
This is how I would do that:
// the input here is a managed C# array
public IntPtr AllocateNativeArray(CameraInfo[] myArray)
{
if (myArray == null || myArray.Count == 0)
return IntPtr.Zero;
// building native structures
var nativeArray = new Aivusdk_CameraInfo[myArray.Length];
for (int i = 0; i < myArray.Length; i++)
{
// TODO: fill properties
}
// allocating unmanaged memory and marshaling elements
int elementSize = Marshal.SizeOf(typeof(Aivusdk_CameraInfo));
IntPtr result = Marshal.AllocHGlobal(nativeArray.Length * elementSize);
for (int i = 0; i < nativeArray.Length; i++)
{
Marshal.StructureToPtr(nativeArray[i], new IntPtr((long)result + i * elementSize), false);
}
return result;
}
After calling the C++ API you might need to free the array (unless the C++ part does that):
private static void FreeNativeArray(IntPtr address, uint length)
{
if (address== IntPtr.Zero)
return;
int elementSize = Marshal.SizeOf(typeof(Aivusdk_CameraInfo));
for (int i = 0; i < count; i++)
{
Marshal.DestroyStructure(new IntPtr((long)address + i * elementSize), typeof(Aivusdk_CameraInfo));
}
Marshal.FreeHGlobal(address);
}
I am currently writing a wrapper for the PhysFS library, and I stumbled across a bit of troubles regarding the marshalling of managed objects. Take for example the PHYSFS_enumerateFilesCallback method, which takes a function pointer and a user-defined pointer as its arguments. How can I pass managed objects to this method? This is what I am currently doing:
// This is the delegate signature
public delegate void EnumFilesCallback(IntPtr data, string origdir, string fname);
// This is the method signature
[DllImport(DLL_NAME, CallingConvention = CallingConvention.Cdecl)]
public static extern void PHYSFS_enumerateFilesCallback(string dir, EnumFilesCallback c, IntPtr d);
Finally, this is what I'm doing to pass an arbitrary object to the method:
// I use the unsafe keyword because the whole Interop class is declared so.
// This code was taken from https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.gchandle(VS.71).aspx
public static void EnumerateFilesCallback(string dir, EnumFilesCallback c, object data)
{
unsafe
{
GCHandle objHandle = GCHandle.Alloc(data);
Interop.PHYSFS_enumerateFilesCallback(dir, c, (IntPtr)objHandle);
objHandle.Free();
}
}
When I run this code:
static void Enum(IntPtr d, string origdir, string fname )
{
System.Runtime.InteropServices.GCHandle handle = (System.Runtime.InteropServices.GCHandle)d;
TestClass c = (TestClass)handle.Target;
Console.WriteLine("{0} {1}", origdir, fname);
}
static void Main(string[] args)
{
PhysFS.Init("");
PhysFS.Mount("D:\\", "/hello", true);
TestClass x = new TestClass() { a = 3, b = 4 }; // This can be any gibberish object
PhysFS.EnumerateFilesCallback("/hello/", Enum, x);
}
The delegate gets called 4 times with legit data, the fifth time it contains garbage data and then it throws an AccessViolationException
I suspect this is because the object gets GCed in between the calls to the delegate. Can anyone shed light on this?
UPDATE: Changing the mounted directory eliminates the rubbish data, yet the exception is still thrown, and still before all the data can be consumed
Have you tried to create the callback and store it as a class static field?
private static EnumFilesCallback callback = new EnumFilesCallback(Enum);
And in your main method:
PhysFS.EnumerateFilesCallback("/hello/", callback, x);
This should probably avoid the GC to collect the local variable holding the delegate object.
Thanks to everyone who invested their time trying to provide an answer! I've finally found the source of the problem and solved it!
The problem was... I am a bit ashamed of it... calling convention.
All the PInvoked methods were declared as cdecl while I forgot to declare the delegates as such, so it created unbalanced stacks and mayhem and whatnot...
no matter what I try. I appear to get garbage results when I marshal the data across! The data after the marshal copy just contains an array of what looks like uninitialized data. Just pure garbage.
Thanks for your help in advance!
C++
typedef uint64_t TDOHandle;
extern "C" DATAACCESSLAYERDLL_API const TDOHandle * __stdcall DB_GetRecords()
{
const Database::TDBRecordVector vec = Database::g_Database.GetRecords();
if (vec.size() > 0)
{
return &vec[0];
}
return nullptr;
}
C#
The declaration
[System.Security.SuppressUnmanagedCodeSecurity()]
[DllImport("DataCore.dll")]
static private extern IntPtr DB_GetRecords();
//The marshalling process
IntPtr ptr_records = DB_GetRecords();
if (ptr_records != null)
{
Byte[] recordHandles = new Byte[DB_GetRecordCount()*sizeof (UInt64)];
Marshal.Copy(ptr_records, recordHandles, 0, recordHandles.Length);
Int64[] int64Array = new Int64[DB_GetRecordCount()];
Buffer.BlockCopy(recordHandles, 0, int64Array, 0, recordHandles.Length);
}
You are returning the address of memory owned by a local variable. When the function returns, the local variable is destroyed. Hence the address you returned is now meaningless.
You need to allocate dynamic memory and return that. For instance, allocate it with CoTaskMemAlloc. Then the consuming C# can deallocate it with a call to Marshal.FreeCoTaskMem.
Or allocate the memory using new, but also export a function from your unamanaged code that can deallocate the memory.
For example:
if (vec.size() > 0)
{
TDOHandle* records = new TDOHandle[vec.size()];
// code to copy content of vec to records
return records;
}
return nullptr;
And then you would export another function that exposed the deallocator:
extern "C" DATAACCESSLAYERDLL_API void __stdcall DB_DeleteRecords(
const TDOHandle * records)
{
if (records)
delete[] record;
}
All that said, it seems that you can obtain the array length before you call the function to populate the array. You do that with DB_GetRecordCount(). In that case you should create an array in your managed code, and pass that to the unmanaged code for it to populate. That side steps all the issues of memory management.
I'll add that there is another way to do it:
public sealed class ULongArrayWithAllocator
{
// Not necessary, default
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
public delegate IntPtr AllocatorDelegate(IntPtr size);
private GCHandle Handle;
private ulong[] allocated { get; set; }
public ulong[] Allocated
{
get
{
// We free the handle the first time the property is
// accessed (we are already C#-side when it is accessed)
if (Handle.IsAllocated)
{
Handle.Free();
}
return allocated;
}
}
// We could/should implement a full IDisposable interface, but
// the point of this class is that you use it when you want
// to let C++ allocate some memory and you want to retrieve it,
// so you'll access LastAllocated and free the handle
~ULongArrayWithAllocator()
{
if (Handle.IsAllocated)
{
Handle.Free();
}
}
// I'm using IntPtr for size because normally
// sizeof(IntPtr) == sizeof(size_t) and vector<>.size()
// returns a size_t
public IntPtr Allocate(IntPtr size)
{
if (allocated != null)
{
throw new NotSupportedException();
}
allocated = new ulong[(long)size];
Handle = GCHandle.Alloc(allocated, GCHandleType.Pinned);
return Handle.AddrOfPinnedObject();
}
}
[DllImport("DataCore.dll", CallingConvention = CallingConvention.StdCall)]
static private extern IntPtr DB_GetRecords(ULongArrayWithAllocator.AllocatorDelegate allocator);
and to use it:
var allocator = new ULongArrayWithAllocator();
DB_GetRecords(allocator.Allocate);
// Here the Handle is freed
ulong[] allocated = allocator.Allocated;
and C++ side
extern "C" DATAACCESSLAYERDLL_API void __stdcall DB_GetRecords(TDOHandle* (__stdcall *allocator)(size_t)) {
...
// This is a ulong[vec.size()] array, that you can
// fill C++-side and can retrieve C#-side
TDOHandle* records = (*allocator)(vec.size());
...
}
or something similar :-) You pass a delegate to the C++ function that can allocate memory C#-side :-) And then C# side you can retrieve the last memory that was allocated. It is important that you don't make more than one allocation C++-side in this way in a single call, because you are saving a single LastAllocated reference, that is "protecting" the allocated memory from the GC (so don't do (*allocator)(vec.size());(*allocator)(vec.size());)
Note that it took me 1 hour to write correctly the calling conventions of the function pointers, so this isn't for the faint of heart :-)
Unmanaged and Managed Memory Regions
I am attempting to execute unmanaged code from a C-library. One of the methods takes a void* as a parameter but under the hood it's cast to a struct of type nc_vlen_t
C struct for nc_vlen_t
/** This is the type of arrays of vlens. */
typedef struct {
size_t len; /**< Length of VL data (in base type units) */
void *p; /**< Pointer to VL data */
} nc_vlen_t;
Executing the method is correct and it works, I am concerned more about the pinning and safe handling of managed and unmanaged memory regions. I want to be as certain as possible that I am not going to cause memory leaks or a SEGFAULT. I wrote a struct that will be marshalled to and from the nc_vlen_t when I execute the C-library method calls.
C# struct
[StructLayout(LayoutKind.Sequential)]
public struct VlenStruct {
public Int32 len;
public IntPtr p; // Data
}
The struct consists of a size_t that indicates the array length and a void * to the data. Inside the library it has attributes that allow it to cast the (void*) to the appropriate numeric types and I've had great success with that so far.
What I want to understand is the best way to handle the memory regions. After reading some articles and other SO questions this is my best guess for how to handle it. I have a class that acts as an arbiter for creating and managing the structs and their memory. I rely on a destructor to free the handle which will unpin the array so that the GC can do it's job.
C# Vlen Helper
public class Vlen {
private GCHandle handle;
private VlenStruct vlen_t;
public Vlen() {
isNull = true;
}
public Vlen(Array t) {
isNull = false;
handle = GCHandle.Alloc(t, GCHandleType.Pinned); // Pin the array
vlen_t.len = t.Length;
vlen_t.p = Marshal.UnsafeAddrOfPinnedArrayElement(t, 0); // Get the pointer for &t[0]
}
~Vlen() {
if(!isNull) {
handle.Free(); // Unpin the array
}
}
public VlenStruct ToStruct() {
VlenStruct retval = new VlenStruct();
retval.len = vlen_t.len;
retval.p = vlen_t.p;
return retval;
}
private bool isNull;
}
C Method Declaration
//int cmethod(const int* typep, void *data)
// cmethod copies the array contents of the vlen struct to a file
// returns 0 after successful write
// returns -1 on fail
[DllImport("somelib.dll", CharSet = CharSet.Ansi, SetLastError = true, ExactSpelling = true, CallingConvention=CallingConvention.Cdecl)]
public static extern Int32 cmethod(ref Int32 typep, ref VlenStruct data);
If I use this class to create the struct is there a possibility that the GC will clean the array before the C-library is called in this situation:
C# Use-case
{
double[] buffer vlenBuffer = new double[] { 0, 12, 4};
Vlen data = new Vlen(vlenBuffer); // The instance now pins buffer
VlenStruct s = data.ToStruct()
Int32 type = VLEN_TYPE;
cmethod(ref type, ref s);
}
Is it possible for the data instance to be cleaned and thereby unpin buffer which could cause unpredictable behavior when executing the external library method?
Yes, you certainly have a problem here. As far as the jitter is concerned, the lifetime of your "data" object ends just before the ToStruct() method returns. Check this answer for the reason why. Which permits the finalizer to run while your unmanaged code is running. Which unpins your array. It would actually take another garbage collection to corrupt the data that the unmanaged code uses. Very rare indeed but not impossible. You are not likely to get an exception either, just random data corruption.
One workaround is to extend the lifetime of the Vlen object beyond the call, like this:
Vlen data = ...
...
cmethod(ref type, ref s);
GC.KeepAlive(data);
Which works but doesn't win any prizes, easy to forget. I would do this differently:
public static void CallHelper<T>(int type, T[] data) {
var hdl = GCHandle.Alloc(data, GCHandleType.Pinned);
try {
var vlen = new nc_vlen();
vlen.len = data.Length;
vlen.data = hdl.AddrOfPinnedObject();
cmethod(ref type, ref vlen);
}
finally {
hdl.Free();
}
}
Usage:
var arr = new int[] { 1, 2, 3 };
CallHelper(42, arr);
Which, beyond avoiding the early collection problem, also keeps the array pinned as short as possible. Do note that ref on the first argument of this function is pretty strange, you would not expect this function to alter the data type.
I am using DllImport to call method in c wrapper library from my own .net class. This method in c dll creates a string variable and returns the pointer of the string.
Something like this;
_declspec(dllexport) int ReturnString()
{
char* retval = (char *) malloc(125);
strcat(retval, "SOMETEXT");
strcat(retval, "SOMETEXT MORE");
return (int)retval;
}
Then i read the string using Marshall.PtrToStringAnsi(ptr). After i get a copy of the string, i simply call another c method HeapDestroy which is in c wrapper library that calls free(ptr).
Here is the question;
Recently while it is working like a charm, I started to get "Attempted to read or write protected memory area" exception. After a deeper analysis, i figured out, i beleive, although i call free method for this pointer, value of the pointer is not cleared, and this fills the heap unattended and makes my iis worker process to throw this exception. By the way, it is an web site project that calls this method in c library.
Would you kindly help me out on this issue?
Sure, here is C# code;
[DllImport("MyCWrapper.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
private extern static int ReturnString();
[DllImport("MyCWrapper.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
private extern static void HeapDestroy(int ptr);
public static string GetString()
{
try
{
int i = ReturnString();
string result = String.Empty;
if (i > 0)
{
IntPtr ptr = new IntPtr(i);
result = Marshal.PtrToStringAnsi(ptr);
HeapDestroy(i);
}
return result;
}
catch (Exception e)
{
return String.Empty;
}
}
What may be the problem is the underlying C code. You are not adding a NULL terminator to the string which strcat relies on (or checking for a NULL return from malloc). It's easy to get corrupted memory in that scenario. You can fix that by doing the following.
retval[0] = '\0';
strcat(retval, "SOMETEXT");
Also part of the problem is that you are playing tricks on the system. It's much better to write it correctly and let the system work on correctly functioning code. The first step is fixing up the native code to properly return the string. One thing you need to consider is that only certain types of memory can be natively freed by the CLR (HGlobal and CoTask allocations). So lets change the function signature to return a char* and use a different allocator.
_declspec(dllexport) char* ReturnString()
{
char* retval = (char *) CoTaskMemAlloc(125);
retval[0] = '\0';
strcat(retval, "SOMETEXT");
strcat(retval, "SOMETEXT MORE");
return retval;
}
Then you can use the following C# signature and free the IntPtr with Marshal.FreeCoTaskMem.
[DllImport("SomeDll.dll")]
public static extern IntPtr ReturnString();
Even better though. When marshalling, if the CLR ever thinks it needs to free memory it will use FreeCoTaskMem to do so. This is typically relevant for string returns. Since you allocated the memory with CoTaskMemAlloc you can save yourself the marshalling + freeing steps and do the following
[DllImport("SomeDll.dll", CharSet=Ansi)]
public static extern String ReturnString();
Freeing memory doesn't clear it, it just frees it up so it can be re-used. Some debug builds will write over the memory for you to make it easier to find problems with values such as 0xBAADFOOD
Callers should allocate memory, never pass back allocated memory:
_declspec(dllexport) int ReturnString(char*buffer, int bufferSize)
{
if (bufferSize < 125) {
return 125;
} else {
strcat(buffer, "SOMETEXT");
strcat(buffer, "SOMETEXT MORE");
return 0;
}
}
Although memory is allocated by the DLL in the same heap as your application, it MAY be using a different memory manager, depending on the library it was linked with. You need to either make sure you're using the same exact library, or add code to release the memory that the DLL allocates, in the DLL code itself.