We Keep Coding

Undefined symbols when trying to use native C++ .so in Mono Pinvoke

Recently I have been trying to get some Point Cloud Library functionality going in my .NET framework application, and considering that there is no completely functional wrapper for PCL for C#, I made my own for a few functions as a test. Something like this:
[DllImport(DllFilePath, CallingConvention = CallingConvention.Cdecl)]
public extern static IntPtr StatisticalOutlierFilter(IntPtr data, int length, int meanK = 50, float mulThresh = 1.0f);
Which calls a function from a C++ library, such as this:
EXPORT VectorXYZ* StatisticalOutlierFilter(VectorXYZ* data, int length, int meanK, float mulThresh) {
auto processedCloud = process.StatisticalOutlierFilter(data, length, meanK, mulThresh);
auto processedVector = convert.ToVectorXYZ(processedCloud);
return processedVector;
}
Where EXPORT is defined such for gcc:
#define EXPORT extern "C" __attribute__ ((visibility ("default")))
And relevant processing function from PCL is implemented such in a class (note that the returned is a boost shared pointer):
PointCloud<PointXYZ>::Ptr Processors::StatisticalOutlierFilter(VectorXYZ* data, int length, int meanK, float mulThresh) {
auto cloud = PrepareCloud(data, length);
PointCloud<PointXYZ>::Ptr cloud_filtered(new PointCloud<PointXYZ>);
StatisticalOutlierRemoval<PointXYZ> sor;
sor.setInputCloud(cloud);
sor.setMeanK(meanK);
sor.setStddevMulThresh(mulThresh);
sor.filter(*cloud_filtered);
return cloud_filtered;
}
This procedure works well with a dll built w/MSVC and running the whole thing on Windows, though the final target is gcc/Linux/Mono, where I get several errors of the following type (this is from mono debug):
'libavpcl_dll.so': '/usr/lib/libavpcl_dll.so: undefined symbol: _ZN3pcl7PCLBaseINS_8PointXYZEE13setInputCloudERKN5boost10shared_ptrIKNS_10PointCloudIS1_EEEE'.
I have investigated quite a bit so far, and have set my CmakeLists.txt to set(CMAKE_CXX_VISIBILITY_PRESET hidden) , therefore, I imagine, only functions I defined as EXPORT should be visible and imported - however, that is not the case, and I get the aforementioned errors. PCL was installed on Windows via vcpkg and on Xubuntu via apt. I am somewhat stumped as to what is the error source, considering the code runs well on windows, and builds without issue on Linux. Thanks.

I've been running into the same issue as you. I solved it by adding each reference library into the CMakeLists.txt file (I was missing the reference files which gave me the similar missing symbol issues).
I'm at the 'I don't know why this worked' stage but I can give you step by step implementation (I'm also trying to use DllImport into .NET on Linux).
Started with this:
https://medium.com/#xaviergeerinck/how-to-bind-c-code-with-dotnet-core-157a121c0aa6
Then added my in-scope files thanks to the main comment here: How to create a shared library with cmake?:
add_library(mylib SHARED
sources/animation.cpp
sources/buffers.cpp
[...]
)
run cmake .
run make -j$(grep -c ^processor /proc/cpuinfo)
copy path to .so file
DllImport path from above to my c# app

Interfacing both Debug and Release DLLs with the same C# code in SWIG?

SWIG lets you specify the DLL to import in C# by using the -dllimport command line argument.
What about importing a DLL whose name depends on whether it is a Debug version or a Release one? This happens with DLLs that follow the Microsoft convention of appending the d suffix to the Debug version, e.g. ucrtbase.dll for the Release version, and ucrtbased.dll for Debug.
If -dllimport allowed to specify a symbolic constant, then the value of such constant could depend on whether DEBUG is defined or not, but that does not seem to be the case.

I think you can do what you want using a static constructor and the SetDllDirectory of suggestion of this this answer.
To test this out I put together this example:
%module foobar
%pragma(csharp) imclasscode=%{
[global::System.Runtime.InteropServices.DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
static extern bool SetDllDirectory(string lpPathName);
static $imclassname() {
if (isDebug)
SetDLLDirectory("path/to/directory/with/debugDll");
else
SetDLLDirectory("path/to/directory/with/regularDll");
}
%}
You have to build it with SWIG like this in order to suppress the default static constructor:
swig3.0 -DSWIG_CSHARP_NO_IMCLASS_STATIC_CONSTRUCTOR -csharp test.i
Quite how you'll distinguish isDebug in real code I'm less clear on - maybe use this to find the current HMODULE and then GetModuleFilename to see if it's your debug build or not.

Why does LoadLibrary fail while DllImportAttribute works?

I'm creating a .NET application for a client that performs I/O with one of their third-party systems. As they regularly change the password of this system, I should retrieve it dynamically by calling a native DLL that they provide in a dedicated directory (not besides my EXE file).
However, I have trouble loading the DLL dynamically using LoadLibraryEx. The weird thing is that I can call the library using the DllImportAttribute.
This is what I have done so far:
According to this SO answer, I use the following code (in a constructor) to try to load the DLL dynamically:
public PasswordProvider(string dllPath)
{
if (!File.Exists(dllPath))
throw new FileNotFoundException($"The DLL \"{dllPath}\" does not exist.");
_dllHandle = NativeMethods.LoadLibraryEx(dllPath, IntPtr.Zero, LoadLibraryFlags.None);
if (_dllHandle == IntPtr.Zero)
throw CreateWin32Exception($"Could not load DLL from \"{dllPath}\".");
var procedureHandle = NativeMethods.GetProcAddress(_dllHandle, GetPasswordEntryPoint);
if (procedureHandle == IntPtr.Zero)
throw CreateWin32Exception("Could not retrieve GetPassword function from DLL.");
_getPassword = Marshal.GetDelegateForFunctionPointer<GetPasswordDelegate>(procedureHandle);
}
When LoadLibraryEx is called, the resulting handle is null, the error code is 126 which usually means that the DLL or one of its dependencies could not be found.
When I call LoadLibraryEx with DoNotResolveDllReferences, then I get a working handle but afterwards, I cannot call GetProcAddress (error code 127) - I suspect that I have to fully load the DLL for this.
When I open the native DLL in Dependencies (which essentially is Dependency Walker for Win10), I can clearly see that one of the statically linked DLLs is missing
However, if I copy the DLL besides my EXE file and use the DllImportAttribute, I can call into the DLL
[DllImport(DllPath, EntryPoint = GetPasswordEntryPoint, CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Unicode)]
private static extern long GetPassword(long systemId, string user, byte[] password);
How is this possible? I thought that the mechanism behind DllImportAttribute uses LoadLibary internally, too. Where does my code differ? Am I missing something obvious?
Just some notes:
I can't just use DllImportAttribute as I cannot specify searching in a dedicated directory this way (the DLL must lie beside my EXE file or in a common Windows location for this to work).
I also tried LoadLibrary instead of LoadLibraryEx but with the same results.
EDIT after Simons comment:
NativeMethods is defined as followed:
private static class NativeMethods
{
[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern IntPtr LoadLibrary(string dllName);
[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern IntPtr LoadLibraryEx(string dllFileName, IntPtr reservedNull, LoadLibraryFlags flags);
[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern IntPtr GetProcAddress(IntPtr moduleHandle, string procedureName);
[DllImport("kernel32.dll")]
public static extern bool FreeLibrary(IntPtr moduleHandle);
}
[Flags]
private enum LoadLibraryFlags : uint
{
None = 0,
DoNotResolveDllReferences = 0x00000001,
LoadIgnoreCodeAuthorizationLevel = 0x00000010,
LoadLibraryAsDatafile = 0x00000002,
LoadLibraryAsDatafileExclusive = 0x00000040,
LoadLibraryAsImageResource = 0x00000020,
LoadLibrarySearchApplicationDir = 0x00000200,
LoadLibrarySearchDefaultDirs = 0x00001000,
LoadLibrarySearchDllLoadDir = 0x00000100,
LoadLibrarySearchSystem32 = 0x00000800,
LoadLibrarySearchUserDirs = 0x00000400,
LoadWithAlteredSearchPath = 0x00000008
}
EDIT after Hans Passant's comment:
The overall goal is the ability to replace / update the native DLL while my application (a Windows Service) is running. I detect a file change and then reload the DLL. I am not quite sure if this is possible with DllImportAttribute without restarting the service.
And I should be more specific on the actual problem: I couldn't load the native DLL using LoadLibraryEx, no matter if it was placed next to my EXE, or in another random folder, or in SysWow64. Why does it work with DllImportAttribute? I'm pretty sure that the missing FastMM subdependency DLL is not present on my system (neither next to the actual DLL, nor in any Windows directory).

It's because the DLL search order path. In windows when application try to load a DLL the underlying system automatically search some path for the DLL ,So let's pretend Windows's DLL search path looks something like this:
A) . <-- current working directory of the executable, highest priority, first check
B) \Windows
C) \Windows\system32
D) \Windows\syswow64 <-- lowest priority, last check
You can read more about the underlying mechanism in this Microsoft documentation.
Search for DLL which your main DLL has dependency to it and find where it store on system, add the directory of it to DLL search path of Windows using AddDllDirectory or SetDllDirectory.
If the dll already loaded into memory by any of running process Windows automatically use it instead of searching, so you can load FastMM DLL into memory using LoadLibrary manually and then try to load the main DLL and it should solve the problem too.

#HansPassant and #David Heffernan are right: I actually tried to load two different versions of the DLL (one of them had the FastMM subdependency, one did not). Thanks for your help and sorry for the inconvenience.

ShSetFolderPath works on win7, doesn't on XP

I'm trying to use ShSetFolderPath function in C#. I work on Win7, I've managed to use ShSetKnownFolderPath and it works fine.
Since this function is unavaible in WinXP, i tried to invoke ShSetFolderPath. Because i'm not familiar with invoking, I've done some searching and found something on some French forum. I don't speak French, but this declaration makes sense (as written in Remarks of function documentation in MSDN library):
[DllImport( "Shell32.dll", CharSet = CharSet.Unicode, EntryPoint = "#232" ) ]
private static extern int SHSetFolderPath( int csidl, IntPtr hToken, uint flags, string path );
I call it like that:
private static int CSIDL_DESKTOP = 0x0000;
public static void SetDesktopPath(string path)
{
int ret;
ret = SHSetFolderPath(CSIDL_DESKTOP, IntPtr.Zero, 0, path);
if (ret != 0)
{
Console.WriteLine(ret);
Console.WriteLine(Marshal.GetExceptionForHR(ret));
}
}
It works in Win7, but in XP function returns -2147024809, which means "Value does not fall within the expected range".
My guess is, it's something wrong with Dll importing. Any idea?

Funny thing.
I've taken another look at CSIDL list. And I've realized I was trying to change some "low-level" reference (i guess) to desktop:
CSIDL_DESKTOP = 0x0000, // <desktop>
While I actually wanted to change just folder location, and i should've use this:
CSIDL_DESKTOPDIRECTORY = 0x0010, // <user name>\Desktop.
And THIS works.
It explains everything. Shame on me.

Nah, that's not it. The error code, converted to hex, is 0x80070057. The 7 indicates a Windows error, 57 is error code 87, ERROR_INVALID_PARAMETER, "The parameter is incorrect".
A couple of possible reasons. First is that entry point #232 isn't actually the entry point for SHSetFolderPath(). You might be calling a different function, it wouldn't know what to do with the argument values you pass. Hard to say, it is an unnamed entry point on XP's version of shell32.dll. Or it could be that XP just isn't happy about you changing the desktop folder path. Not that surprising, there's a heckofalot it has to do to actually implement that, refreshing all Explorer.exe views, rebuilding the desktop contents and whatnot.
Check this thread for possible help.

Dynamically loading a dll in C#

I have a window to use for editing. The editor should load a dll (which I have full control of) in response to the user's selection to know how to display the information visually.
(They're dll's, as a user will not necessarily want or need every single display model, and also allow new ones to be added without messing around with the main project)
They will all simply be stored in a subdirectory (for now anyway)
I'm pretty sure I can enumerate the available dlls but I need to do 2 more things that I'm not sure on
1) Some way to get metadata from\on the dll, so I can build the lists of possible display selections...
2) Load the selected dll, and unload it as necessary
Any suggestions would be greatly appreciated.

If you are using raw dll's and not .NET assemblies then here are some handy P/Invokes for you:
[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
private static extern IntPtr LoadLibrary(string lpFileName);
[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
private static extern void SetDllDirectory(string lpPathName);
[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
privatestatic extern int GetModuleFileName(IntPtr module, [Out] StringBuilder fileName, int size);
[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
private static bool FreeLibrary(IntPtr module);
[DllImport("kernel32.dll", CharSet=CharSet.Auto)]
private IntPtr GetProcAddress(IntPtr hModule, string lpProcName);
Note that SetDllDirectory may need some protection as it is not available on all versions of windows (Windows 2000, in particular doesn't have it).
And in use:
SetDllDirectory(candidateFolder);
IntPtr dllHandle = LoadLibrary(dllName);
if (dllHandle != IntPtr.Zero)
{
_dllHandle = dllHandle;
_location = candidateFolder;
_fullPath = Path.Combine(candidateFolder, dllName);
IntPtr p = GetProcAddress(_dllHandle, procName);
if (p == IntPtr.Zero)
throw new ArgumentException("procName");
SomeDelegateType d = (SomeDelegateType)Marshal.GetDelegateForFunctionPointer(p, typeof(SomeDelegateType));
d(/* args */);
}
otherwise, you will be using Assembly methods. Looking at assembly level attributes or object level attributes is a good way to get extra information, although if what you want is a plug-in system, you should use a plug-in system, like the Managed Add-In Framework at CodePlex. See also this SO question and answer.

Take a look at the Castle Windsor framework. It is designed to handle all of your requirements including DLL unloading. It's also free and open source.

I don't know if changing how your program works is an option, but, you could use dependency injection for this, as long as they adhere to a certain interface.
The user selects, you dynamically set class to be loaded, and then just get an instance of the class.
I am not dealing with the unloading, I am just thinking about how you could possibly get classes, and since plinth already gave links to the functions for actually handling the dll, I think I will just end here.

For a native module, the simplest way to get "metadata" would be to define some C-exported (non-name-mangled) functions that return the information you want. At their simplest, these would return pointers to static data within the modules, e.g.:
extern "C" const char* GetModuleDescription();
...
const char* GetModuleDescription() { return "Dummy Module"; }
You would then load each ".dll" file in the directory using LoadLibrary, load and call your known exports from it using GetProcAddress. If you can't load a file or find the exports, then it's not a valid plugin module, so skip it.
Once you're done with a module, you can call FreeLibrary. Windows will then unload the module from your address space.

OK, I;ve figured out I need to use a second AppDomain, load the dll into that, and then I can unload the AppDomain as required.
string SignalSystemDLLPath = AppDomain.CurrentDomain.BaseDirectory + MyApp.Properties.Resources.SystemModuleFolder;
AppDomainSetup info = new AppDomainSetup();
info.ApplicationBase = DLLPath;
DLLDomain = AppDomain.CreateDomain("EditorDomain", null, info);
DLLPath is set to the subdir that holds the dll's.
I then foreach on all the dll's to get the AssemblyName, then later
I use
DLLDomain.Load(SelectedAssemblyName)
to load the DLL. I keep getting FileNotFound exceptions though.
After much googling I've decided its to much work at the moment, and I can refactor it later If I really need to do it...
Thank you for your replies though!

Found out how to do this very easy using MEF, simply use a DirectoryCatalog pointed at your plugin dir, and as long as you have matching [Export]s and [Import]s it works great.