I'm developing an Unity app which needs to dynamically load native libraries from outside where the app is installed, for some reason I cannot set the absolute path to DllImport before compiling (such as read the library path in a .txt at runtime and load it), and I don't want to use platform specific API such as LoadLibrary() on Windows or dlopen on Linux because it's inconvenient. And I have been struggled for several days.
I know that the search path can be adjusted by SetDllDirectory() on Windows from this post, and it works well when testing on .NET Framework applications.
However, it does not work in Unity which is based on mono 2.0, it just throws DllNotFoundException at runtime, but it works fine when I use absolute path in DllImport or copy the dll into my Unity project (I'm sure that the code is same)
The next way I tried is environment variable, and it does not work on both .NET and Mono, this post explained that CLR never refreshes the environment during process execution.
The third way I tried is to load the native library with platform specific API such as LoadLibrary() on Windows and dlopen() on Linux first, and then Dllimport may find that the library with same name has already been loaded, then it will use the load library to find function pointers, just as this post did. And I get the same result. The top answer of that question says we can write a wrapper class which uses platform specific API to explicitly load library and get functions pointers, instead of an approach focusing Dllimport, but it is not what I want.
If my guess is right, according to mono's document, DllImportAttribute calls LoadLibrary or dlopen internally at runtime to load a library into memory space. So it follows the search rules of specific OS platform, for example windows:
The directory from which the application loaded.
The current directory
The system directory. Use the GetSystemDirectory() function to get the path of this directory.
The 16-bit system directory.
The Windows directory. Use the GetWindowsDirectory() function to get
the path of this directory.
The directories that are listed in the PATH environment variable.
and Linux:
A colon-separated list of directories in the user’s LD_LIBRARY_PATH
environment variable. This is a frequently-used way to allow native
shared libraries to be found by a CLI program.
The list of libraries cached in /etc/ld.so.cache. /etc/ld.so.cache
is created by editing /etc/ld.so.conf and running ldconfig(8).
Editing /etc/ld.so.conf is the preferred way to search additional
directories, as opposed to using LD_LIBRARY_PATH, as this is more
secure (it’s more difficult to get a trojan library into
/etc/ld.so.cache than it is to insert it into LD_LIBRARY_PATH).
/lib, followed by /usr/lib.
By the way, I also tried to set LD_LIBRARY_PATH at runtime, but it does not work because LD_LIBRARY_PATH will be parsed only once when a process started, which is similar to PATH environment variable on Windows.
So my question is:
Why does the same code performs differently on .NET Framework and Mono? Does Mono just ignore the effect of SetDllDirectory() on Windows? What does DllImportAttribute actually do in Mono?
Is there any way to adjust search path for Unity/Mono apps at runtime, just using DllImport rather than platform specific APIs such as LoadLibrary() and dlopen()?
Unfortunately the answer is that this behaviour is different between Mono on Linux, and .Net on windows, so you are just going to have to deal with that.
The best option, if you know where each DLL is located (e.g. you can put this into a config file), is to load each one explicitly yourself using LoadLibrary or dlopen. This must be done prior to the first call to a DllImport function. DllImport then need not specify a path.
This way you know exactly which DLL you are getting, and can load them in whatever order is correct, if this is an issue.
If for some reason you really don't want to do that, I suggest you create a function like MySetDllDirectory which on Windows calls SetDllDirectory and on Linux sets LD_LIBRARY_PATH. In this way the changes can be isolated to a single module.
Related
I am working on a .NET Core project. This should run on Linux without problems. However, I need to call a library that has been created with C++. Traditionally, C# developers do this with DllImport&DllExport.
However, I do not want to import or export dll's in a production environment. Over there it needs to run on Linux. Is there a Linux equivalent of DllImport/DllExport? On Linux, so files are used instead of dll files. I am not smart when it comes to low level things so I look for a quick solution and a clear example. Something like an "SoImport"?
You can use .so file also in DllImport.
[DllImport ("libc.so")]
private static extern int getpid ();
From the dlopen(3) man page, the necessary shared libraries needed by the program are searched for in the following order:
A colon-separated list of directories in the user’s LD_LIBRARY_PATH environment variable. This is a frequently-used way to allow native shared libraries to be found by a CLI program.
The list of libraries cached in /etc/ld.so.cache. /etc/ld.so.cache is created by editing /etc/ld.so.conf and running ldconfig(8). Editing /etc/ld.so.conf is the preferred way to search additional directories, as opposed to using LD_LIBRARY_PATH, as this is more secure (it’s more difficult to get a trojan library into /etc/ld.so.cache than it is to insert it into LD_LIBRARY_PATH).
/lib, followed by /usr/lib.
More details can be found here library handling
edit:
DllImport will fail if libc.so contains any unresolved symbols. Please verify with
ldd -r -d libc.so
and make sure no unresolved symbols are present there.
I have a C# COM-Interop assembly which I am calling from a Visual Basic 6 application. This assembly makes HTTP requests to send and retrieve JSON.
The assembly works fine when being testing with a C# test client.
However, when using it from with the VB6 app, the following error is returned:
"Could not load file or assembly 'Newtonsoft.Json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified."
The Newtonsoft.Json.dll is located within the same folder as the COM-Interop DLL (TLB).
Does the Newtonsoft.Json.dll need to be explicitly loaded? Or maybe placed in the GAC?
Hans provided a great explanation for why this happens. Let me offer a workaround for making this work without having to register the Json DLL in the GAC or copying it to the VB6 EXE directory.
In your COM-visible C# library, we can tell the .NET runtime environment to search for the Json DLL in the directory of the C# library instead of the "usual" paths. We do that by attaching our own handler to the AssemblyResolve event:
AppDomain.CurrentDomain.AssemblyResolve += (sender, e) =>
{
// We only want this workaround for one particular DLL
if (e.Name != "Newtonsoft.Json")
return null;
var myLibraryFolder = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
var path = Path.Combine(myLibraryFolder, "Newtonsoft.Json.dll");
return Assembly.LoadFrom(path);
};
Notes about this workaround:
This code only works if it is executed in your C# library before doing anything that might cause the jitter to load the JSON library. For example, neither your library nor any other .NET library in your VB6 process must call any method referencing types from the JSON library before this code is executed.
You modify the the behaviour of the whole process, not just your library. If your VB6 process uses another library using JSON, your "redirect" affects the other library as well.
This is a standard DLL Hell problem, it is caused by using the /codepage option for Regasm.exe. Or, more commonly, the Project > Properties > Build tab > "Register for COM interop" checkbox. Both do the same thing, they write the path to the DLL into the registry. It is a very good option to use when you are busy developing and testing the project, it avoids having to re-register the DLL into the GAC every single time you make a change.
But what it does not do is help the CLR find any dependencies. The normal probing rules remain in effect, it looks for an appname.exe.config file in the directory where the EXE is stored. And first looks in the GAC, next in the EXE path for dependencies. Configuration remains under control of the usual victim of DLL Hell, whomever has to maintain the EXE. Frequently the end-user. So, explicitly, it does not look in the directory where your [ComVisible] DLL is stored.
It is the mild kind of DLL Hell, just a plain file-not-found mishap. Much milder than the nasty kind, finding a file with the right name but the wrong version. In general a strong problem with Newtonsoft.Json.dll, there are about 35 versions in the wild. Having so many versions and it being such a popular library also begets the other kind of nastiness, the program using another COM server that also uses the DLL. But almost inevitably a different version. Tends to happen long after you declared your project finished. One of them is going to lose, 50-50 odds that it is you. 100% odds for the end-user.
Yes, the GAC solves this problem. Each library gets the version they ask for. Ideally Newtonsoft would solve this problem for you with an installer that deploys the DLL into the GAC. But it is not the kind of commitment that open source library writers ever want to provide. They want (and need) to make it your problem. Microsoft does this, but they also have Windows Update to ensure that critical bug and security fixes get deployed. And have a large number of people working on making sure that any new revisions are always backwards compatible with the original release so the version number doesn't have to change.
Do note that you can take advantage of Microsoft's commitment. You can also use the DataContractJsonSerializer and JavascriptSerializer classes to get this job done. Part of the framework, they rarely get it wrong.
Meanwhile, do keep mind that is just a file-not-found problem. You don't have to use the GAC on your dev machine, and it is better if you don't, it is just as easy to copy the file into the right place to keep the CLR happy. Which is the same directory as your VB6 test program. And, extra quirk with VB6, into C:\Program Files (x86)\Visual Studio\VB6 if you want to use the VB6 debugger. Do use the GAC when you deploy.
I have a c++/CLI library that is in turn calling a c# library. That is fine, it is linking implicitly and all is good with the world. But for various reasons the libraries are not getting quite the prefect treatment by our automated build process, and the libraries are not finding each other unless we move the libraries to locations that we would rather not have them in, and would rather not fold into our build process.
It is suggested to me that we/I could write a post-build event that uses XCOPY. but lets say we don't want to do that.
Another suggestion is to explicitly load the dll. Windows says that to link explicitly "Applications must make a function call to explicitly load the DLL at run time." The problem is that Microsoft's example is not enough for my small mind to understand how to proceed with this idea. Worse, the only example I could find is out of date. Perhaps I am not using the right search terms but I am having difficulty finding more about it with google.
How do we explicitly Link a c++/Cli Library to a C# .dll?
----edit
OK, How do we explicitly Link a C++/CLI code, which exports a library using __declspec(), to a C# .dll.
There is no such thing as a "C++/CLI library", only assemblies are supported. There is no explicit or implicit linking, binding always happens at runtime. Assemblies are found at runtime by the CLR, the rules it uses to locate them are described in detail in the MSDN library.
Copying all dependencies into the same directory as the EXE is the sane way to go about it while you are developing the code. Well supported by build system, the C# and C++ rules are however different. C++ projects build to the solution's Debug directory, C# projects build to the EXE project's bin\Debug directory. So yes, altering a C++ project's Output Directory setting or copying files with a post build event is usually required to get everything together.
I'm working with an external DLL to consume an OCR device using a wrapper written by me. I have made tests on the wrapper and it works perfectly. But, when I use a WinForms project to consume the client class of the wrapper (located an another project), an error arises when calling C# methods imported from the DLL (using [DLLImport(...)]) saying that the DLL is not registered.
The error says:
"DLL Library function no found. Check registry install path."
All executions have been made in debug mode.
I've compared both projects configuration. The most relevant difference is that Test project is oriented to Any CPU and WinForms app only points to x86.
What could it be?
Updates
I've tried to register the dll using Regsvr32.exe but it didn't work. I thought about using Gacutil.exe but it required to uninstall all frameworks beyond .net framework 1.1...
I was wondering... at testing environment probably everything works well because testing framework has its dll's or executable files (or something like that) totally registered in windows, so those are trusted dlls. It is possible that debug generated dlls are not trusted by windows and therefore this problem arises?
I've created a form in the same troubling project and then I call the OCRWrapper from a button I've added to it. The OCR's worked!!. Unfortunately, it is difficult to rewrite the first form because we have invested a lot of hours in it; so, I'm still wondering what do I need to change in the troubling form...
I started again the form's development from scratch and added all the components related to it; everything worked well, the OCR read succesfully all data. When I loaded a combo box using a call to an ObjectContext and the error appeared again... I'm using an entity framework connected to Oracle.
I have a theory.
Let's imagine the following situation:
The ocr.dll depends on some other native DLL, lets call it other.dll [A].
In case this is a static dependency, you'll see it through Dependency Walker.
If its dynamic, you can use Sysinternals Process Explorer to monitor DLL loading in your working test project at run-time.
Your ADO.NET provider uses native DLLs under the hood (this is certainly true for ODP.NET) which depend on other.dll [B], which happens to have the same name but is actually a different DLL (or at least a different version) compared to other.dll [A].
Then, in run-time, this might happen:
When you connect to the database, ADO.NET provider dynamically loads its native DLLs, including the other.dll [B].
Then you try to call a function from OCR DLL. The P/Invoke tries to load the OCR DLL dynamically and succeeds, but the other.dll [B] is already loaded and ocr.dll tries to use some function from it, instead from other.dll [A] where it actually exists.
Welcome to DLL hell. So what can you do?
Try varying the order of calls to ocr.dll and ADO.NET provider to see anything changes. If you are (very) lucky, other.dll [A] might actually be a newer version that is still backward-compatible with other.dll [B] and things migh magically start to work.
Try another version of ADO.NET provider.
Try another ADO.NET provider.
Try getting a statically-linked ocr.dll from your vendor (i.e. no run-time dependency on other.dll [A]).
So, the call to the DLL works from a single button, but it does not work from a complex form. I'd say that there is an undefined behavior going on. The question remains whether it is you, that wrote the marshalling incorrectly, or it the DLL that is badly written.
Since we do not have access to the source code of the DLL, maybe you can post the prototype of the function, and all relevant struct definitions, and the DllImport line that you wrote for it?
Google can't find that error message which means(not definitely though :)) it is not a system message but a custom one coming from the code in the dll. So the dll does something dodgy. I guess it tries to double dispatch your call to another function internally.
Few things I suggest you try:
Run a x86 configuration. In the project properties -> Build tab set the platform to x86. this is assuming the dll is an x86 dll.
dumpbin /headers orc.dll
File Type: DLL
FILE HEADER VALUES
14C machine (**x86**)
4 number of sections
4CE7B6FC time date stamp Sat Nov 20 11:54:36 2010
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
2102 characteristics
Executable
32 bit word machine
DLL
This command line should tell you the bitness. In case it is a 64 bit run a 64 bit config instead but I bet it is 32 bit.
Do not include the dll in the project. I guess you do that already. Make sure the dll is in a folder that is in the %PATH% environment variable. When you run this at command prompt:
where ocr.dll
should tell you where the dll is. If it doesn't add the folder where the dll is installed to the %PATH%.
Is it necessary to register a compiled DLL (written in C# .NET) on a target machine.
The target machine will have .NET installed, is it enough to simply drop the DLL onto the target machine?
I think you're confusing things a little. Registering a dll has never been needed in order to use it.
Using a dll requires only to load it (given a known location or if the library is in the system path) and get the address of the function you wanted to use.
Registering the dll was used when distributing COM or ActiveX objects which need to add certain entries to the windows registry. In order to use a COM service (for example) you need to reference a GUID — that is, a unique identifier — which allows you to get a handle to the dll that implements the service (or provide access to it). Sometimes you can make reference to a fully-qualified name and get the same results.
In order for all that to work the dll needed to be registered. This "registration" process just creates several entries in the registry, but mainly these two: one associating a GUID with the location of the dll (so that you can reference it through the GUID without knowing where is it exactly located) and a second one associating the full name with the GUID. But again, this is just for COM or ActiveX objects.
When you develop an application in .NET, the libraries referenced on your project are automatically loaded when they're needed without you having to worry about locating or loading them. In order to to that, the framework checks two locations for the referenced libraries.
The first location is the application path.
The second location is the GAC.
The GAC (Global Assembly Cache) allows you to effectively register a dll to be used throughout the system and works as an evolution of the old registering mechanism.
So basically you just need to put the dll in the same folder of the application.
You need to "drop" it into a directory where the application needing it will find it.
If there are multiple applications, or you want to "drop" the file somewhere other than the application directory, you generally need to either adjust the PATH variable, or register the assembly in the Global Assembly Cache (GAC).
It is usually enough to drop the dll into the folder of your app on the target machine.
If the dll must be available to other applications then you may want to consider the GAC.
If you wish to access the assembly via com+. An example would be using a type defined in a .NET assembly from a non .NET application, such as a VB6 winforms app.
If you plan on accessing the assembly from another .NET application, you don't have to do anything. If your assembly has a strong name, it probably is a good idea to drop it in the GAC. Otherwise, just drop it in the directory of the application that will be referencing it.
One of the great selling points of .NET for the Windows platform when it came onto the scene is that by default, .NET assembly DLLs don't have to be registered and can be consumed privately by an application by merely putting them in the same folder as the EXE file. That was a great stride forward because it enabled developers to avoid the fray of DLL/COM hell.
Shared DLL/COM modules proved to be one of the greatest design mistakes of Windows as it lead to instability of applications that users installed. Installing a new app could well screw up an app that had been working just fine - because the new app introduced newer versions of shared DLL/COM modules. (It proved in practice to be too much of a burden for developers to properly manage fine-grained version dependencies.)
It's one thing to manage versions of modules with a build repository system like Maven. Maven works extremely well doing what it does.
It's an entirely different matter, though, to deal with that problem in an end-user runtime environment spread across a population of millions of users.
The .NET GAC is by no means a sufficient solution to this age-old Windows problem.
Privately consumed DLL assemblies continue to be infinitely preferable. It's a no-brainer way to go as diskspace is extremely cheap these days (~$100 can by a terabyte drive at Fry's these days). There is nothing to be gained with sharing assemblies with other products - and yet company reputation to loose when things go south for the poor user.
Actually there is NO need to register a dll in .NET on the target machine.
If you reference a .dll in your application, click on the referenced .dll under references in your project, look at the properties and set Isolated to TRUE.
This will now automatically include this .dll in your project and your application will use the copy of the .dll included in your project without any need to register it on the target system.
To see a working Example of this look here:
http://code.msdn.microsoft.com/SEHE
The .dll in question will need to be registered on the system where you build your application for this to work properly. However once you build your project, there will not be any need to register the .dll in question on any system you deploy your application or program.
An additional benefit of using this method, is that even if in the future, another .dll is registered with the same name on the target system in question, your project will continue to use the .dll you deployed with. This is very handy where a .dll has many versions and you wish to maintain some stability, like using the one you tested with, yet all other applications will use the registered .dll unless they use the isolated = true method as well.
The example above is one of those cases, there are many versions of Skype4COM which is a Skype API .dll and can change often.
This method allows the above example to use the API .dll that the project was tested with, each time a user installs a new version of Skype, it is possible that a modified version of this .dll is installed.
Also, there are some Skype clients that do not install this .dll, the business version of the Skype client for example, is smaller, and does not include this .dll, so in this case, the project does not fail on that .dll missing and not being registered because it is included in the project as isolated = true.
An application can use a .NET dll by simply having it present in the same folder with the application.
However if you want other third-party applications to find the DLL and use it they would also have to include it in their distribution. This may not be desirable.
An alternative is to have the DLL registered in the GAC (Global Assembly Cache).