We have a C# application and we need to provide Python scripting interface. Now it's implemented in a way that:
A managed scripting library (compiled as a DLL) does all the core work
A C++/CLI wrapper (wraps around the managed library and is compiled as a static library) does all the native-managed data translation
A boost extension project (BOOST_PYTHON_MODULE) (compiled as a dynamic library with the extension manually changed to pyd) talks to the C++/CLI wrapper layer.
The problem I have is that every time any python app tries to use the interface, it will complain the core managed assembly cannot be found. I have to place it either in the Python installation directory, or deploy it into the GAC.
I enabled the Windows Fusion logging and saw that it will only probe the assembly in the Python directory, I wonder if there is a way to direct Python to search for the .NET assembly in our application installation directory, or rather, any custom directory?
I tried all approaches in this post but that does not work for me. Any help will be highly appreciated!
PS:
We know that there are other python options like IronPython or Python .NET but we need to be backward compatible so a C++ interface is also required.
In case you are curious what error I get, here it is:
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'ScriptingInterface, Version=1.3.9.0, Culture=neutral, PublicKeyToken=160df4f7a3973cc2' or one of its dependencies. The system cannot find the file specified.
at NativeCommunicator.{ctor}(NativeCommunicator* )
Esentially to fix the problem you have to:
In your python script as a very first line you have to call "init".
"Init" is a function which you add in your .pyd module
In Init method you simply register a hook for resolving assemblies.
In the hook you have a chance to load your assemblies from whatever directory you want.
Link below might have be helpful for you:
Loading Mixed-Mode C++/CLI .dll (and dependencies) dynamically from unmanaged c++
Related
Since version 3.0, .NET installs a bunch of different 'reference assemblies' under C:\Program Files\Reference Assemblies\Microsoft...., to support different profiles (say .NET 3.5 client profile, Silverlight profile). Each of these is a proper .NET assembly that contains only metadata - no IL code - and each assembly is marked with the ReferenceAssemblyAttribute. The metadata is restricted to those types and member available under the applicable profile - that's how intellisense shows a restricted set of types and members. The reference assemblies are not used at runtime.
I learnt a bit about it from this blog post.
I'd like to create and use such a reference assembly for my library.
How do I create a metadata-only assembly - is there some compiler flag or ildasm post-processor?
Are there attributes that control which types are exported to different 'profiles'?
How does the reference assembly resolution at runtime - if I had the reference assembly present in my application directory instead of the 'real' assembly, and not in the GAC at all, would probing continue and my AssemblyResolve event fire so that I can supply the actual assembly at runtime?
Any ideas or pointers to where I could learn more about this would be greatly appreciated.
Update: Looking around a bit, I see the .NET 3.0 'reference assemblies' do seem to have some code, and the Reference Assembly attribute was only added in .NET 4.0. So the behaviour might have changed a bit with the new runtime.
Why? For my Excel-DNA ( http://exceldna.codeplex.com ) add-in library, I create single-file .xll add-in by packing the referenced assemblies into the .xll file as resources. The packed assemblies include the user's add-in code, as well as the Excel-DNA managed library (which might be referenced by the user's assembly).
It sounds rather complicated, but works wonderfully well most of the time - the add-in is a single small file, so no installation of distribution issues. I run into (not unexpected) problems because of different versions - if there is an old version of the Excel-DNA managed library as a file, the runtime will load that instead of the packed one (I never get a chance to interfere with the loading).
I hope to make a reference assembly for my Excel-DNA managed part that users can point to when compiling their add-ins. But if they mistakenly have a version of this assembly at runtime, the runtime should fail to load it, and give me a chance to load the real assembly from resources.
To create a reference assembly, you would add this line to your AssemblyInfo.cs file:
[assembly: ReferenceAssembly]
To load others, you can reference them as usual from your VisualStudio project references, or dynamically at runtime using:
Assembly.ReflectionOnlyLoad()
or
Assembly.ReflectionOnlyLoadFrom()
If you have added a reference to a metadata/reference assembly using VisualStudio, then intellisense and building your project will work just fine, however if you try to execute your application against one, you will get an error:
System.BadImageFormatException: Cannot load a reference assembly for execution.
So the expectation is that at runtime you would substitute in a real assembly that has the same metadata signature.
If you have loaded an assembly dynamically with Assembly.ReflectionOnlyLoad() then you can only do all the reflection operations against it (read the types, methods, properties, attributes, etc, but can not dynamically invoke any of them).
I am curious as to what your use case is for creating a metadata-only assembly. I've never had to do that before, and would love to know if you have found some interesting use for them...
If you are still interested in this possibility, I've made a fork of the il-repack project based on Mono.Cecil which accepts a "/meta" command line argument to generate a metadata only assembly for the public and protected types.
https://github.com/KarimLUCCIN/il-repack/tree/xna
(I tried it on the full XNA Framework and its working afaik ...)
Yes, this is new for .NET 4.0. I'm fairly sure this was done to avoid the nasty versioning problems in the .NET 2.0 service packs. Best example is the WaitHandle.WaitOne(int) overload, added and documented in SP2. A popular overload because it avoids having to guess at the proper value for *exitContext" in the WaitOne(int, bool) overload. Problem is, the program bombs when it is run on a version of 2.0 that's older than SP2. Not a happy diagnostic either. Isolating the reference assemblies ensures that this can't happen again.
I think those reference assemblies were created by starting from a copy of the compiled assemblies (like it was done in previous versions) and running them through a tool that strips the IL from the assembly. That tool is however not available to us, nothing in the bin/netfx 4.0 tools Windows 7.1 SDK subdirectory that could do this. Not exactly a tool that gets used often so it is probably not production quality :)
You might have luck with the Cecil Library (from Mono); I think the implementation allows ILMerge functionality, it might just as well write metadata only assemblies.
I have scanned the code base (documentation is sparse), but haven't found any obvious clues yet...
YYMV
I have a C# COM-Interop assembly which I am calling from a Visual Basic 6 application. This assembly makes HTTP requests to send and retrieve JSON.
The assembly works fine when being testing with a C# test client.
However, when using it from with the VB6 app, the following error is returned:
"Could not load file or assembly 'Newtonsoft.Json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified."
The Newtonsoft.Json.dll is located within the same folder as the COM-Interop DLL (TLB).
Does the Newtonsoft.Json.dll need to be explicitly loaded? Or maybe placed in the GAC?
Hans provided a great explanation for why this happens. Let me offer a workaround for making this work without having to register the Json DLL in the GAC or copying it to the VB6 EXE directory.
In your COM-visible C# library, we can tell the .NET runtime environment to search for the Json DLL in the directory of the C# library instead of the "usual" paths. We do that by attaching our own handler to the AssemblyResolve event:
AppDomain.CurrentDomain.AssemblyResolve += (sender, e) =>
{
// We only want this workaround for one particular DLL
if (e.Name != "Newtonsoft.Json")
return null;
var myLibraryFolder = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
var path = Path.Combine(myLibraryFolder, "Newtonsoft.Json.dll");
return Assembly.LoadFrom(path);
};
Notes about this workaround:
This code only works if it is executed in your C# library before doing anything that might cause the jitter to load the JSON library. For example, neither your library nor any other .NET library in your VB6 process must call any method referencing types from the JSON library before this code is executed.
You modify the the behaviour of the whole process, not just your library. If your VB6 process uses another library using JSON, your "redirect" affects the other library as well.
This is a standard DLL Hell problem, it is caused by using the /codepage option for Regasm.exe. Or, more commonly, the Project > Properties > Build tab > "Register for COM interop" checkbox. Both do the same thing, they write the path to the DLL into the registry. It is a very good option to use when you are busy developing and testing the project, it avoids having to re-register the DLL into the GAC every single time you make a change.
But what it does not do is help the CLR find any dependencies. The normal probing rules remain in effect, it looks for an appname.exe.config file in the directory where the EXE is stored. And first looks in the GAC, next in the EXE path for dependencies. Configuration remains under control of the usual victim of DLL Hell, whomever has to maintain the EXE. Frequently the end-user. So, explicitly, it does not look in the directory where your [ComVisible] DLL is stored.
It is the mild kind of DLL Hell, just a plain file-not-found mishap. Much milder than the nasty kind, finding a file with the right name but the wrong version. In general a strong problem with Newtonsoft.Json.dll, there are about 35 versions in the wild. Having so many versions and it being such a popular library also begets the other kind of nastiness, the program using another COM server that also uses the DLL. But almost inevitably a different version. Tends to happen long after you declared your project finished. One of them is going to lose, 50-50 odds that it is you. 100% odds for the end-user.
Yes, the GAC solves this problem. Each library gets the version they ask for. Ideally Newtonsoft would solve this problem for you with an installer that deploys the DLL into the GAC. But it is not the kind of commitment that open source library writers ever want to provide. They want (and need) to make it your problem. Microsoft does this, but they also have Windows Update to ensure that critical bug and security fixes get deployed. And have a large number of people working on making sure that any new revisions are always backwards compatible with the original release so the version number doesn't have to change.
Do note that you can take advantage of Microsoft's commitment. You can also use the DataContractJsonSerializer and JavascriptSerializer classes to get this job done. Part of the framework, they rarely get it wrong.
Meanwhile, do keep mind that is just a file-not-found problem. You don't have to use the GAC on your dev machine, and it is better if you don't, it is just as easy to copy the file into the right place to keep the CLR happy. Which is the same directory as your VB6 test program. And, extra quirk with VB6, into C:\Program Files (x86)\Visual Studio\VB6 if you want to use the VB6 debugger. Do use the GAC when you deploy.
As the title says I want to load a C++ program into a C# AppDomain using C#.
I cannot use AppDomain.ExecuteAssembly() because it has to be a C# or another non C++ language to use this.
Is it possible to load a C++ programm?
As far as I know AppDomain is a logical region inside a process, in which one or more assemblies can be loaded. That being said, I don't think that this is even feasible.
However, you have the ability you invoke native C++ DLL from a .NET assembly. Here is a helpful link, in which you will some options you have in order to invoke native C++ DLL from a .NET assembly.
side note
I cannot use AppDomain.ExecuteAssembly() because it has to be a C#
program to use this.
This is not true. If you write your program in any .NET language, the compiled version of your code would be an assembly, which will contain the same code and data files. So, AppDomain.ExecuteAssembly() can execute any .net assembly, without this assembly have been created by a C# compiler.
I wrote a Win32-DLL (with clr support in VS 2010/13, c++) as extension for another/old VB6 app and use the opensource-dll PDFSharp.
It works fine, but if the "PDFSharp.dll" removed from Directory the Application crashes if the program try to load my dll.
I want to include the Sharp DLL into mine, so that only one DLL is needed.
I tried to add it to resources, and load/catch the error during run time by
AppDomain^ root = AppDomain::CurrentDomain;
root->CurrentDomain->AssemblyResolve += gcnew ResolveEventHandler(MyResolveEventHandler);
in the first Function that the app calls, but my Problem is, the app/dll crashes before i can handle something.
ILMerge can't help, because it is a Win32/net(clr) DLL not a 100% NET-DLL.
C++/CLI mixed-mode DLLs have two sets of references: the native imports in the PE header, and the .NET assembly references. Problems finding the native imports will cause the symptom you observed, that loading the assembly fails early during load and cannot be intercepted and recovered.
It's not clear to me why the native dependency rules are applicable here. For a true native dependency that needs to be located using an alternate search order under your control, delay-loading could be applied. But that can't be used with a referenced .NET assembly.
In any case, the simplest fix is to not need a separate assembly at all. Your goal is single file deployment, and the ideal single file deployment scenario is when all the code is contained in a single DLL and you don't need to unpack a second file at runtime.
For pure .NET assemblies, there is an ILMerge tool that combines multiple DLLs into a single file. But your case has a C++/CLI mixed mode DLL, not pure MSIL.
Using multiple languages in a native program generally works a little bit differently. Instead of producing a complete executable from each toolset, native code standardizes an object file format (Windows .obj, Linux .o) which all the various toolsets know how to produce, and then the link step can link together object files from a variety of languages. The object files are often bundled into static libraries. (A static library is just an archive of object files, with a symbol index) Because the C++/CLI toolset is patterned on native C++, it uses this model as well.
The .NET version of this language-independent "object file" which can be further linked is a .netmodule file. Internally, it is a .NET assembly without a manifest. Functionally, it acts like a static library. And the C++/CLI link.exe can link together C# (and VB, and F#, etc) .netmodule static libraries together with the C++/CLI object files and static libraries, and native object files and libraries, when it creates the mixed-mode assembly.
This isn't the most straightforward process, because while it is supported by the underlying toolchains, the Visual Studio project options dialog boxes don't have a UI for either creating or consuming .netmodule static libraries.
For the C# side to produce a .netmodule, you should open your .csproj file and change the <OutputType> setting to module. Then reopen the project in Visual Studio and build as usual.
On the C++/CLI side, the project options dialog allows you to customize the compile and link command-lines. Change the linker command to include /link and the name of the .netmodule file.
If you've done it right, the C++/CLI linker will create a single mixed-mode DLL with all the types and code from both the C# and C++/CLI source files. And all the internal usage between C# and C++/CLI will be already resolved, so you won't have to worry about missing dependencies at run time. Well, at least not these dependencies; any you didn't choose to link in will still be handled normally.
I took a program written in C/C++ and modified it's main function to accept some arguments as input and return a variable as output and created a Win32 DLL out of it. I then created a .NET DLL which uses InterOp to access the first DLL. Now when I load the .NET DLL in my C# app I get a System.DllNotFoundException from the DLL which is really baffling me as there were never memory issues with the program and both Win32/.NET dlls are located in the same directory (apart from modifying the main function, the code has not really changed).
The solution was provided in this thread, which was my original question some time ago. I'm pretty sure that answer is correct but I'm just missing something.
You can download my VS solution Here. The solution contains three projects: the Win32 DLL, the .NET DLL, and a winform app that references the .NET DLL (but when trying to test gives the DLL exception). Any help or debugging guidance would be greatly appreciated.
UPDATE: I have tried all the tips/suggestions below but I still get the exact same error. If it makes things easier, my VS solution is available to download in the hyperlink above.
Make sure you have placed the win32 dll on /windows/system32 folder(if only the dll name is passed to DllImport)
Alternatively you can also pass the full path of the dll to the DllImport Attribute.
Use a tool such as Dependency Walker to make sure you are not missing out on any dependent assembly.