We Keep Coding

C# Dynamic compile and replace/reload of assembly from within same assembly

I have several issues with several SDK's comming from OEM manufacturers for specific devices. SDK is usually based on C or C++ dll, so I have a lot of Marshaling going around (a lot===YOU CAN'T EVEN IMAGINE). Problem start with next version of SDK when they extend some functions or some structures, they effectively break compatibility. In past I have made copy of our library supporting their device and start making changes to support new SDK. But each time our library was only for specific SDK, and upgrades of our systems were tough (Installation script if one heavy thing also ~ 3 GB install).
I have 78 projects in solution, commonly 4-5 libraries for each OEM Manufacturer, this is without any service tools. And Yesterday I said NO MORE. Started research on subject how to recompile C# code in runtime and reload/replace same assembly without quiting App.
And the result is the following:
- Class file that defines external C/C++ dll API was referenced from external Project referencing only System.dll. And me being insane I've already had each SDK version changes wrapped around #if #elif #endif so I could recompile last version of our library to support previous version of SDK. But that was maybe only once done, I've used #defines along with CSharpCodeProvider to recompile this assembly in runtime. Idea was like this:
Application loading ...
Open main SDK file get file version (extract version and identify it).
Load original External Assembly in new AppDomain (so I could destroy domain later).
Extract current version from external assembly.
Destroy new AppDomain to release hook from external assembly.
If versions mismatch, recompile external assembly (source code for external assembly is embedded within parent assembly), and replace original DLL with just compiled one.
Continue loading application...
So far this test approach works on one live demo system, and I was amazed. Switching from one to another SDK was flawless without any hick-ups.
And also code recompiles it self only when SDK version changes. So with safe guard I could say this is my first Metamorphic code I've wrote, that recompiles/changes it self from runtime.
Unfortunately this approach requires me to add one more Project for each OEM Manufacturers SDK. Which effectively kills my first though why I said NO MORE. True I now have only two libraries to maintain per one OEM manufacturer, and there will be no more projects added after this. But...
I wonder is there better approach which could allow me to replace DLL of currently loaded assembly in runtime from true within same assembly? Or change executing code on "fly" each time, this mainly includes Marshaled function, classes, structures, constants, ...?
Please notice code should be maintained from within same project without any externals. Also please notice this project exposes only hard-coded interface to "outside" world (Interface is referenced Interface only project - is more complex than I wrote). But this "outside" world is blind to any OEM specific stuff, which was the point using interface to have exactly same behavior across any OEM Device.
Any ideas? thoughts? suggestions?

Stepping Into / Debugging Interface Implementer

I have two projects in my solution Bridge and BridgeInterface being used by my applications update process and it's throwing a FileLoadException when I try to run use the Initialize() method from the Bridge class.
The problem is I can't step into this method, and I have run out of ideas, I need to see where the fault lies in the code!
The Bridge projects where not originally originally part of the solution, so I added both the projects and have pointed to their PDB symbol files. (The .DLL's are showing as symbols loaded in the modules window.)
I also have "Just My Code" unchecked and all projects in the solution are targetting the .NET 4 Full Profile.
Checked to see if the implementer was using DebuggerStepperBoundary or DebuggerStepThroughAttribute no such luck.
Any ideas?

Change the assembly references in your main project to use project references.
This way VS can compile the debug versions of the bridge projects and properly keep track of what's going on so you can step into it.

How can C# use a legacy DLL simply without registration(regsvr32)

Situation
I run a build system that executes many builds for many project. To avoid one build impacting another we lock down the build user to only its workspace. Builds run as a non privileged users who only have write ability to the workspace.
Challenge
During our new build we need to use a legacy 3rdparty DLL that exposes its interface through COM. The dev team wants to register the build(regsrv32.exe) but our build security regime blocks this activity. If we relax the regime then the 3rdparty DLL will impact other builds and if I have two build which need two different versions I may have the wrong build compile against the wrong version (a very real possibility).
Question
Are there any other options besides registration to handle legacy DLLs which expose their interface via COM?
Thanks for the help
Peter

For my original answer to a similar question see: TFS Build server and COM references - does this work?
A good way to compile .NET code that references COM components without the COM components being registered on the build server is to use the COMFileReference reference item in your project/build files instead of COMReference. A COMFileReference item looks like this:
<ItemGroup>
<COMFileReference Include="MyComLibrary.dll">
<EmbedInteropTypes>True</EmbedInteropTypes>
</COMFileReference>
</ItemGroup>
Since Visual Studio provides no designer support for COMFileReference, you must edit the project/build file by hand.
During a build, MSBuild extracts the type library information from the COM DLL and creates an interop assembly that can be either standalone or embedded in the calling .NET assembly.
Each COMFileReference item can also have a WrapperTool attribute but the default seemed to work for me just fine. The EmbedInteropTypes attribute is not documented as being applicable to COMFileReference, but it seems to work as intended.
See https://learn.microsoft.com/en-ca/visualstudio/msbuild/common-msbuild-project-items#comfilereference for a little more detail. This MSBuild item has been available since .NET 3.5.
It's a shame that no-one seems to know anything about this technique, which to me seems simpler than the alternatives. It's actually not surprising since I could only find just the one above reference to it on-line. I myself discovered this technique by digging into MSBuild's Microsoft.Common.targets file.

There's a walkthrough on registration-free COM here:
http://msdn.microsoft.com/en-us/library/ms973913.aspx
And excruciating detail here:
http://msdn.microsoft.com/en-us/library/aa376414
(the root of that document is actually here: http://msdn.microsoft.com/en-us/library/dd408052 )
Also, for building in general, you should be able to use Tlbimp or tlbexp to create a TLB file that you can use for building, assuming the point of registering is just to be able to compile successfully, and not to run specific tests.

Installation tools such as Installshield can extract the COM interfaces from the DLLs and add them to the registry. It can also use the self-registration process of the DLL (which I believe is what regsvr does), but this is not a Microsoft installer best practice.

in .NET COM is normally done thru Interop in order to register .DLL in .NET they are called Assemblies and that can be done several ways.. by adding references via VS IDE at the project level, or writing code that Loads and unloads the assembly.. by .Config file that haas the reference to the assembly as well as the using of that reference within the project... GAC.
If you have access to the 3rd party .DLL's you can GAC them, and reference them in your project
you can add a using to your .cs file header as well as add the reference to the project by right clicking on reference --> add Reference ...
you can also do the above step as well as set the copy local = true in the properties for that .dll.. I hope that this gives you some ideas.. keep in mind that .NET assemblies are Managed code so there are several ways to Consume those 3rd party .DLL's using other methods within C# like LoadFromAssembly ect..

Thanks for all the help.
We changed from early-binding to late-binding because we never really needed the DLL at compile time. This pushed the registration requirement from the build server to the integration test server (where we execute the installer which handles the registration). We try to keep the build system pristine and have easy-to-reset integration systems.
Thanks again
Peter

What is the recommendation for adding a library reference in a project?

When I need to add a reference towards a library, I've always been told to use the "add existing project" method, and referencing the project itself inside my solution.
But here in my new company, the use another method. They have a server which holds the compiled dll's, and keep versions of them so they can reference older versions when a change is too important to refactor older apps.
While I find this system really complicated (I guess there is a lot of work if a program pointing an older version of the dll is updated and needs some changes in this dll), they seem to find it pretty convenient.
What are the best practices for this? Linking the dll directly? Linking the project? And why? Any information is welcome!
Thanks in advance !

I usually take a copy of the compiled dll (if the source project is not available or if I don't need the source) and put it in a folder inside my solution, and then reference that. I check it in to source control along with my project.
I am of the opinion that you should be able to check out a project and build it directly from source control without having to go hunting dlls etc.

The Add existing project method is useful when you want to reference a library project which is develop side by side and you want to test/use it's types and methods and this (Add existing project) method will not be used to add the reference of pre-compiled (dll) files.
Read Project Reference (MSDN) article.

You would include it as a Project when you want to simultaneously work on the library. And that would happen mostly for small(ish) libs that will be distributed with your program (bin folder).
When an assembly is (going to be) installed in the GAC, and thus has its own release cycle, it makes more sense to reference the binary only.
Several combination of the above are possible too.

Adding Compiled dlls is mainly when the code is more or less locked (Architecture level code ) which you hardly ever changes
eg
1) communication layer(remoting/wcf)
2) Generic Gui layer (Wizards/dialog boxes)
3) Security layer (azman stuff)
you only need to change when your product is going to another direction say it uses to use .net remoting as communication now it will be using WCF
Using projects as reference when you are frequently changes referenced projects
also Visual studio works out nicely order of building the projects.

Although your company's approach is probably not very common among Microsoft developers, it is used rather successfully in the Java world. In the long run it is probably better controlled than any alternative, but without a fair amount of support scripts/programs (which, for instance, update projects/solutions automatically when needed) it can easily become unmanageable. In the Java world it is directly supported by tools such as Maven.

C# to C++/CLI to C DLL System.IO.FileNotFoundException

I'm getting System.IO.FileNotFoundException: The specified module could not be found when running C# code that calls a C++/CLI assembly which in turn calls a pure C DLL. It happens as soon as an object is instantiated that calls the pure C DLL functions.
BackingStore is pure C.
CPPDemoViewModel is C++/CLI calling BackingStore it has a reference to BackingStore.
I tried the simplest possible case - add a new C# unit test project that just tries to create an object defined in CPPDemoViewModel . I added a reference from the C# project to CPPDemoViewModel .
A C++/CLI test project works fine with just the added ref to CPPDemoViewModel so it's something about going between the languages.
I'm using Visual Studio 2008 SP1 with .Net 3.5 SP1. I'm building on Vista x64 but have been careful to make sure my Platform target is set to x86.
This feels like something stupid and obvious I'm missing but it would be even more stupid of me to waste time trying to solve it in private so I'm out here embarrassing myself!
This is a test for a project porting a huge amount of legacy C code which I'm keeping in a DLL with a ViewModel implemented in C++/CLI.
edit
After checking directories, I can confirm that the BackingStore.dll has not been copied.
I have the standard unique project folders created with a typical multi-project solution.
WPFViewModelInCPP
BackingStore
CPPViewModel
CPPViewModelTestInCS
bin
Debug
Debug
The higher-level Debug appears to be a common folder used by the C and C++/CLI projects, to my surprise.
WPFViewModelInCPP\Debug contains BackingStore.dll, CPPDemoViewModel.dll, CPPViewModelTest.dll and their associated .ilk and .pdb files
WPFViewModelInCPP\CPPViewModelTestInCS\bin\Debug contains CPPDemoViewModel and CPPViewModelTestInCS .dll and .pdb files but not BackingStore. However, manually copying BackingStore into that directory did not fix the error.
CPPDemoViewModel has the property Copy Local set which I assume is responsible for copying its DLL when if is referenced. I can't add a reference from a C# project to a pure C DLL - it just says A Reference to Backing Store could not be added.
I'm not sure if I have just one problem or two.
I can use an old-fashioned copying build step to copy the BackingStore.dll into any given C# project's directories, although I'd hoped the new .net model didn't require that.
DependencyWalker is telling me that the missing file is GPSVC.dll which has been suggested indicates security setting issues. I suspect this is a red herring.
edit2
With a manual copy of BackingStore.dll to be adjacent to the executable, the GUI now works fine. The C# Test Project still has problems which I suspect is due to the runtime environment of a test project but I can live without that for now.

Are the C and C++ DLLs in the same directory as the C# assembly that's executing?
You may have to change your project output settings so that the C# assembly and the other DLLs all end up in the same folder.
I've often used the Dependency Walker in cases like this; it's a sanity check that shows that all the dependencies can actually be found.
Once your app is running, you may also want to try out Process Monitor on the code you are running, to see which DLLs are being referenced, and where they are located.

The answer for the GUI, other than changing output settings, was the addition of a Pre-Build Step
copy $(ProjectDir)..\Debug\BackingStore.* $(TargetDir)
The answer for the Test projects was to add the missing DLL to the Deployment tab of the testrunconfig. You can either do so by directly editing the default LocalTestRun.testrunconfig (appears in Solution under Solution Items) or right-click the Solution and Add a new test run config, which will then appear under the main Test menu.
Thanks for the answers on this SO question on test configurations for leading me to the answer.

The reason why this happens is because you either are loading DLLMAIN from managed code, before the CRT has an opportunity to be initialized. You may not have any managed code, be executed DIRECTLY or INDERECTLY from an effect of DllMain notifications. (See: Expert C++/CLI: .Net for Visual C++ Programmers, chapter 11++).
Or you have no native entrypoint defined wahtsoever, yet you have linked to MSVCRT. The CLR is automatically initialized for you with /clr, this detail causes a lot of confusion and must be taken into account. A mixed mode DLL actually delay loads the CLR through the use of hot-patching all of the managed entry point vtables in your classes.
A number of class initialization issues surround this topic, loader lock and delay loading CLR are a bit trickey sometimes. Try to declare global's static and do not use #pragma managed/unmanaged, isolate your code with /clr per-file.
If you can not isolate your code from the managed code, and are having trouble, (after taking some of these steps), you can also look towards hosting the CLR yourself and perhaps going through the effort of creating a domain manager, that would ensure your fully "in-the-loop" of runtime events and bootstrapping.
This is exactally why, it has nothting todo with your search path, or initialization. Unfortunately the Fusion log viewer does not help that much (which is the usual place to look for .NET CLR assembly binding issues not dependency walker).
Linking statically has nothing todo with this either. You can NOT statically link a C++/CLI application which is mixed mode.
Place your DLLMAIN function into a file by itself.
Ensure that this file does NOT have /CLR set in the build options (file build options)
Make sure your linking with /MD or /MDd, and all your dependencies which you LINK use the exact same CRT.
Evaluate your linker's settings for /DEFAULTLIB and /INCLUDE to identify any possiable reference issues, you can declare a prototype in your code and use /INCLUDE to override default library link resolution.
Good luck, also check that book it's very good.

Make sure the target system has the correct MS Visual C runtime, and that you are not accidentally building the C dll with a debug runtime.

This is an interesting dilemma. I've never heard of a problem loading native .DLLs from C++/CLI after a call into it from C# before. I can only assume the problem is as #Daniel L suggested, and that your .DLL simply isn't in a path the assembly loader can find.
If Daniel's suggestion doesn't work out, I suggest you try statically linking the native C code to the C++/CLI program, if you can. That would certainly solve the problem, as the .DLL would then be entirely absorbed into the C++/CLI .DLL.

Had the same problem switching to 64-bit Vista. Our application was calling Win32 DLLs which was confusing the target build for the application. To resolve it we did the following:
Go to project properties;
Select Build tab;
Change 'Platform target:' option to x86;
Rebuild the application.
When I re-ran the application it worked.