We Keep Coding

C# Dynamic compile and replace/reload of assembly from within same assembly

I have several issues with several SDK's comming from OEM manufacturers for specific devices. SDK is usually based on C or C++ dll, so I have a lot of Marshaling going around (a lot===YOU CAN'T EVEN IMAGINE). Problem start with next version of SDK when they extend some functions or some structures, they effectively break compatibility. In past I have made copy of our library supporting their device and start making changes to support new SDK. But each time our library was only for specific SDK, and upgrades of our systems were tough (Installation script if one heavy thing also ~ 3 GB install).
I have 78 projects in solution, commonly 4-5 libraries for each OEM Manufacturer, this is without any service tools. And Yesterday I said NO MORE. Started research on subject how to recompile C# code in runtime and reload/replace same assembly without quiting App.
And the result is the following:
- Class file that defines external C/C++ dll API was referenced from external Project referencing only System.dll. And me being insane I've already had each SDK version changes wrapped around #if #elif #endif so I could recompile last version of our library to support previous version of SDK. But that was maybe only once done, I've used #defines along with CSharpCodeProvider to recompile this assembly in runtime. Idea was like this:
Application loading ...
Open main SDK file get file version (extract version and identify it).
Load original External Assembly in new AppDomain (so I could destroy domain later).
Extract current version from external assembly.
Destroy new AppDomain to release hook from external assembly.
If versions mismatch, recompile external assembly (source code for external assembly is embedded within parent assembly), and replace original DLL with just compiled one.
Continue loading application...
So far this test approach works on one live demo system, and I was amazed. Switching from one to another SDK was flawless without any hick-ups.
And also code recompiles it self only when SDK version changes. So with safe guard I could say this is my first Metamorphic code I've wrote, that recompiles/changes it self from runtime.
Unfortunately this approach requires me to add one more Project for each OEM Manufacturers SDK. Which effectively kills my first though why I said NO MORE. True I now have only two libraries to maintain per one OEM manufacturer, and there will be no more projects added after this. But...
I wonder is there better approach which could allow me to replace DLL of currently loaded assembly in runtime from true within same assembly? Or change executing code on "fly" each time, this mainly includes Marshaled function, classes, structures, constants, ...?
Please notice code should be maintained from within same project without any externals. Also please notice this project exposes only hard-coded interface to "outside" world (Interface is referenced Interface only project - is more complex than I wrote). But this "outside" world is blind to any OEM specific stuff, which was the point using interface to have exactly same behavior across any OEM Device.
Any ideas? thoughts? suggestions?

Create placeholder .NET assembly [duplicate]

Since version 3.0, .NET installs a bunch of different 'reference assemblies' under C:\Program Files\Reference Assemblies\Microsoft...., to support different profiles (say .NET 3.5 client profile, Silverlight profile). Each of these is a proper .NET assembly that contains only metadata - no IL code - and each assembly is marked with the ReferenceAssemblyAttribute. The metadata is restricted to those types and member available under the applicable profile - that's how intellisense shows a restricted set of types and members. The reference assemblies are not used at runtime.
I learnt a bit about it from this blog post.
I'd like to create and use such a reference assembly for my library.
How do I create a metadata-only assembly - is there some compiler flag or ildasm post-processor?
Are there attributes that control which types are exported to different 'profiles'?
How does the reference assembly resolution at runtime - if I had the reference assembly present in my application directory instead of the 'real' assembly, and not in the GAC at all, would probing continue and my AssemblyResolve event fire so that I can supply the actual assembly at runtime?
Any ideas or pointers to where I could learn more about this would be greatly appreciated.
Update: Looking around a bit, I see the .NET 3.0 'reference assemblies' do seem to have some code, and the Reference Assembly attribute was only added in .NET 4.0. So the behaviour might have changed a bit with the new runtime.
Why? For my Excel-DNA ( http://exceldna.codeplex.com ) add-in library, I create single-file .xll add-in by packing the referenced assemblies into the .xll file as resources. The packed assemblies include the user's add-in code, as well as the Excel-DNA managed library (which might be referenced by the user's assembly).
It sounds rather complicated, but works wonderfully well most of the time - the add-in is a single small file, so no installation of distribution issues. I run into (not unexpected) problems because of different versions - if there is an old version of the Excel-DNA managed library as a file, the runtime will load that instead of the packed one (I never get a chance to interfere with the loading).
I hope to make a reference assembly for my Excel-DNA managed part that users can point to when compiling their add-ins. But if they mistakenly have a version of this assembly at runtime, the runtime should fail to load it, and give me a chance to load the real assembly from resources.

To create a reference assembly, you would add this line to your AssemblyInfo.cs file:
[assembly: ReferenceAssembly]
To load others, you can reference them as usual from your VisualStudio project references, or dynamically at runtime using:
Assembly.ReflectionOnlyLoad()
or
Assembly.ReflectionOnlyLoadFrom()
If you have added a reference to a metadata/reference assembly using VisualStudio, then intellisense and building your project will work just fine, however if you try to execute your application against one, you will get an error:
System.BadImageFormatException: Cannot load a reference assembly for execution.
So the expectation is that at runtime you would substitute in a real assembly that has the same metadata signature.
If you have loaded an assembly dynamically with Assembly.ReflectionOnlyLoad() then you can only do all the reflection operations against it (read the types, methods, properties, attributes, etc, but can not dynamically invoke any of them).
I am curious as to what your use case is for creating a metadata-only assembly. I've never had to do that before, and would love to know if you have found some interesting use for them...

If you are still interested in this possibility, I've made a fork of the il-repack project based on Mono.Cecil which accepts a "/meta" command line argument to generate a metadata only assembly for the public and protected types.
https://github.com/KarimLUCCIN/il-repack/tree/xna
(I tried it on the full XNA Framework and its working afaik ...)

Yes, this is new for .NET 4.0. I'm fairly sure this was done to avoid the nasty versioning problems in the .NET 2.0 service packs. Best example is the WaitHandle.WaitOne(int) overload, added and documented in SP2. A popular overload because it avoids having to guess at the proper value for *exitContext" in the WaitOne(int, bool) overload. Problem is, the program bombs when it is run on a version of 2.0 that's older than SP2. Not a happy diagnostic either. Isolating the reference assemblies ensures that this can't happen again.
I think those reference assemblies were created by starting from a copy of the compiled assemblies (like it was done in previous versions) and running them through a tool that strips the IL from the assembly. That tool is however not available to us, nothing in the bin/netfx 4.0 tools Windows 7.1 SDK subdirectory that could do this. Not exactly a tool that gets used often so it is probably not production quality :)

You might have luck with the Cecil Library (from Mono); I think the implementation allows ILMerge functionality, it might just as well write metadata only assemblies.
I have scanned the code base (documentation is sparse), but haven't found any obvious clues yet...
YYMV

COM-Interop assembly not finding a native (.Net) dependancy when called from Vb

I have a C# COM-Interop assembly which I am calling from a Visual Basic 6 application. This assembly makes HTTP requests to send and retrieve JSON.
The assembly works fine when being testing with a C# test client.
However, when using it from with the VB6 app, the following error is returned:
"Could not load file or assembly 'Newtonsoft.Json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified."
The Newtonsoft.Json.dll is located within the same folder as the COM-Interop DLL (TLB).
Does the Newtonsoft.Json.dll need to be explicitly loaded? Or maybe placed in the GAC?

Hans provided a great explanation for why this happens. Let me offer a workaround for making this work without having to register the Json DLL in the GAC or copying it to the VB6 EXE directory.
In your COM-visible C# library, we can tell the .NET runtime environment to search for the Json DLL in the directory of the C# library instead of the "usual" paths. We do that by attaching our own handler to the AssemblyResolve event:
AppDomain.CurrentDomain.AssemblyResolve += (sender, e) =>
{
// We only want this workaround for one particular DLL
if (e.Name != "Newtonsoft.Json")
return null;
var myLibraryFolder = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
var path = Path.Combine(myLibraryFolder, "Newtonsoft.Json.dll");
return Assembly.LoadFrom(path);
};
Notes about this workaround:
This code only works if it is executed in your C# library before doing anything that might cause the jitter to load the JSON library. For example, neither your library nor any other .NET library in your VB6 process must call any method referencing types from the JSON library before this code is executed.
You modify the the behaviour of the whole process, not just your library. If your VB6 process uses another library using JSON, your "redirect" affects the other library as well.

This is a standard DLL Hell problem, it is caused by using the /codepage option for Regasm.exe. Or, more commonly, the Project > Properties > Build tab > "Register for COM interop" checkbox. Both do the same thing, they write the path to the DLL into the registry. It is a very good option to use when you are busy developing and testing the project, it avoids having to re-register the DLL into the GAC every single time you make a change.
But what it does not do is help the CLR find any dependencies. The normal probing rules remain in effect, it looks for an appname.exe.config file in the directory where the EXE is stored. And first looks in the GAC, next in the EXE path for dependencies. Configuration remains under control of the usual victim of DLL Hell, whomever has to maintain the EXE. Frequently the end-user. So, explicitly, it does not look in the directory where your [ComVisible] DLL is stored.
It is the mild kind of DLL Hell, just a plain file-not-found mishap. Much milder than the nasty kind, finding a file with the right name but the wrong version. In general a strong problem with Newtonsoft.Json.dll, there are about 35 versions in the wild. Having so many versions and it being such a popular library also begets the other kind of nastiness, the program using another COM server that also uses the DLL. But almost inevitably a different version. Tends to happen long after you declared your project finished. One of them is going to lose, 50-50 odds that it is you. 100% odds for the end-user.
Yes, the GAC solves this problem. Each library gets the version they ask for. Ideally Newtonsoft would solve this problem for you with an installer that deploys the DLL into the GAC. But it is not the kind of commitment that open source library writers ever want to provide. They want (and need) to make it your problem. Microsoft does this, but they also have Windows Update to ensure that critical bug and security fixes get deployed. And have a large number of people working on making sure that any new revisions are always backwards compatible with the original release so the version number doesn't have to change.
Do note that you can take advantage of Microsoft's commitment. You can also use the DataContractJsonSerializer and JavascriptSerializer classes to get this job done. Part of the framework, they rarely get it wrong.
Meanwhile, do keep mind that is just a file-not-found problem. You don't have to use the GAC on your dev machine, and it is better if you don't, it is just as easy to copy the file into the right place to keep the CLR happy. Which is the same directory as your VB6 test program. And, extra quirk with VB6, into C:\Program Files (x86)\Visual Studio\VB6 if you want to use the VB6 debugger. Do use the GAC when you deploy.

How to create a generic use library

I have an image editor user control(c# .net v2.0). Its used in thousands of computers as an activex component. I want the component also be available for windows forms and possible other uses.
For activex usage i add <object> tag in html code and call the component with clsid(a static guid). So if i build and distribute a newer version it works without changing any client code.
I want windows forms to be able use the same distributed libraries. And they should not reference a specific version so i can update the component without changing the programs that use it.
I use regasm to register for com. But i dont know how to reference it from visual studio(like referencing with clsid?)
May be in visual stuido when i choose add reference and select COM tab i shuld see my component in the list.
note: i tried to add the assemly to the global cache using these lines but it did not work out-or i coulnt understand if anything has changed :)
"C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\tlbexp" ImageEditorComp.dll /out:ImageEditorComp.tlb
regasm /tlb:ImageEditorComp.tlb ImageEditorComp.dll
"C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\gacutil" /I ImageEditorComp.dll
Any suggestions appreciated,
Regards

This is not possible, you'll invoke the infamous and dreaded DLL Hell problem. A stone cold hard rule in COM is that you have to change the [Guid] attribute values on public interfaces when you make a breaking change in either the publicly visible interfaces or the implementation of them. Changing the guids ensures that you don't overwrite the registry keys of an old version of your component when you use Regasm.exe. Existing programs that use your component and were not recompiled to use the latest version will continue running without problems. The typical outcome of DLL Hell is a nasty hardware exception like AccessViolation, very difficult to troubleshoot.
None of which applies in your specific case here. There is no point in trying to use the component through COM. It is a .NET assembly, just add the reference to it directly. The IDE will in fact stop you from adding a reference to the interop library. But not the .tlb. The GAC keeps you out of DLL Hell, assuming you properly increment [AssemblyVersion].

I figured out a solution.
To explain step by step:
1- Create the component with all needed properties for com.(Sign the assembly, use interfaces for com, make assembly com visible)
On the client machine
2- Register the assembly with regasm(i recommend adding safety flags too).
3- Add the assembly to the global cache using gacutil(or msi installer)
I figured out when you call a specific version of an assembly gac is searched in the first place so if its installed in GAC, referenced codebase path is never used.
When using as activeX you address the component with GUID. Since regasm adds the assembly name and version the GUID is representing, web browser directly uses component from GAC.
When using from a desktop application, reference the assebmly directly and set copylocal property to false. Similarly, in the client machine windows will find the assembly located at GAC itself.
Here is a useful link about the subject.
http://www.simple-talk.com/dotnet/visual-studio/build-and-deploy-a-.net-com-assembly/
Hope it saves other peoples time :)

In .NET, is there a need to register the DLL?

Is it necessary to register a compiled DLL (written in C# .NET) on a target machine.
The target machine will have .NET installed, is it enough to simply drop the DLL onto the target machine?

I think you're confusing things a little. Registering a dll has never been needed in order to use it.
Using a dll requires only to load it (given a known location or if the library is in the system path) and get the address of the function you wanted to use.
Registering the dll was used when distributing COM or ActiveX objects which need to add certain entries to the windows registry. In order to use a COM service (for example) you need to reference a GUID — that is, a unique identifier — which allows you to get a handle to the dll that implements the service (or provide access to it). Sometimes you can make reference to a fully-qualified name and get the same results.
In order for all that to work the dll needed to be registered. This "registration" process just creates several entries in the registry, but mainly these two: one associating a GUID with the location of the dll (so that you can reference it through the GUID without knowing where is it exactly located) and a second one associating the full name with the GUID. But again, this is just for COM or ActiveX objects.
When you develop an application in .NET, the libraries referenced on your project are automatically loaded when they're needed without you having to worry about locating or loading them. In order to to that, the framework checks two locations for the referenced libraries.
The first location is the application path.
The second location is the GAC.
The GAC (Global Assembly Cache) allows you to effectively register a dll to be used throughout the system and works as an evolution of the old registering mechanism.
So basically you just need to put the dll in the same folder of the application.

You need to "drop" it into a directory where the application needing it will find it.
If there are multiple applications, or you want to "drop" the file somewhere other than the application directory, you generally need to either adjust the PATH variable, or register the assembly in the Global Assembly Cache (GAC).

It is usually enough to drop the dll into the folder of your app on the target machine.
If the dll must be available to other applications then you may want to consider the GAC.

If you wish to access the assembly via com+. An example would be using a type defined in a .NET assembly from a non .NET application, such as a VB6 winforms app.
If you plan on accessing the assembly from another .NET application, you don't have to do anything. If your assembly has a strong name, it probably is a good idea to drop it in the GAC. Otherwise, just drop it in the directory of the application that will be referencing it.

One of the great selling points of .NET for the Windows platform when it came onto the scene is that by default, .NET assembly DLLs don't have to be registered and can be consumed privately by an application by merely putting them in the same folder as the EXE file. That was a great stride forward because it enabled developers to avoid the fray of DLL/COM hell.
Shared DLL/COM modules proved to be one of the greatest design mistakes of Windows as it lead to instability of applications that users installed. Installing a new app could well screw up an app that had been working just fine - because the new app introduced newer versions of shared DLL/COM modules. (It proved in practice to be too much of a burden for developers to properly manage fine-grained version dependencies.)
It's one thing to manage versions of modules with a build repository system like Maven. Maven works extremely well doing what it does.
It's an entirely different matter, though, to deal with that problem in an end-user runtime environment spread across a population of millions of users.
The .NET GAC is by no means a sufficient solution to this age-old Windows problem.
Privately consumed DLL assemblies continue to be infinitely preferable. It's a no-brainer way to go as diskspace is extremely cheap these days (~$100 can by a terabyte drive at Fry's these days). There is nothing to be gained with sharing assemblies with other products - and yet company reputation to loose when things go south for the poor user.

Actually there is NO need to register a dll in .NET on the target machine.
If you reference a .dll in your application, click on the referenced .dll under references in your project, look at the properties and set Isolated to TRUE.
This will now automatically include this .dll in your project and your application will use the copy of the .dll included in your project without any need to register it on the target system.
To see a working Example of this look here:
http://code.msdn.microsoft.com/SEHE
The .dll in question will need to be registered on the system where you build your application for this to work properly. However once you build your project, there will not be any need to register the .dll in question on any system you deploy your application or program.
An additional benefit of using this method, is that even if in the future, another .dll is registered with the same name on the target system in question, your project will continue to use the .dll you deployed with. This is very handy where a .dll has many versions and you wish to maintain some stability, like using the one you tested with, yet all other applications will use the registered .dll unless they use the isolated = true method as well.
The example above is one of those cases, there are many versions of Skype4COM which is a Skype API .dll and can change often.
This method allows the above example to use the API .dll that the project was tested with, each time a user installs a new version of Skype, it is possible that a modified version of this .dll is installed.
Also, there are some Skype clients that do not install this .dll, the business version of the Skype client for example, is smaller, and does not include this .dll, so in this case, the project does not fail on that .dll missing and not being registered because it is included in the project as isolated = true.

An application can use a .NET dll by simply having it present in the same folder with the application.
However if you want other third-party applications to find the DLL and use it they would also have to include it in their distribution. This may not be desirable.
An alternative is to have the DLL registered in the GAC (Global Assembly Cache).