My application is in C# and the 3. party I am working with is closed source C++. I have access to the third-party library as .dll's and .h files.
I access the code using Platform Invoke
When using the third-party library I start by calling a function InitBase(...) which creates and returns a Base struct. But it is only allowed to have ONE base at a time.
One base can only perform one task. I want to be able to perform multiple tasks at a time (multithreaded).
Is it possible call the 3. party library with platform invoke from different threads where the threads don't interfere with each other in the third-party library? I would prefer each thread have a complete separate memory chuck while in the third-party library, so from the third-party libraries point of view, it is still the only instance existing.
If it is not possible I have to run my application in multiple processes as the only way to make it "multithreaded".
As I read your question, the library constrains you to a single instance of this object per instances of the DLL. Different threads in the same process will in the normal way of things use the same instance of a DLL. So it would seem that threads are not helpful to you.
One possible solution would be to use multiple processes, and IPC, as you yourself suggest.
Another option would be to load separate instances of the DLL into the process. The loader will re-use DLLs if they have the same name. If you make multiple copies of the DLL, the loader will not be able to re-use already loaded instances. You should be able to use hardlinks to avoid needing to use extra disk space for each copy, although I've never done that myself. Whenever I've done this I've copied the DLL to the temp folder and not worried about the disk space.
This multiple DLL instance technique won't always work (things may be shared other than global variables), but it often does. You would be well advised to contact the developer of the DLL for assistance.
Related
What is the most important use of AppDomains in C#?
The single most important use is that your code has to have one - i.e. everything you write in C# executes in an AppDomain. That is quite important ;-p
If you mean additional app-domains:
When using plugins and other untrusted code, it allows you both isolation, and the ability to unload them (you can't unload assemblies - only entire app-domains).
I'm using it currently to load dynamically generated dlls, so that I can unload them.
They also allow you to set different configuration files, trust levels, etc - but have associated costs of complexity and remoting.
MSDN has a section on app-domains, here.
I can't tell you what the most important use is, since that depends on the situation.
AppDomains are useful for sandboxing parts of your application. You can load extensions in an AppDomain and unload them again - something you cannot otherwise do. You can assign specific rights to AppDomains. Per default objects in different AppDomains cannot access each other.
AppDomains can be viewed as lightweight processes as they give you many of the same features. However, unlike a Process new AppDomains do not have their own thread per default. You have to manage AppDomains and threads yourself.
Also, AppDomains all share the same managed heap. This is usually not a problem, but it may have surprising effects as some instances like strings are shared among AppDomains. For regular use this is not an issue, but if you use strings for locking, threads in different AppDomain can affect each other.
In general, it's not so daily coding practice to use AppDomains, this could be considered something as an advanced concept.. but, starting from this simple thing, it's important to better understand concepts behind the word "AppDomain".
In terms of architecture, and taking it simple as possible, an AppDomain is an isolation container even in terms of memory addressing, inside it all the assemblies needed by an application are loaded and executed, even if this concept is more complicated to explain in details (I hope it's not about yor question to going so deeper).
Starting from there, the AppDomain class first of all is used to obtain access to the application related executing application domain, this could be done via the Singleton property implementation AppDomain.CurrentDomain. In this way it's possible to:
obtain access the loaded assemblies;
obtain access to the appdomain-shared data slots;
intems marshalling, in terms of unwrapping created instances from loaded assemblies in created domains.
Then, the AppDomain class is used to:
create more "domains" in the same process;
executing assemblies in the process;
manage the appdomain's loading/unloading process.
It could be useful to take a view of the code of the new Microsoft framework (not yet released) MEF (Managed Extesibility Framework) which is truly based on concepts like AppDomains creations and unload, dynamically loaded assemblies.
As a simple example of things and examples of what you can do with AppDomains, I can share this link.
I hope I answered your question.
A C# AppDomain is a logically isolated container inside which .NET code run. When you run any .NET code it always runs in a default appdomain.
Do watch this 30 minutes youtube video What is C# AppDomain ? which explains AppDomain in more detail.
But let me still try to explain in more detail. Lets say you get a third party DLL and you want to use it in your application. But you also suspect that the third party can have some malicious code so you would like to run the third party DLL in a constrained environment. Like you do not want the third party to access your c: drive or delete files and so on.
So you can create two AppDomains one which is for the third party and one for your own C# classes. For the third party appdomain you will apply security constraint that it can not access c: drive and for your C# DLLs you will have a unrestricted app domain.
Please read my blog for standard application of runtime loading of DLLs and cross-communication using AppDomain. https://blog.vcillusion.co.in/sending-events-through-application-domain-boundary/
Runtime Loading and unloading of DLLs: I worked on a project where DLLs are loaded at runtime by the user, and during program execution, the methods are executed using Reflection and unloaded during the program run.
Securing my Main Execution Program: We are loading the DLL dynamically so any exception that occurred in that dynamically loaded DLL didn't affect my main AppDomain. In case of corruption scenarios, we have the option of efficiently unloading and loading the DLL again.
Cross-AppDomain Communication: We can dynamically load any two DLLs at runtime in different AppDomain and make them communicate with each other.
I'm designing a solution that I believe will work well in a plugin style architecture.
The "Main Program" will provide services such as database access, messaging to remote clients, logging and other functions.
Each "Plugin" will be able to use those functions and contain custom functions that will perform "Plugin" specific logic.
I need to be able to install and uninstall the plugins at runtime without the "Main Program" being restarted. Uninstalls will mainly occur when new versions of the "Plugin" are available and need to be added to the "Main Program".
If I create each plugin as a DLL and place it in a folder I can signal the "Main Program" to load them and all is fine. What I'm not sure of is how to correctly unload them when no longer needed, without creating a memory leak.
If I do a Application.Load on the assembly the only way to unload it is to unload the whole AppDomain. I am not very familiar with how AppDomains work but this seems like it would cause issues.
Has anyone done this sort of thing before and can offer some advice on the best way to do this without creating instability or memory leaks?
In general, you need to load every plugin into its own AppDomain. There is no other way to make it unloadable. It doesn't make any memory leaks (if you don't make them yourself). And it is stable if you implement it right. It is not an easy task, but this topic is too big to explain in details, it would rather take a big article or a book.
If we create a singleton class in a dll and use it in another dll which instantiates this singleton class, and an exe uses this dll at a point of time, will these two dlls get unloaded immediately after their job or will they get unloaded only when the main exe exits?
Disclaimer: I am not an expert on memory or processes/app domains, so I may have some specifics wrong, but I firmly believe that my conclusion is valid - feel free to comment and correct any specific errors.
No, the DLL is not unloaded until the app domain that contains it (typically the main process for windows/console apps and the web site for ASP.NET apps) is closed.
From MSDN:
There is no way to unload an individual assembly without unloading all of the application domains that contain it.
So you can manually create (and destroy) app domains, which will unload any DLLs, but then you have to marshal data across the domain boundaries, which may be worse than just keeping the DLL in memory.
The objects created by the DLL may be removed from memory (if nothing holds a reference to them), but the code itself will remain in memory. In the case of a pure singleton, it will stay in memory until the app closes (since the class that manages it will always have a reference). You could remove the reference to the singleton from the factory class, but you still may have other code that references it, so you risk having multiple instances of your "singleton". If that's not a concern, that may be a valid strategy to dispose of the singleton when you're done with it.
Think about most apps - they respond to human interaction in some way. Users can click any button, select any menu item, etc. in any order that the app allows. How is an application supposed to know for certain that code within a DLL will never get called again? Assemblies are relatively expensive to load (in part since they must be JITted), so it would be inefficient to unload them if there's any chance that they'll get loaded again.
Bottom line - since the memory footprint of a DLL is usually relatively small (since it just contains code and metadata) and is only loaded once for an entire domain, unless you experience significant memory pressure from it that can't me relived by managing other memory better, I wouldn't go to the trouble of trying to unload it.
Sometimes you need to use a particular third-party library, like in my case, one that loads up PowerPoints and allows the user to modify them in code. We discovered that this particular library has some memory leaks, but we would still like to use it because these leaks only occur in one particular scenario that occurs very rarely. You can see objects lying around despite trying to dispose all references to them, and despite having these objects go out of scope, and despite having manually invoked garbage collection. For sure, this library creates leaks. Our application is single-threaded.
Now, that being said, I am wondering if there is any way to clean up all memory that the library has used during runtime. Are there any ways to unload and reload the DLL that might cause all memory allocations from that library to be cleared, or anything that we can do at runtime at all to clean the memory that this third party library uses and then potentially reload the library in our application?
You could investigate loading the referenced library inside a custom AppDomain; an app-domain is a unit of isolation inside a process - and can be unloaded. It does, however, require you to do some communications between the two app-domains (the default domain and the hosted domain); MarshalByRefObject is the easiest trick there.
Alternatively: just use an entire separate process for this work. On windows, creating a process is relatively expensive, but not so expensive that you should never do it. Shutting down the process when done is the equivalent of nuking it from orbit. You can always re-spawn another process later.
There is an entire series of "How to" topics about AppDomain linked from MSDN here
What is the most important use of AppDomains in C#?
The single most important use is that your code has to have one - i.e. everything you write in C# executes in an AppDomain. That is quite important ;-p
If you mean additional app-domains:
When using plugins and other untrusted code, it allows you both isolation, and the ability to unload them (you can't unload assemblies - only entire app-domains).
I'm using it currently to load dynamically generated dlls, so that I can unload them.
They also allow you to set different configuration files, trust levels, etc - but have associated costs of complexity and remoting.
MSDN has a section on app-domains, here.
I can't tell you what the most important use is, since that depends on the situation.
AppDomains are useful for sandboxing parts of your application. You can load extensions in an AppDomain and unload them again - something you cannot otherwise do. You can assign specific rights to AppDomains. Per default objects in different AppDomains cannot access each other.
AppDomains can be viewed as lightweight processes as they give you many of the same features. However, unlike a Process new AppDomains do not have their own thread per default. You have to manage AppDomains and threads yourself.
Also, AppDomains all share the same managed heap. This is usually not a problem, but it may have surprising effects as some instances like strings are shared among AppDomains. For regular use this is not an issue, but if you use strings for locking, threads in different AppDomain can affect each other.
In general, it's not so daily coding practice to use AppDomains, this could be considered something as an advanced concept.. but, starting from this simple thing, it's important to better understand concepts behind the word "AppDomain".
In terms of architecture, and taking it simple as possible, an AppDomain is an isolation container even in terms of memory addressing, inside it all the assemblies needed by an application are loaded and executed, even if this concept is more complicated to explain in details (I hope it's not about yor question to going so deeper).
Starting from there, the AppDomain class first of all is used to obtain access to the application related executing application domain, this could be done via the Singleton property implementation AppDomain.CurrentDomain. In this way it's possible to:
obtain access the loaded assemblies;
obtain access to the appdomain-shared data slots;
intems marshalling, in terms of unwrapping created instances from loaded assemblies in created domains.
Then, the AppDomain class is used to:
create more "domains" in the same process;
executing assemblies in the process;
manage the appdomain's loading/unloading process.
It could be useful to take a view of the code of the new Microsoft framework (not yet released) MEF (Managed Extesibility Framework) which is truly based on concepts like AppDomains creations and unload, dynamically loaded assemblies.
As a simple example of things and examples of what you can do with AppDomains, I can share this link.
I hope I answered your question.
A C# AppDomain is a logically isolated container inside which .NET code run. When you run any .NET code it always runs in a default appdomain.
Do watch this 30 minutes youtube video What is C# AppDomain ? which explains AppDomain in more detail.
But let me still try to explain in more detail. Lets say you get a third party DLL and you want to use it in your application. But you also suspect that the third party can have some malicious code so you would like to run the third party DLL in a constrained environment. Like you do not want the third party to access your c: drive or delete files and so on.
So you can create two AppDomains one which is for the third party and one for your own C# classes. For the third party appdomain you will apply security constraint that it can not access c: drive and for your C# DLLs you will have a unrestricted app domain.
Please read my blog for standard application of runtime loading of DLLs and cross-communication using AppDomain. https://blog.vcillusion.co.in/sending-events-through-application-domain-boundary/
Runtime Loading and unloading of DLLs: I worked on a project where DLLs are loaded at runtime by the user, and during program execution, the methods are executed using Reflection and unloaded during the program run.
Securing my Main Execution Program: We are loading the DLL dynamically so any exception that occurred in that dynamically loaded DLL didn't affect my main AppDomain. In case of corruption scenarios, we have the option of efficiently unloading and loading the DLL again.
Cross-AppDomain Communication: We can dynamically load any two DLLs at runtime in different AppDomain and make them communicate with each other.