We Keep Coding

AppDomain Assembly not found when loaded from byte array

Please bear with me, I spent 30+ hours trying to get this work - but without success.
At the start of my program I load an Assembly (dll) in bytearray and delete it afterwards.
_myBytes = File.ReadAllBytes(#"D:\Projects\AppDomainTest\plugin.dll");
Later on in the program I create a new Appdomain, load the byte array and enumerate the types.
var domain = AppDomain.CreateDomain("plugintest", null, null, null, false);
domain.Load(_myBytes);
foreach (var ass in domain.GetAssemblies())
{
Console.WriteLine($"ass.FullName: {ass.FullName}");
Console.WriteLine(string.Join(Environment.NewLine, ass.GetTypes().ToList()));
}
The types get correctly listed:
ass.FullName: plugin, Version=1.0.0.0, Culture=neutral,PublicKeyToken=null
...
Plugins.Test
...
Now I want to create an instance of that type in the new AppDomain
domain.CreateInstance("plugin", "Plugins.Test");
This call results in System.IO.FileNotFoundException and I don't know why.
When I look in ProcessExplorer under .NET Assemblies -> Appdomain: plugintest I see that the assembly is loaded correctly in the new appdomain.
I suspect the exception to occur because the assembly is searched again on disk. But why does the program want to load it again?
How can I create an instance in a new appdomain with an assembly loaded from byte array?

The main problem here is thinking that you can instantiate a plugin while executing code in your primary appdomain.
What you need to do instead, is create a proxy type which is defined in an already loaded assembly, but instantiated in the new appdomain. You can not pass types across app domain boundaries without the type's assembly being loaded in both appdomains. For instance, if you want to enumerate the types and print to console as you do above, you should do so from code which is executing in the new app domain, not from code that is executing in the current app domain.
So, lets create our plugin proxy, this will exist in your primary assembly and will be responsible for executing all plugin related code:
// Mark as MarshalByRefObject allows method calls to be proxied across app-domain boundaries
public class PluginRunner : MarshalByRefObject
{
// make sure that we're loading the assembly into the correct app domain.
public void LoadAssembly(byte[] byteArr)
{
Assembly.Load(byteArr);
}
// be careful here, only types from currently loaded assemblies can be passed as parameters / return value.
// also, all parameters / return values from this object must be marked [Serializable]
public string CreateAndExecutePluginResult(string assemblyQualifiedTypeName)
{
var domain = AppDomain.CurrentDomain;
// we use this overload of GetType which allows us to pass in a custom AssemblyResolve function
// this allows us to get a Type reference without searching the disk for an assembly.
var pluginType = Type.GetType(
assemblyQualifiedTypeName,
(name) => domain.GetAssemblies().Where(a => a.FullName == name.FullName).FirstOrDefault(),
null,
true);
dynamic plugin = Activator.CreateInstance(pluginType);
// do whatever you want here with the instantiated plugin
string result = plugin.RunTest();
// remember, you can only return types which are already loaded in the primary app domain and can be serialized.
return result;
}
}
A few key points in the comments above I will reiterate here:
You must inherit from MarshalByRefObject, this means that the calls to this object can be proxied across app-domain boundaries using remoting.
When passing data to or from the proxy class, the data must be marked [Serializable] and also must be in a type which is in the currently loaded assembly. If you need your plugin to return some specific object to you, say PluginResultModel then you should define this class in a shared assembly which is loaded by both assemblies/appdomains.
Must pass an assembly qualified type name to CreateAndExecutePluginResult in its current state, but it would be possible to remove this requirement by iterating the assemblies and types yourself and removing the call to Type.GetType.
Next, you need to create the domain and run the proxy:
static void Main(string[] args)
{
var bytes = File.ReadAllBytes(#"...filepath...");
var domain = AppDomain.CreateDomain("plugintest", null, null, null, false);
var proxy = (PluginRunner)domain.CreateInstanceAndUnwrap(typeof(PluginRunner).Assembly.FullName, typeof(PluginRunner).FullName);
proxy.LoadAssembly(bytes);
proxy.CreateAndExecutePluginResult("TestPlugin.Class1, TestPlugin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null");
}
Going to say this again because it's super important and I didn't understand this for a long time: when you're executing a method on this proxy class, such as proxy.LoadAssembly this is actually being serialized into a string and being passed to the new app domain to be executed. This is not a normal function call and you need to be very careful what you pass to/from these methods.

This call results in System.IO.FileNotFoundException and I don't know why. I suspect the exception to occur because the assembly is searched again on disk. But why does the program want to load it again?
The key here is understanding loader contexts, there's an excellent article on MSDN:
Think of loader contexts as logical buckets within an application domain that hold assemblies. Depending on how the assemblies were being loaded, they fall into one of three loader contexts.
Load context
LoadFrom context
Neither context
Loading from byte[] places the assembly in the Neither context.
As for the Neither context, assemblies in this context cannot be bound to, unless the application subscribes to the AssemblyResolve event. This context should generally be avoided.
In the code below we use the AssemblyResolve event to load the assembly in the Load context, enabling us to bind to it.
How can I create an instance in a new appdomain with an assembly loaded from byte array?
Note this is merely a proof of concept, exploring the nuts and bolts of loader contexts. The advised approach is to use a proxy as described by #caesay and further commented upon by Suzanne Cook in this article.
Here's an implementation that doesn't keep a reference to the instance (analogous to fire-and-forget).
First, our plugin:
Test.cs
namespace Plugins
{
public class Test
{
public Test()
{
Console.WriteLine($"Hello from {AppDomain.CurrentDomain.FriendlyName}.");
}
}
}
Next, in a new ConsoleApp, our plugin loader:
PluginLoader.cs
[Serializable]
class PluginLoader
{
private readonly byte[] _myBytes;
private readonly AppDomain _newDomain;
public PluginLoader(byte[] rawAssembly)
{
_myBytes = rawAssembly;
_newDomain = AppDomain.CreateDomain("New Domain");
_newDomain.AssemblyResolve += new ResolveEventHandler(MyResolver);
}
public void Test()
{
_newDomain.CreateInstance("plugin", "Plugins.Test");
}
private Assembly MyResolver(object sender, ResolveEventArgs args)
{
AppDomain domain = (AppDomain)sender;
Assembly asm = domain.Load(_myBytes);
return asm;
}
}
Program.cs
class Program
{
static void Main(string[] args)
{
byte[] rawAssembly = File.ReadAllBytes(#"D:\Projects\AppDomainTest\plugin.dll");
PluginLoader plugin = new PluginLoader(rawAssembly);
// Output:
// Hello from New Domain
plugin.Test();
// Output:
// Assembly: mscorlib
// Assembly: ConsoleApp
foreach (var asm in AppDomain.CurrentDomain.GetAssemblies())
{
Console.WriteLine($"Assembly: {asm.GetName().Name}");
}
Console.ReadKey();
}
}
The output shows CreateInstance("plugin", "Plugins.Test") is successfully called from the default app domain, although it has no knowledge of the plugin assembly.

Have you tried providing the Assemblies full name, In your case
domain.CreateInstance("plugin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null", "Plugins.Test");

Rehosted Workflow Designer throws TypeLoadException upon Load

I'm trying to rehost the workflow designer in my WPF app.
However, when I try to initialise it I get a TypeLoadException dialog with the following message:
"Could not load type
'Reporting.Primitives.Documents.IDocField`1' from
assembly 'Reporting.Primitives, Version=1.0.0.0,
Culture=neutral, PublicKeyToken=null'."
I've tried stripping it down and have gotten to the simple invocation code here:
var wnd = new Window();
var grid = new Grid();
wnd.Content = grid;
var met = new DesignerMetadata();
met.Register();
var d = new WorkflowDesigner();
d.Load(new Sequence());
grid.Children.Add(d.View);
wnd.Show();
It is the call to Load that causes the exception.
Here's the weird part: There's no type in the solution called IDocField<T>.
There's an IDocField and a DocField<T> : IDocField, so it seems to be inventing this IDocField<T> type from somewhere.
I don't have any reflection calls looking for an IDocField<T> either.
I've also tried moving this code around the app into various modules that aren't directly referencing Reporting.Primitives to no avail.
There is an IUiDocField<T> interface floating around.
Curiously enough, I did write an interface IDocField<T> some time ago, but it's not in the solution now, so I don't see why it would be causing problems.

As usual It's always your (my) fault.
The app pulls in plugins via a directory using reflection, and it appears there was an old dll in there that was referencing the IDocField<T> interface.
It would seem that this wasn't a problem until the workflow designer attempted to probe the assembly for type information that something finally noticed it was referencing something that wasn't there!
facepalm

Is it possible to use Gephi compiled with IKVM in a website?

I'm currently trying to load and use the Gephi Toolkit from within a .Net 4 C# website.
I have a version of the toolkit jar file compiled against the IKVM virtual machine, which works as expected from a command line application using the following code:
var controller = (ProjectController)Lookup.getDefault().lookup(typeof(ProjectController));
controller.closeCurrentProject();
controller.newProject();
var project = controller.getCurrentProject();
var workspace = controller.getCurrentWorkspace();
The three instances are correctly instantiated in a form similar to org.gephi.project.impl.ProjectControllerImpl#8ddb93.
If however I run the exact same code, with the exact same using statements & references, the very first line loading the ProjectController instance returns null.
I have tried a couple of solutions
Firstly, I have tried ignoring the Lookup.getDefault().lookup(type) call, instead trying to create my own instances:
var controller = new ProjectControllerImpl();
controller.closeCurrentProject();
controller.newProject();
var project = controller.getCurrentProject();
var workspace = controller.getCurrentWorkspace();
This fails at the line controller.newProject();, I think because internally (using reflector) the same Lookup.getDefault().lookup(type) is used in a constructor, returns null and then throws an exception.
Secondly, from here: Lookup in Jython (and Gephi) I have tried to set the %CLASSPATH% to the location of both the toolkit JAR and DLL files.
Is there a reason why the Lookup.getDefault().lookup(type) would not work in a web environment? I'm not a Java developer, so I am a bit out of my depth with the Java side of this.
I would have thought it possible to create all of the instances myself, but haven't been able to find a way to do so.
I also cannot find a way of seeing why the ProjectController load returned null. No exception is thrown, and unless I'm being very dumb, there doesn't appear to be a method to see the result of the attempted load.
Update - Answer
Based on the answer from Jeroen Frijters, I resolved the issue like this:
public class Global : System.Web.HttpApplication
{
public Global()
{
var assembly = Assembly.LoadFrom(Path.Combine(root, "gephi-toolkit.dll"));
var acl = new AssemblyClassLoader(assembly);
java.lang.Thread.currentThread().setContextClassLoader(new MySystemClassLoader(acl));
}
}
internal class MySystemClassLoader : ClassLoader
{
public MySystemClassLoader(ClassLoader parent)
: base(new AppDomainAssemblyClassLoader(typeof(MySystemClassLoader).Assembly))
{ }
}
The code ikvm.runtime.Startup.addBootClassPathAssemby() didn't seem to work for me, but from the provided link, I was able to find a solution that seems to work in all instances.

This is a Java class loader issue. In a command line app your main executable functions as the system class loader and knows how to load assembly dependencies, but in a web process there is no main executable so that system class loader doesn't know how to load anything useful.
One of the solutions is to call ikvm.runtime.Startup.addBootClassPathAssemby() to add the relevant assemblies to the boot class loader.
For more on IKVM class loading issues see http://sourceforge.net/apps/mediawiki/ikvm/index.php?title=ClassLoader

How do I pass references as method parameters across AppDomains?

I have been trying to get the following code to work(everything is defined in the same assembly) :
namespace SomeApp{
public class A : MarshalByRefObject
{
public byte[] GetSomeData() { // }
}
public class B : MarshalByRefObject
{
private A remoteObj;
public void SetA(A remoteObj)
{
this.remoteObj = remoteObj;
}
}
public class C
{
A someA = new A();
public void Init()
{
AppDomain domain = AppDomain.CreateDomain("ChildDomain");
string currentAssemblyPath = Assembly.GetExecutingAssembly().Location;
B remoteB = domain.domain.CreateInstanceFromAndUnwrap(currentAssemblyPath,"SomeApp.B") as B;
remoteB.SetA(someA); // this throws an ArgumentException "Object type cannot be converted to target type."
}
}
}
What I'm trying to do is pass a reference of an 'A' instance created in the first AppDomain to the child domain and have the child domain execute a method on the first domain. In some point on 'B' code I'm going to call 'remoteObj.GetSomeData()'. This has to be done because the 'byte[]' from 'GetSomeData' method must be 'calculated' on the first appdomain.
What should I do to avoid the exception, or what can I do to achieve the same result?

The actual root cause was your dll was getting loaded from different locations in the two different app domains. This causes .NET to think they are different assemblies which of course means the types are different (even though they have the same class name, namespace etc).
The reason Jeff's test failed when run through a unit test framework is because unit test frameworks generally create AppDomains with ShadowCopy set to "true". But your manually created AppDomain would default to ShadowCopy="false". This would cause the dlls to be loaded from different locations which leads to the nice "Object type cannot be converted to target type." error.
UPDATE: After further testing, it does seem to come down to the ApplicationBase being different between the two AppDomains. If they match, then the above scenario works. If they are different it doesn't (even though I've confirmed that the dll is loaded into both AppDomains from the same directory using windbg) Also, if I turn on ShadowCopy="true" in both of my AppDomains, then it fails with a different message: "System.InvalidCastException: Object must implement IConvertible".
UPDATE2: Further reading leads me to believe it is related to Load Contexts. When you use one of the "From" methods (Assembly.LoadFrom, or appDomain.CreateInstanceFromAndUnwrap), if the assembly is found in one of the normal load paths (the ApplicationBase or one of the probing paths) then is it loaded into the Default Load Context. If the assembly isn't found there, then it is loaded into the Load-From Context. So when both AppDomains have matching ApplicationBase's, then even though we use a "From" method, they are both loaded into their respective AppDomain's Default Load Context. But when the ApplicationBase's are different, then one AppDomain will have the assembly in its Default Load Context while the other has the assembly in it's Load-From Context.

I can duplicate the issue, and it seems to be related to TestDriven.net and/or xUnit.net. If I run C.Init() as a test method, I get the same error message. However, if I run C.Init() from a console application, I do not get the exception.
Are you seeing the same thing, running C.Init() from a unit test?
Edit: I'm also able to duplicate the issue using NUnit and TestDriven.net. I'm also able to duplicate the error using the NUnit runner instead of TestDriven.net. So the problem seems to be related to running this code through a testing framework, though I'm not sure why.

This is a comment to #RussellMcClure but as it is to complex for a comment I post this as an answer:
I am inside an ASP.NET application and turning off shadow-copy (which would also solve the problem) is not really an option, but I found the following solution:
AppDomainSetup adSetup = new AppDomainSetup();
if (AppDomain.CurrentDomain.SetupInformation.ShadowCopyFiles == "true")
{
var shadowCopyDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
if (shadowCopyDir.Contains("assembly"))
shadowCopyDir = shadowCopyDir.Substring(0, shadowCopyDir.LastIndexOf("assembly"));
var privatePaths = new List<string>();
foreach (var dll in Directory.GetFiles(AppDomain.CurrentDomain.SetupInformation.PrivateBinPath, "*.dll"))
{
var shadowPath = Directory.GetFiles(shadowCopyDir, Path.GetFileName(dll), SearchOption.AllDirectories).FirstOrDefault();
if (!String.IsNullOrWhiteSpace(shadowPath))
privatePaths.Add(Path.GetDirectoryName(shadowPath));
}
adSetup.ApplicationBase = shadowCopyDir;
adSetup.PrivateBinPath = String.Join(";", privatePaths);
}
else
{
adSetup.ApplicationBase = AppDomain.CurrentDomain.SetupInformation.ApplicationBase;
adSetup.PrivateBinPath = AppDomain.CurrentDomain.SetupInformation.PrivateBinPath;
}
This will use the shadow-copy directory of the main app-domain as the application-base and add all shadow-copied assemblies to the private path if shadow-copy is enabled.
If someone has a better way of doing this please tell me.

How to host an IronPython engine in a separate AppDomain?

I am using the below code to execute the IronPython Script on separate "appDomain" from c#.
(I used this approach to resolve memory leakage issue)
The scripts which take a lesser time (less than 3 mins), executes fine.
But if the script which takes a longer time (more than 5mins) throws an exception saying
-> System.Runtime.Remoting.RemotingException: Object '/011b230e_2f28_4caa_8bbc_92fabb63b311/vhpajnwe48ogwedf6zwikqow_4.rem'
using System;
using Microsoft.Scripting;
namespace PythonHostSamle
{
class Program
{
static void Main(string[] args)
{
AppDomain sandbox = AppDomain.CreateDomain("sandbox");
var engine = IronPython.Hosting.Python.CreateEngine(sandbox);
var searchPaths = engine.GetSearchPaths();
searchPaths.Add(#"C:\Python25\Lib");
searchPaths.Add(#"C:\RevitPythonShell");
engine.SetSearchPaths(searchPaths);
ScriptScope scope = engine.ExecuteFile("C:\Python25\Test.py")
// Script takes morethan 5mins to execute(sleep in the script)
ObjectHandle oh = scope.GetVariableHandle("GlobalVariableName")
// System throws following exception
//System.Runtime.Remoting.RemotingException:
// Object '/011b230e_2f28_4caa_8bbc_92fabb63b311/vhpajnwe48ogwedf6zwikqow_4.rem'
Console.ReadKey();
}
}
}

Overriding InitializeLifetimeServices and returning null would be the normal approach. I doubt that's possible in your case. Including the <lifetime> element in the app.config file is another approach.