We Keep Coding

AppDomain support is dead in UnityEngine, any way to unload dll?

I am kinda stuck on not being able to dispose .NET 3.5 dlls from the process.
AppDomain support is off in unity, there is no way to unload a dll from the process using the .NET api, because the C# functions are not implemented.
Anyone could get me some hints on how / where should I start to remove the dll from the memory / process somehow, so I can re-load the dll whenever I want?

Alright, after this time I thought that this is some sort of heavy research, and there are no publications related to this. So here is what you have to do. First head over to https://github.com/mono/mono/branches/all and determine which mono version you are going to require. In my case I was doing this for an old game, and I needed the 2014 version of mono.
My project is discontinued so there is no point for me to keep this as a secret. The following examples will show you a way, but It probably won't be enough for you to get what you want on a newer mono version.
Using this way, you are able to unload every dll in a new AppDomain. You can extend It to create multiple appdomains, or unload only a specific dll. If It is working on a newer mono version then please let me know! Credits go to me, and 4g3v.
Once you have done that you are going to need exactly the same environment, and compilers for that version. In my case that was the Visual Studio 2010 compilers, and I didn't need to refactor most of the things.
You will need more than just following my instructions, you will have to play with mono, and get to know how the project works.
So hence as stated: C# API doesn't support AppDomains, but mono by default does. You just need to make some improvements, and extensions for It. Here is what you need to do:
Define two new functions in mono.def for example
mono_rb_create_domain and
mono_rb_unload_domain
The above two functions will be responsible to create, and dispose a domain.
Head over to: mono/metadata/object.c
Find function mono_runtime_unhandled_exception_policy_set and add (We will create the function later below):
mono_add_internal_call("YourDLLNameSpace.Icalls::mono_rb_load_plugin", ves_icall_mono_rb_load_plugin); //Last mono function unity calls before adding their own icalls (mono_runtime_unhandled_exception_policy_set). Adding them at runtime doesn't work, so this should be a pretty good place.
The above code will define a C# function that will be able to handle the loading of a custom DLL loaded in our own AppDomain. Make sure that your C# class, and function is public. Reminder: This should be somewhere in your unity project already. (For example Assembly-CSharp or whatever). It is crucial, because this will be handling the loading of your new dlls, and they will go to a new appdomain.
[MethodImpl(MethodImplOptions.InternalCall)]
public extern Assembly mono_rb_load_plugin(IntPtr data, uint dataLen);
Alright, we have to add some additional checks to avoid crashes in unity/unity_liveness.c Look for function mono_add_process_object and make it look like the following. This might differ in newer mono versions. =) What we have done here is basically ensuring that the received object has a VTable (which should be the class), and It isn't null.
static void mono_add_process_object (MonoObject* object, LivenessState* state)
{
gboolean has_references = 0;
MonoClass* klass; // Define the class
if (object && !IS_MARKED(object))
{
klass = GET_VTABLE(object)->klass; // Get the VTable
// Ensure that the class isn't f***ed up. Read: https://en.wikipedia.org/wiki/Hexspeak
if(klass == NULL || klass == 0xBAADF00D || klass == 0xFEEEFEEE)
{
return;
}
has_references = klass->has_references;
if(has_references || should_process_value(object,state) != LIVENESS_DONT_PROCESS)
{
if (array_is_full(state->all_objects))
array_safe_grow(state, state->all_objects);
array_push_back(state->all_objects, object);
MARK_OBJ(object);
}
// Check if klass has further references - if not skip adding
if (has_references)
{
if(array_is_full(state->process_array))
array_safe_grow(state, state->process_array);
array_push_back(state->process_array, object);
}
}
}
The above code will ensure that the processed class isn't faulty, or points somewhere else where It shouldn't.
Let's create our domain handlers.
Make sure to create this under the mono project.
My header file was named as mono_rustbuster.h, and contained:
static MonoDomain* rustBusterDomain;
GHashTable* pluginHashTable;
void mono_method_info_object();
MonoReflectionAssembly* ves_icall_mono_rb_load_plugin(void* objectPtr, char* data, guint32 dataLen) MONO_INTERNAL;
Then we created mono_rustbuster.c, and wrote the following:
#include "metadata\metadata-internals.h"
#include "metadata\image.h"
#include "metadata\assembly.h"
#include "metadata\debug-helpers.h"
#include "metadata\class-internals.h"
#include "metadata\object-internals.h"
static MonoDomain* rustBusterDomain;
GHashTable* pluginHashTable;
void mono_method_info_object()
{
}
int mono_rb_create_domain()
{
pluginHashTable = g_hash_table_new(g_str_hash, g_str_equal);
rustBusterDomain = mono_domain_create_appdomain("PluginDomain", NULL);
return 0x01;
}
int mono_rb_unload_domain()
{
mono_domain_unload(rustBusterDomain);
return 0x01;
}
MonoReflectionAssembly* ves_icall_mono_rb_load_plugin(void* objectPtr, char* data, guint32 dataLen)
{
MonoAssembly* ass;
MonoImage* img;
MonoImageOpenStatus status;
MonoDomain* current;
char *assNameBuf;
current = mono_domain_get();
mono_domain_set(rustBusterDomain, FALSE);
img = mono_image_open_from_data_full(data, dataLen, TRUE, NULL, FALSE);
ass = mono_assembly_load_from_full(img, "", &status, FALSE);
assNameBuf = (char*)malloc(256);
sprintf(assNameBuf, "%s", ass->aname.name);
g_hash_table_insert(pluginHashTable, (gpointer)assNameBuf, (gpointer)ass);
mono_domain_set(current, FALSE);
return mono_assembly_get_object(rustBusterDomain, ass);
}
After this setup your loaded DLLs might whine about missing references. This mostly happens when you load a new DLL in all by yourself using these functions. We added some layers into mono/metadata/assembly.c for this fix.
Find mono_assembly_load_reference This method works with the assembly's references, find where It is calling the reference variable and add:
if(reference == NULL && strstr(image->name, "data-"))
{
reference = (MonoAssembly*)g_hash_table_lookup(pluginHashTable, aname.name);
}
Hint: Mono add's data- to all of the dll's, so that way we can use that memory pointer to find the references. Basically fixes the unresolved referencess.
Head over to mono/metadata/class.c and find: mono_class_is_assignable_from
Before function would return data using the last function check if the class names are equal.
The last return looks something like this:
return mono_class_has_parent (oklass, klass);
Add:
if(!mono_class_has_parent (oklass, klass))
{
if(strstr(klass->image->name, "data-"))
{
if(!strcmp((oklass)->supertypes [(klass)->idepth - 1]->name, klass->name))
{
//OutputDebugStringA("mono_class_is_assignable_from(): Class names are equal so true is returned");
return TRUE;
}
}
}
The above code will add fixes to ScriptableObject cast exceptions.
You are sort of done. You may encounter additional issues. Here is how it works in C#:
[DllImport("mono.dll")]
internal static extern int mono_rb_create_domain();
[DllImport("mono.dll")]
internal static extern int mono_rb_unload_domain();
Handle loading of a new dll:
var icalls = new Icalls();
int domaincreation = RustBuster.mono_rb_create_domain();
byte[] bytes = getyourdllsbytearraysomehow;
IntPtr pluginMem = Marshal.AllocHGlobal(bytes.Length);
for (int i = 0; i < bytes.Length; i++)
{
Marshal.WriteByte(pluginMem, i, bytes[i]);
}
assembly = icalls.mono_rb_load_plugin(pluginMem, (uint) bytes.Length);
Unload:
int domaincheck2 = RustBuster.mono_rb_unload_domain();

Evaluating nested property calls from a string

I am currently reading the "500 Lines or Less" book, the chapter for creating a Template Engine from Ned Batchelder.
Their example is using Python. In their template engine they are building code as a string and then they are calling exec (docs) to evaluate the string as Python code.
def get_globals(self):
"""Execute the code, and return a dict of globals it defines."""
# A check that the caller really finished all the blocks they started.
assert self.indent_level == 0
# Get the Python source as a single string.
python_source = str(self)
# Execute the source, defining globals, and return them.
global_namespace = {}
exec(python_source, global_namespace)
return global_namespace
This is very convenient, because they can easily evaluate expressions in the template such as {{object.property.property}}
With C# as my main programming language I am wondering how can this be achieved (in the context of building a template engine as in the book)?
Research and thoughts
First I don't believe there is an exec equivalent in C#.
One way I can think of it is to recursively use Reflection to get the List of properties of an object (handling checks for Null References), but I don't like this from performance point of view.
Another way is to use Roslyn's ScriptEngine class (which I haven't used so correct me if I am wrong). But I am afraid that this won't be good because this is supposed to be a library and it won't be able to be used with older versions of C# and .NET. Example

Q: First I don't believe there is an exec equivalent in C#.
As for compling C# code, CS-Script library can be used to achieve this in various ways.
For example:
dynamic script = CSScript.Evaluator
.LoadCode(#"using System;
using Your.Custom.Relevant.Namespace;
public class Executer
{
public object Execute()
{
return SomeStaticClass.array[123];
}
}");
int result = script.Execute();
//shorter way
int a = (int)CSScript.Evaluator.Evaluate("some.namespace.SomeStaticClass.array[123]");
Read more here: http://www.csscript.net/
CS-Script isn't made for templating.
Unless you create it yourself by manipulating the strings before you compile them.
But how can I pass some Context for the template engine
You can pass a context into a function like this:
dynamic script = CSScript.Evaluator
.LoadCode(#"
using System;
using Namespace.Of.The.Context;
public class Executer {
public string Execute(Context ctx) {
return ctx.Person.Firstname + ctx.Person.Lastname;
}
}");
int result = script.Execute(new Context(new Person("Rick", "Roll")));
Q: Can I call CSScript from a normal C# application lets say a Web App?
A: Yes.
S-Script currently targets Microsoft implementation of CLR (.NET
2.0/3.0/3.5/4.0/4.5) with full support on Mono.
Basically if it runs C#, it can be compiled accordingly to the .net-framework that the library is executed on, so if your project is ran on .net4.5, any feature of that .net version is available including any external references in your project too.

You can use Microsoft.CSharp.CSharpCodeProvider in order to compile code on fly.
https://msdn.microsoft.com/en-us/library/microsoft.csharp.csharpcodeprovider.aspx
Like this:
static void Main(string[] args)
{
string source =
#"
namespace Test
{
public class Test
{
public void HelloWorld()
{
System.Console.WriteLine(""Hello World"");
}
}
}
";
var options = new Dictionary<string, string> { {"CompilerVersion", "v3.5"} };
var provider = new CSharpCodeProvider(options);
var compilerParams = new CompilerParameters{GenerateInMemory = true, GenerateExecutable = false };
var results = provider.CompileAssemblyFromSource(compilerParams, source);
var method = results.CompiledAssembly.CreateInstance("Test.Test");
var methodInfo = method.GetType().GetMethod("HelloWorld");
methodInfo.Invoke(method, null);
}

Custom keyword coloring in Visual Studio 2010+

I'm trying to add custom coloring for only certain keywords in my Visual Studio editor for C# code. I want to be able to color any type that implements IDisposable as a different color. Ideally I'd like to create a simple list of classes/interfaces that derive from IDisposable in some sort of configuration that I can edit. (Although if you said there was a method/plugin that would automatically find all disposable types and color them independently that would be the Holy Grail).
I've done a ton of research and it looks like an "editor classifier" extension might do the trick. However I created one that merely tries to color the word "Stream" and although it does hit my code that attempts to highlight that word, it does not end up highlighted in the editor.
I have added my VS extension to Github here
This really seems like this should be fairly straightforward but I have gone down many alleys on this one only to find dead-ends. Is there a simpler way to do this, or is my extension broken?
Update
Very strange. I just ran my extension again and although it does not highlight the text in the editor it highlights all instances of "Stream" in the popup text when you hover over a type/variable! Is there any way to get it to apply to the editor?

Depending on wether you are using Jetbrains Resharper or not you may write a plugin for that. That way you are able not only to add visual notification of IDisposable on a variable but also provide quickfixes if, and only if, it is not beeing called, which is what i am assuming you want to catch. Mind you that i can imagine that there's already a R# plugin for that. I know i've considered this too, but i was too lazy to write a plugin for that.
Don't get me wrong btw - If you're not using r# yet you should consider trying it out.
Among others you'd be working with this: API-QuickFix
There are also ways to define custom keywords, as resharper does, given by a custom markup and apply quickfixes to that.
PS: No i don't work at jetbrains. it's just that good :)
UPDATE:
potential VS Extension fix?
check this one out: MSDN Link Highlighting Text
I tried opening your github project but couldn't so i thought i'll just check msdn instead. it seems you are deriving from the wrong class to fulfill your needs?
MSDN keyword "Editors - Extending the Editor - Walkthrough: Highlighting Text"
I know SO wants code on the site, but msdn links going down is rather unlikely and with the given information the content can be found easily enough :)

I'm a bit late to the party, but hey, why not throw my 2 cents in.
As you've explained in your question, your project has two basic parts:
Finding the classes that implement IDisposable
Highlighting them
The first is by far the hardest, though not impossible. A word-list based approach is probably the simplest, though it should be possible with Roslyn to figure out on the fly which classes inherit IDisposible.
You could also always resort to loading the project's compiled .exe/.dll in the background after a build and figuring out what the types are there, but you'd still have to write some magic glue code to figure out what short class names in the code referred to what actual full-name classes in the assembly.
The second part, highlighting, is quite easy once you know how to do it (it helps that I've spent the last several months working full-time on extending VS). Of course, with Visual Studio, nothing is as simple as it looks (despite the efforts of Microsoft to try to make it user-friendly). So, I've built a sample extension that highlights just classes named "Stream" within C# files to get you started.
The relevant code is below, and the full project source is on GitHub). It starts with a classification-tagger provider:
[Export(typeof(ITaggerProvider))]
[ContentType("CSharp")]
[TagType(typeof(ClassificationTag))]
[Name("HighlightDisposableTagger")]
public class HighlightDisposableTaggerProvider : ITaggerProvider
{
[Import]
private IClassificationTypeRegistryService _classificationRegistry = null;
[Import]
private IClassifierAggregatorService _classifierAggregator = null;
private bool _reentrant;
public ITagger<T> CreateTagger<T>(ITextBuffer buffer) where T : ITag
{
if (_reentrant)
return null;
try {
_reentrant = true;
var classifier = _classifierAggregator.GetClassifier(buffer);
return new HighlightDisposableTagger(buffer, _classificationRegistry, classifier) as ITagger<T>;
}
finally {
_reentrant = false;
}
}
}
Then the tagger itself:
public class HighlightDisposableTagger : ITagger<ClassificationTag>
{
private const string DisposableFormatName = "HighlightDisposableFormat";
[Export]
[Name(DisposableFormatName)]
public static ClassificationTypeDefinition DisposableFormatType = null;
[Export(typeof(EditorFormatDefinition))]
[Name(DisposableFormatName)]
[ClassificationType(ClassificationTypeNames = DisposableFormatName)]
[UserVisible(true)]
public class DisposableFormatDefinition : ClassificationFormatDefinition
{
public DisposableFormatDefinition()
{
DisplayName = "Disposable Format";
ForegroundColor = Color.FromRgb(0xFF, 0x00, 0x00);
}
}
public event EventHandler<SnapshotSpanEventArgs> TagsChanged = delegate { };
private ITextBuffer _subjectBuffer;
private ClassificationTag _tag;
private IClassifier _classifier;
private bool _reentrant;
public HighlightDisposableTagger(ITextBuffer subjectBuffer, IClassificationTypeRegistryService typeService, IClassifier classifier)
{
_subjectBuffer = subjectBuffer;
var classificationType = typeService.GetClassificationType(DisposableFormatName);
_tag = new ClassificationTag(classificationType);
_classifier = classifier;
}
public IEnumerable<ITagSpan<ClassificationTag>> GetTags(NormalizedSnapshotSpanCollection spans)
{
if (_reentrant) {
return Enumerable.Empty<ITagSpan<ClassificationTag>>();
}
var tags = new List<ITagSpan<ClassificationTag>>();
try {
_reentrant = true;
foreach (var span in spans) {
if (span.IsEmpty)
continue;
foreach (var token in _classifier.GetClassificationSpans(span)) {
if (token.ClassificationType.IsOfType(/*PredefinedClassificationTypeNames.Identifier*/ "User Types")) {
// TODO: Somehow figure out if this refers to a class which implements IDisposable
if (token.Span.GetText() == "Stream") {
tags.Add(new TagSpan<ClassificationTag>(token.Span, _tag));
}
}
}
}
return tags;
}
finally {
_reentrant = false;
}
}
}
I've only tested this on VS2010, but it should work for VS2013 too (the only thing that might be different is the class classification name, but that's easy to discover with a well-placed breakpoint). I've never written an extension for VS2012, so I can't comment on that, but I know it's quite close to VS2013 in most respects.

So, one possible solution(I believe this one works):
1) Create your own content type which inherits from csharp.
2) Create new TextViewCreationListener which will swap out all "csharp" content types with your own one, thus potentially "disarming" all the other classifiers.
3) Register your classifier to handle your own content type.
Here is some of the code:
[Export(typeof(IVsTextViewCreationListener))]
[ContentType("csharp")]
[TextViewRole(PredefinedTextViewRoles.Editable)]
class TextViewCreationListener : IVsTextViewCreationListener {
internal readonly IVsEditorAdaptersFactoryService _adaptersFactory;
[Import] internal IContentTypeRegistryService ContentTypeRegistryService = null;
[ImportingConstructor]
public TextViewCreationListener(IVsEditorAdaptersFactoryService adaptersFactory) {
_adaptersFactory = adaptersFactory;
}
#region IVsTextViewCreationListener Members
public void VsTextViewCreated(VisualStudio.TextManager.Interop.IVsTextView textViewAdapter) {
var textView = _adaptersFactory.GetWpfTextView(textViewAdapter);
var myContent = ContentTypeRegistryService.GetContentType(MyContentType);
if(myContent == null)
{
ContentTypeRegistryService.AddContentType(MyContentType, new[] {"csharp"});
myContent = ContentTypeRegistryService.GetContentType(MyContentType);
}
// some kind of check if the content type is not already MyContentType.
textView.TextBuffer.ChangeContentType(myContent, null);
}
#endregion
}
And now, just modify your IClassifierProvider to register with your own content type, as such: [ContentType(MyContentType)]
Iin your own IClassifier, you can basically do your own calculation and once you think you can't handle the stuff, you could pass the control to other classifiers.
If you use MEF and import IClassifierAggregatorService, you can get a "MASTER-classifier" which will run all the logic for you. I haven't implemented it yet, but I've suggestes something similiar in the past, and it seemed to work. Alternatively you could maybe use [ImportMany] with List<IClassifier> and filter out the csharp ones?!

How to find dll files containing nunit tests

I have a folder with many dlls. One of them contains nunit tests (functions marked with [Test] attribute). I want to run nunit test from c# code. Is there any way to locate the right dll?
thank you

You can use Assembly.LoadFile method to load a DLL into an Assembly object. Then use the Assembly.GetTypes method to get all the types defined in the assembly. Then using the GetCustomAttributes method you can check if the type is decorated with the [TestFixture] attribute. If you want it quick 'n dirty, you could just call .GetType().ToString() on each attribute and check if the string contains "TestFixtureAttribute".
You can also check for the methods inside each type. Use the method Type.GetMethods to retrieve them, and use GetCustomAttributes on each of them, this time searching for "TestAttribute".

Just in case somebody needs working solution. As you can't unload assemblies, which were loaded this way, it's better to load them in another AppDomain.
public class ProxyDomain : MarshalByRefObject
{
public bool IsTestAssembly(string assemblyPath)
{
Assembly testDLL = Assembly.LoadFile(assemblyPath);
foreach (Type type in testDLL.GetTypes())
{
if (type.GetCustomAttributes(typeof(NUnit.Framework.TestFixtureAttribute), true).Length > 0)
{
return true;
}
}
return false;
}
}
AppDomainSetup ads = new AppDomainSetup();
ads.PrivateBinPath = Path.GetDirectoryName("C:\\some.dll");
AppDomain ad2 = AppDomain.CreateDomain("AD2", null, ads);
ProxyDomain proxy = (ProxyDomain)ad2.CreateInstanceAndUnwrap(typeof(ProxyDomain).Assembly.FullName, typeof(ProxyDomain).FullName);
bool isTdll = proxy.IsTestAssembly("C:\\some.dll");
AppDomain.Unload(ad2);

Invoking Method on Object Instantiated From DLL

I am having some trouble with assemblies and DLL's.
instrument_ is declared as an object and I'm creating an instance of "PP150" from the dll whose path is specified by path_.
string className = ContineoProperties.getSingleton().getClassName(path_);
assembly_ = Assembly.LoadFrom(path_);
Type classType = assembly_.GetType("Instrument." + className);
instrument_ = Activator.CreateInstance(classType);
Later I to call the method isntrument_.instrumentCommand(cmd.getCommandName())
The error I get is with when i call the method.
'object' does not contain a definition for 'instrumentCommand'
The isntrument_ is created fine. its just the method call that's giving me a problem. The method does exist in the "PP150.dll". Do I need some DLLImport to allow it to recognize it as a function?
Thanks,
P

If object type is not known in compile time,
To call a method defined on an object, you must use Reflection.
MethodInfo mInfo = classType.GetMethod("instrumentCommand");
mInfo.Invoke(instrument_, new Object[] { _parameters});

The compiler is never going to recognize the methods on a type that you are loading via reflection (e.g. using Assembly.GetType() and Activator.CreateInstance()). Unless you have the type metadata available at build time, you will always get that error if you try to call methods that are not defined on Object itself.
You have two options for making that kind of method call. Both of them require you to give up type safety, the only difference is the amount of work required. In both cases, if you make a mistake, the compiler will not tell you -- you will get a runtime exception instead.
Declare instrument_ as dynamic instead of object. This, obviously, only works in .NET 4.0, but it accomplishes exactly what you're trying to do. The method call will be dispatched at runtime, so as long as the instance that instrument_ references actually has a method call with the appropriate name, it will work.
Use reflection to call the method. You're already using reflection to load the type, so you are halfway there. You would need to add something like this:
// The array of types is the parameter list; assuming instrumentCommand takes
// a string it would look like this:
MethodInfo method = classType.GetMethod("instrumentCommand", new Type[] { typeof(string) });
method.Invoke(instrument_, new object[] { cmd.getCommandName() });

This happens because Activator.CreateInstance returns an object. I would create a separate DLL for the interface which is implemented by the class you want to instantiate. Both the DLL containing this class, and the executable should reference the DLL containing the interface. This way you could cast the object returned by Activator.CreateInstance to the interface, and call its methods:
IInstrument.dll:
interface IInstrument
{
void instrumentCommand(string cmd);
}
Instrument.dll (add IInstrument.dll as reference):
class Instrument : IInstrument
{
public void instrumentCommand(string cmd)
{
// ... implementation ...
}
}
InstrumentApp.exe (add IInstrument.dll as reference):
class Program
{
public static void Main()
{
// ... load Instrument.dll into assembly object ...
// ... load the type from the assembly ...
IInstrument instrument_ = (IInstrument)Activator.CreateInstance(classType);
instrument_.instrumentCommand(cmd.getCommandName());
}
}

The most simple thing would be to link agains PP150.
If you did link against the dll you must use Assembly.LoadFile or Assembly.Load and not LoadFrom because the last one will cause the assembly load to load your assembly in the LoadFrom loader context which will alter type identity.
Suppose you load the Type T from Assembly A via LoadFrom and you link against A as well.
object CreateTypeFrom()
{
var A = Assembly.LoadFrom(#"xxxx");
return A.CreateInstance("T");
}
void Test()
{
object t = CreateTypeFrom();
T RealT = new T(); // no prob
T Castedt = (T)t; // this will throw an InvalidCastException
T isNull = t as T; // this will result in a null instance
}
As you can see although you did create two times an instance of T they cannot be casted to due to different loader context which will make the type pretty useless.
To get rid of these things you could simply use Reflection to create a proxy type which will forward your calls to the proxy type. If you are using .NET 4 you can take advantage of the DLR to find the best matching methods at runtime. The code below creats a Version object and returns it as dynamic object. Then I do call the Major property to an integer and print it out to console. This does work with no exceptions nor compile time errors if you are using .NET 4 or later.
dynamic CreateTypeFrom()
{
var assembly = typeof(string).Assembly;
return assembly.CreateInstance("System.Version", true, BindingFlags.CreateInstance, null, new object[] { 1, 2, 3, 4 }, null, null);
}
[TestMethod]
public void Test()
{
var t = CreateTypeFrom();
int major = t.Major;
Console.WriteLine(major);
}