I have an unusual requirement for some code I am working on. I'm using an unreliable 3rd party library to do some barcode scanning (it stops working after running too many times). To get around this problem I decided to do the work in a separate AppDomain and then to unload the AppDomain when I am finished. This is a simplistic, but accurate, picture of what I am doing:
string domainID = Guid.NewGuid().ToString();
AppDomainSetup setup = new AppDomainSetup();
AppDomain domain = AppDomain.CreateDomain(domainID, null, setup);
string result = null;
try
{
domain.SetData("stream", stream);
domain.DoCallBack(ScanningContext.DoWork);
result = domain.GetData("result") as string;
}
finally
{
AppDomain.Unload(domain);
}
return result;
public static void DoWork()
{
Stream s = AppDomain.CurrentDomain.GetData("stream") as Stream;
ObjectHandle handle = AppDomain.CurrentDomain.CreateInstance("Scanning",
"Scanner");
Scanning.Scanner scanner = (Scanning.Scanner)handle.Unwrap();
Scanning.Result[] results = scanner.Scan(s);
AppDomain.CurrentDomain.SetData("result", results[0].Text);
}
"Scanner" is a wrapper class around the library I'm using. It sits in the "Scanning" assembly; a separate project just for this purpose.
ScanningContext.DoWork is a static method that sits in my service's assembly.
My problem with this method is that there is a memory leak some where. Memory keeps growing and growing (when this code is called, of course) until OutOfMemoryExceptions are thrown.
I can't find the leak any where. All of my streams are being disposed. All of my byte arrays are being nulled. I'm clearing lists, everything that has worked for me in the past. I'm about 90% confident that the leak is related to this AppDomain stuff. This is my first time using it so I'm probably doing something wrong.
I'm open to another approach besides AppDomains. I do require the ability to return results from the "Scanner" class, so spawning a process is not an option.
The AppDomain.Unload method starts a separate thread to unload the domain, which may fail for various reasons (threads executing unmanaged code is a problem). Here is a sample code that checks if the app domain is unloaded (taken from msdn docs):
try
{
Console.WriteLine();
// Note that the following statement creates an exception because the domain no longer exists.
Console.WriteLine("child domain: " + domain.FriendlyName);
}
catch (AppDomainUnloadedException e)
{
Console.WriteLine("The appdomain MyDomain does not exist.");
}
Related
I'm working with .NET Framework 4.8 and NuGet libraries: OpenCvSharp4, OpenCvSharp4.runtime.win, OpenCvSharp4.Windows and I'm trying to code a simple WebcamStreaming class that takes as input a WPF System.Windows.Controls.Image and exposes some Start and Stop methods to show the camera output.
I did it, and it works, but there are two things I'm not sure about:
CPU gets very high while streaming, around 30/40%. I think it is the VideoCapture read frames loop, so I put an await Task.Delay(10) to free some resources on each cycle and it is working. Now the CPU is always < 10%. But is this the correct solution? Am I doing something wrong earlier?
When I close the application, I get always this exception: Managed Debugging Assistant 'DisconnectedContext' : 'Transition into COM context 0x12f6648 for this RuntimeCallableWrapper failed with the following error: The object invoked has disconnected from its clients. (Exception from HRESULT: 0x80010108 (RPC_E_DISCONNECTED)). This is typically because the COM context 0x12f6648 where this RuntimeCallableWrapper was created has been disconnected or it is busy doing something else. Releasing the interfaces from the current COM context (COM context 0x12f6700). This may cause corruption or data loss. To avoid this problem, please ensure that all COM contexts/apartments/threads stay alive and are available for context transition, until the application is completely done with the RuntimeCallableWrappers that represents COM components that live inside them.'.
Do you know what is it and what I'm doing wrong?
The code
You can find the whole example on a GitHub repository I've created.
The interesting part of the frames fetch loop:
_previewTask = Task.Run(async () =>
{
using (var frame = new Mat())
{
while (!_cancellationTokenSource.IsCancellationRequested)
{
_videoCapture.Read(frame);
if (!frame.Empty())
{
_lastFrame = BitmapConverter.ToBitmap(frame);
var lastFrameBitmapImage = _lastFrame.ToBitmapSource();
lastFrameBitmapImage.Freeze();
_imageControlForRendering.Dispatcher.Invoke(() => _imageControlForRendering.Source = lastFrameBitmapImage);
await Task.Delay(10);
}
}
}
}, _cancellationTokenSource.Token);
The disposal process (I call it when the application gets closed):
public void Dispose()
{
_cancellationTokenSource?.Cancel();
if (_videoCapture?.IsDisposed == false)
{
_videoCapture?.Dispose();
_videoCapture = null;
}
_lastFrame?.Dispose();
}
Update for the second point
The problem was caused by the fact that the thread in which I was calling Dispose wasn't the same thread in which I created VideoCapture.
I'm running into an issue when using my V8Engine instance, it appears to have a small memory leak, and disposing of it, as well as forcing the garbage collection doesn't seem to help much. It will eventually throw an AccessViolationException on V8Enging local_m_negine = new V8Engine() claiming a Fatal error in heap setup, Allocation failed - process out of memory and Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Monitoring the program's memory usage through Task manager whilst running confirms that it is leaking memory, around 1000 KB every couple of seconds I think. I suspect it is the variables being declared within the executed script not being collected, or something to do with the GlobalObject.SetProperty method. Calling V8Engine.ForceV8GarbageCollection(), V8Engine.Dispose() and even GC.WaitForPendingFinalizers() & GC.Collect() doesn't prevent this memory being leaked (Although it is worth noting that it seems to leak it slower with these commands in place, and I know I shouldn't use GC but it was there as a last resort to see if it would fix the issue.)
A tangential issue that could also provide a solution is the inability to clear the execution context for V8Engine. I am required to dispose and re-instantiate the engine for each script, which I believe is where the memory leak is happening, otherwise I run into issues where variables have already been declared, causing V8Engine.Execute() to throw an exception saying such.
I can definitely confirm that the memory leak is something to do with the V8Engine Implementation, as running the older version of this program that uses Microsoft.JScript has no such memory leak, and the memory used remains consistent.
The affected code is as follows;
//Create the V8Engine and dispose when done
using (V8Engine local_m_engine = new V8Engine())
{
//Set the Lookup instance as a global object so that the JS code in the V8.Net wrapper can access it
local_m_engine.GlobalObject.SetProperty("Lookup", m_lookup, null, true, ScriptMemberSecurity.ReadOnly);
//Execute the script
result = local_m_engine.Execute(script);
//Please just clear everything I can't cope.
local_m_engine.ForceV8GarbageCollection();
local_m_engine.GlobalObject.Dispose();
}
EDIT:
Not sure how useful this will be but I've been running some memory profiling tools on it and have learnt that after running an isolated version of the original code, My software ends up with a large amount of instances of IndexedObjectList's full of null values (see here: http://imgur.com/a/bll5K). It appears to have one instance of each class for each V8Engine instance that is made, but they aren't being disposed or freed. I cant help but feel like I'm missing a command or something here.
The code I'm using to test and recreate the memory leak that the above implementation causes is as follows:
using System;
using V8.Net;
namespace V8DotNetMemoryTest
{
class Program
{
static void Main(string[] args)
{
string script = #" var math1 = 5;
var math2 = 10;
result = 5 + 10;";
Handle result;
int i = 0;
V8Engine local_m_engine;
while (true)
{
//Create the V8Engine and dispose when done
local_m_engine = new V8Engine();
//Set the Lookup instance as a global object so that the JS code in the V8.Net wrapper can access it
//local_m_engine.GlobalObject.SetProperty("Lookup", m_lookup, null, true, ScriptMemberSecurity.ReadOnly);
//Execute the script
result = local_m_engine.Execute(script);
Console.WriteLine(i++);
result.ReleaseManagedObject();
result.Dispose();
local_m_engine.Dispose();
GC.WaitForPendingFinalizers();
GC.Collect();
local_m_engine = null;
}
}
}
}
Sorry, I had no idea this question existed. Make sure to use the v8.net tag.
Your problem is this line:
result = local_m_engine.Execute(script);
The result returned is never disposed. ;) You are responsible for returned handles. Those handles are struct values, not class objects.
You could also do using (result = local_m_engine.Execute(script)) { ... }
There is a new version released. I am finally resurrecting this project again as I will need it for the FlowScript VPL project - and it now supports .Net Standard as well for cross-platform support!
I have a method that was created by someone else and do not have access to the source. This Method creates a PDF document on the file system, but I'm having issues with memory when I start passing it 200+ images.
From what I can tell it loads the object into memory in order to render the PDF, and then outputs to the file system. When I use a for or foreach loop to interate through my files and output them I can see in task manager the memory is building slowly, it does dip from time to time, but in general it's always on the way up.
Here's my code:
for(int i = 0; i < Documents.Count; i++)
{
IDocument document = Documents[i];
try
{
string exportFileName = string.Format("{0}_{1}_{2}_{3}", batchName, document.Field("NHS File\\SectionName").Text, document.Field("NHS File\\Classification Result").Text, dateTime);
string imageFileName = exportFileName;
int tries = 0;
while (File.Exists(Path.Combine(processingLocation, imageFileName + ".pdf")))
{
imageFileName = string.Format("{0}_{1}", exportFileName, tries++);
}
document.SaveAs((Path.Combine(processingLocation, imageFileName)), ExportOptions.GetImageOptions());
}
catch(Exception ex)
{
Processing.ReportError(ex.ToString());
return;
}
finally
{
Marshal.ReleaseComObject(document);
Marshal.FinalReleaseComObject(document);
document = null;
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
GC.WaitForPendingFinalizers();
}
}
The issue is with the 'document.SaveAs();'
I've tried calling the garbage collectors and release COM objects, adding and removing memory pressure, but nothing seems to work I continually get "System.Runtime.InteropServices.COMException (0x80004005): Not enough memory!" when it gets to 200 files.
There seems to be no close(), Dispose(), or Finalise() options with this.
Is there a way of calling this document.SaveAs() so that the memory used by the object is controlled and released by me?
Thanks in advance.
If you have no control over the 3rd party code, you have little options to clean up the memory. The nature of the code seems to imply that it is heavy in COM Interop or platform invoke, which are a sure-fire way to leak memory if done incorrectly. GC.Collect doesn't collect natively allocated memory since it is outside of the garbage collector's knowledge.
I would recommend putting your logic in a separate process, and when the processing is complete, letting the process exit. Exiting a process is a very reliable way to clean up resources and memory.
Move your code into a console app, and use Process.Start to start your new command line utility from your current code. Use WaitForExit if you want to do is synchronously, and you can pass in arguments to the process such as the path of files or directories to process.
I have a library that I use that uses WCF to call an http service to get settings. Normally the first call takes ~100 milliseconds and subsequent calls takes only a few milliseconds. But I have found that when I create a new AppDomain the first WCF call from that AppDomain takes over 2.5 seconds.
Does anyone have an explanation or fix for why the first creation of a WCF channel in a new AppDomain would take so long?
These are the benchmark results(When running without debugger attached in release in 64bit), notice how in the second set of numbers the first connections takes over 25x longer
Running in initial AppDomain
First Connection: 92.5018 ms
Second Connection: 2.6393 ms
Running in new AppDomain
First Connection: 2457.8653 ms
Second Connection: 4.2627 ms
This isn't a complete example but shows most of how I produced these numbers:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Running in initial AppDomain");
new DomainRunner().Run();
Console.WriteLine();
Console.WriteLine("Running in new thread and AppDomain");
DomainRunner.RunInNewAppDomain("test");
Console.ReadLine();
}
}
class DomainRunner : MarshalByRefObject
{
public static void RunInNewAppDomain(string runnerName)
{
var newAppDomain = AppDomain.CreateDomain(runnerName);
var runnerProxy = (DomainRunner)newAppDomain.CreateInstanceAndUnwrap(typeof(DomainRunner).Assembly.FullName, typeof(DomainRunner).FullName);
runnerProxy.Run();
}
public void Run()
{
AppServSettings.InitSettingLevel(SettingLevel.Production);
var test = string.Empty;
var sw = Stopwatch.StartNew();
test += AppServSettings.ServiceBaseUrlBatch;
Console.WriteLine("First Connection: {0}", sw.Elapsed.TotalMilliseconds);
sw = Stopwatch.StartNew();
test += AppServSettings.ServiceBaseUrlBatch;
Console.WriteLine("Second Connection: {0}", sw.Elapsed.TotalMilliseconds);
}
}
The call to AppServSettings.ServiceBaseUrlBatch is creating a channel to a service and calling a single method. I have used wireshark to watch the call and it only takes a milliseconds to get a response from the service. It creates the channel with the following code:
public static ISettingsChannel GetClient()
{
EndpointAddress address = new EndpointAddress(SETTINGS_SERVICE_URL);
BasicHttpBinding binding = new BasicHttpBinding
{
MaxReceivedMessageSize = 1024,
OpenTimeout = TimeSpan.FromSeconds(2),
SendTimeout = TimeSpan.FromSeconds(5),
ReceiveTimeout = TimeSpan.FromSeconds(5),
ReaderQuotas = { MaxStringContentLength = 1024},
UseDefaultWebProxy = false,
};
cf = new ChannelFactory<ISettingsChannel>(binding, address);
return cf.CreateChannel();
}
From profiling the app it shows that in the first case constructing the channel factory and creating the channel and calling the method takes less than 100 milliseconds
In the new AppDomain constructing the channel factory took 763 milliseconds, 521 milliseconds to create the channel, 1,098 milliseconds to call the method on the interface.
TestSettingsRepoInAppDomain.DomainRunner.Run() 2,660.00
TestSettingsRepoInAppDomain.AppServSettings.get_ServiceBaseUrlBatch() 2,543.47
Tps.Core.Settings.Retriever.GetSetting(string,!!0,!!0,!!0) 2,542.66
Tps.Core.Settings.Retriever.TryGetSetting(string,!!0&) 2,522.03
Tps.Core.Settings.ServiceModel.WcfHelper.GetClient() 1,371.21
Tps.Core.Settings.ServiceModel.IClientChannelExtensions.CallWithRetry(class System.ServiceModel.IClientChannel) 1,098.83
EDIT
After using perfmon with the .NET CLR Loading object I can see that when it loads the second AppDomain it is loading way more classes into memory than it does initially. The first flat line is a pause I put in after the first appdomain, there it has 218 classes loaded. The second AppDomain causes 1,944 total classes to be loaded.
I assume its the loading of all these classes that is taking up all of the time, so now the question is, what classes is it loading and why?
UPDATE
The answer turns out to be because of the fact that only one AppDomain is able to take advantage of the native image system dlls. So the slowness in the second appdomain was it having to rejit all of the System.* dlls used by wcf. The first appdomain could use the pre ngened native versions of those dlls, so it didn't have the same startup cost.
After investigating the LoaderOptimizationAttribute that Petar suggested, that indeed seemed to fix the issue, using either MultiDomain or MultiDomainHost results in the second AppDomain to take the same amount of time as the first time to access stuff over wcf
Here you can see the default option, note how in the second AppDomain none of the assemblies say Native, meaning they all had to be rejitted, which is what was taking all of the time
Here is after adding the LoaderOptimization(LoaderOptimization.MultiDomain) to Main. You can see that everything is loaded into the shared AppDomain
Here is after user LoaderOptimization(LoaderOptimization.MultiDomainHost) to main. You can see that all system dlls are shared, but my own dlls and any not in the GAC are loaded seperately into each AppDomain
So for the service that prompted this question using MultiDomainHost is the answer, because it has fast startup time and I can unload AppDomains to remove the dynamically built assemblies that the service uses
You can decorate your Main with LoaderOptimization attribute to tell the CLR loader how to load classes.
[LoaderOptimization(LoaderOptimization.MultiDomain)]
MultiDomain - Indicates that the application will probably have many domains that use the same code, and the loader must share maximal internal resources across application domains.
Do you have an HTTP proxy defined in IE? (maybe an auto configure script). This can be a cause.
Otherwise I would guess it is the time that takes to load all the dlls. Try to deparate the proxy creation from the actull call to the service, to see what's taking the time.
I found the following article that talks about how only the first AppDomain can use native image dlls, so a child appdomain will always be forced to JIT lots of stuff that the initial AppDomain doesn't have to. This could lead to the performancce impact I am seeing, but would it be possible to somehow not get this performance penalty?
If there is a native image for the assembly, only the first AppDomain
can use the native image. All other AppDomains will have to
JIT-compile the code which can result in a significant CPU cost.
Background
I have a Windows service that uses various third-party DLLs to perform work on PDF files. These operations can use quite a bit of system resources, and occasionally seem to suffer from memory leaks when errors occur. The DLLs are managed wrappers around other unmanaged DLLs.
Current Solution
I'm already mitigating this issue in one case by wrapping a call to one of the DLLs in a dedicated console app and calling that app via Process.Start(). If the operation fails and there are memory leaks or unreleased file handles, it doesn't really matter. The process will end and the OS will recover the handles.
I'd like to apply this same logic to the other places in my app that use these DLLs. However, I'm not terribly excited about adding more console projects to my solution, and writing even more boiler-plate code that calls Process.Start() and parses the output of the console apps.
New Solution
An elegant alternative to dedicated console apps and Process.Start() seems to be the use of AppDomains, like this: http://blogs.geekdojo.net/richard/archive/2003/12/10/428.aspx
I've implemented similar code in my application, but the unit tests have not been promising. I create a FileStream to a test file in a separate AppDomain, but don't dispose it. I then attempt to create another FileStream in the main domain, and it fails due to the unreleased file lock.
Interestingly, adding an empty DomainUnload event to the worker domain makes the unit test pass. Regardless, I'm concerned that maybe creating "worker" AppDomains won't solve my problem.
Thoughts?
The Code
/// <summary>
/// Executes a method in a separate AppDomain. This should serve as a simple replacement
/// of running code in a separate process via a console app.
/// </summary>
public T RunInAppDomain<T>( Func<T> func )
{
AppDomain domain = AppDomain.CreateDomain ( "Delegate Executor " + func.GetHashCode (), null,
new AppDomainSetup { ApplicationBase = Environment.CurrentDirectory } );
domain.DomainUnload += ( sender, e ) =>
{
// this empty event handler fixes the unit test, but I don't know why
};
try
{
domain.DoCallBack ( new AppDomainDelegateWrapper ( domain, func ).Invoke );
return (T)domain.GetData ( "result" );
}
finally
{
AppDomain.Unload ( domain );
}
}
public void RunInAppDomain( Action func )
{
RunInAppDomain ( () => { func (); return 0; } );
}
/// <summary>
/// Provides a serializable wrapper around a delegate.
/// </summary>
[Serializable]
private class AppDomainDelegateWrapper : MarshalByRefObject
{
private readonly AppDomain _domain;
private readonly Delegate _delegate;
public AppDomainDelegateWrapper( AppDomain domain, Delegate func )
{
_domain = domain;
_delegate = func;
}
public void Invoke()
{
_domain.SetData ( "result", _delegate.DynamicInvoke () );
}
}
The unit test
[Test]
public void RunInAppDomainCleanupCheck()
{
const string path = #"../../Output/appdomain-hanging-file.txt";
using( var file = File.CreateText ( path ) )
{
file.WriteLine( "test" );
}
// verify that file handles that aren't closed in an AppDomain-wrapped call are cleaned up after the call returns
Portal.ProcessService.RunInAppDomain ( () =>
{
// open a test file, but don't release it. The handle should be released when the AppDomain is unloaded
new FileStream ( path, FileMode.Open, FileAccess.ReadWrite, FileShare.None );
} );
// sleeping for a while doesn't make a difference
//Thread.Sleep ( 10000 );
// creating a new FileStream will fail if the DomainUnload event is not bound
using( var file = new FileStream ( path, FileMode.Open, FileAccess.ReadWrite, FileShare.None ) )
{
}
}
Application domains and cross-domain interaction is a very thin matter, so one should make sure he really understands how thing work before doing anything... Mmm... Let's say, "non-standard" :-)
First of all, your stream-creating method actually executes on your "default" domain (surprise-surprise!). Why? Simple: the method that you pass into AppDomain.DoCallBack is defined on an AppDomainDelegateWrapper object, and that object exists on your default domain, so that is where its method gets executed. MSDN doesn't say about this little "feature", but it's easy enough to check: just set a breakpoint in AppDomainDelegateWrapper.Invoke.
So, basically, you have to make do without a "wrapper" object. Use static method for DoCallBack's argument.
But how do you pass your "func" argument into the other domain so that your static method can pick it up and execute?
The most evident way is to use AppDomain.SetData, or you can roll your own, but regardless of how exactly you do it, there is another problem: if "func" is a non-static method, then the object that it's defined on must be somehow passed into the other appdomain. It may be passed either by value (whereas it gets copied, field by field) or by reference (creating a cross-domain object reference with all the beauty of Remoting). To do former, the class has to be marked with a [Serializable] attribute. To do latter, it has to inherit from MarshalByRefObject. If the class is neither, an exception will be thrown upon attempt to pass the object to the other domain. Keep in mind, though, that passing by reference pretty much kills the whole idea, because your method will still be called on the same domain that the object exists on - that is, the default one.
Concluding the above paragraph, you are left with two options: either pass a method defined on a class marked with a [Serializable] attribute (and keep in mind that the object will be copied), or pass a static method. I suspect that, for your purposes, you will need the former.
And just in case it has escaped your attention, I would like to point out that your second overload of RunInAppDomain (the one that takes Action) passes a method defined on a class that isn't marked [Serializable]. Don't see any class there? You don't have to: with anonymous delegates containing bound variables, the compiler will create one for you. And it just so happens that the compiler doesn't bother to mark that autogenerated class [Serializable]. Unfortunate, but this is life :-)
Having said all that (a lot of words, isn't it? :-), and assuming your vow not to pass any non-static and non-[Serializable] methods, here are your new RunInAppDomain methods:
/// <summary>
/// Executes a method in a separate AppDomain. This should serve as a simple replacement
/// of running code in a separate process via a console app.
/// </summary>
public static T RunInAppDomain<T>(Func<T> func)
{
AppDomain domain = AppDomain.CreateDomain("Delegate Executor " + func.GetHashCode(), null,
new AppDomainSetup { ApplicationBase = Environment.CurrentDirectory });
try
{
domain.SetData("toInvoke", func);
domain.DoCallBack(() =>
{
var f = AppDomain.CurrentDomain.GetData("toInvoke") as Func<T>;
AppDomain.CurrentDomain.SetData("result", f());
});
return (T)domain.GetData("result");
}
finally
{
AppDomain.Unload(domain);
}
}
[Serializable]
private class ActionDelegateWrapper
{
public Action Func;
public int Invoke()
{
Func();
return 0;
}
}
public static void RunInAppDomain(Action func)
{
RunInAppDomain<int>( new ActionDelegateWrapper { Func = func }.Invoke );
}
If you're still with me, I appreciate :-)
Now, after spending so much time on fixing that mechanism, I am going to tell you that is was purposeless anyway.
The thing is, AppDomains won't help you for your purposes. They only take care of managed objects, while unmanaged code can leak and crash all it wants. Unmanaged code doesn't even know there are such things as appdomains. It only knows about processes.
So, in the end, your best option remains your current solution: just spawn another process and be happy about it. And, I would agree with the previous answers, you don't have to write another console app for each case. Just pass a fully qualified name of a static method, and have the console app load your assembly, load your type, and invoke the method. You can actually package it pretty neatly in a very much the same way as you tried with AppDomains. You can create a method called something like "RunInAnotherProcess", which will examine the argument, get the full type name and method name out of it (while making sure the method is static) and spawn the console app, which will do the rest.
You don't have to create many console applications, you can create a single application that will receive as parameter the full qualified type name. The application will load that type and execute it.
Separating everything into tiny processes is the best method to really dispose all the resources. An application domain cannot do full resources disposing, but a process can.
Have you considered opening a pipe between the main application and the sub applications? This way you could pass more structured information between the two applications without parsing standard output.