For reasons of memory isolation and stability I need to execute some methods/classes as sub-processes, not threads.
I found the following library that allows me to do just that: https://github.com/tmds/Tmds.ExecFunction.
However, I can't seem to find a way to get the process ID of the child process.
ExecFunction.Run(() => new NewSubProcessClass());
The code above will generate a new process and will even generate console output in a Linux Docker container. The problem is, that I have no process ID.
public static Process Start(Action action, Action<ExecFunctionOptions> configure = null);
The method I quoted above looked like it should do the trick, however, when I replace
ExecFunction.Run(() => new NewSubProcessClass());
with
Process process = ExecFunction.Start(() => new NewSubProcessClass());
I no longer get any console output.
UPDATE 0:
The suggestion from #bazzilic
process.WaitForExit();
is the solution to the problem.
If you examine the source code for Tmds.ExecFunction, both methods .Run(...) (here) and .Start(...) (here) under the hood call the same private method .Start(...) (source here). The two differences are:
the public .Start(...) method returns the process object from the tuple returned by the private .Start(...) method whereas .Run(...) does not;
the .Run(...) method also supplies waitForExit: true in the parameters.
In the code of the private .Start(...) method, there is this portion:
if (waitForExit)
{
process.WaitForExit();
}
Other than that, the two public methods are identical. The difference in your code behavior is most likely due to not waiting for the child process to finish in your parent process. Try this:
Process process = ExecFunction.Start(() => new NewSubProcessClass());
process.WaitForExit();
PS: Keep in mind that Process is IDisposable, so might be better to do
using ( Process process = ExecFunction.Start(() => new NewSubProcessClass()) ) {
process.WaitForExit();
}
Related
I am facing an odd behaviour of the Process class when accessing MainWindowHandle in .NET Core (3.1).
Consider the following function:
bool TestProcess()
{
var process = Process.Start("notepad");
try
{
for (var attempt = 0; attempt < 20; ++attempt)
{
process?.Refresh();
if (process?.MainWindowHandle != IntPtr.Zero)
{
return true;
}
Thread.Sleep(100);
}
return false;
}
finally
{
process?.Kill();
}
}
If the function is run in .NET Framework, it returns true as I would expect. However, when using .NET Core (3.1 in my case), it returns false instead.
Now the part that is puzzling me even more:
If I set a breakpoint anywhere after the process is started but before the MainWindowHandle property is read (or if I simply step over the code at least once between those lines), the function returns true.
If I set a breakpoint after the MainWindowHandle property is read, the function will return false. At that point, it does not matter anymore if I step over the code, set more breakpoints, etc.; the result is always false.
What could be going on and how could I fix it?
A few more details that may or may not be relevant:
The same issue can occur with other processes as long as they have a GUI (I originally discovered it with a WPF app). Try dfrgui instead for example.
Some processes such as calc seem to spawn a separate process for the actual GUI, so the behaviour changes slightly:
In .NET Core, the function still returns false, but the GUI remains open, and the breakpoint trick does not work anymore.
In .NET Framework, an InvalidOperationException is thrown in the MainWindowHandle line due to the process having exited already.
I am using Visual Studio 2019 (16.4.5), ReSharper 2019.3.2, .NET Core SDK 3.1.101 and Windows 10 (Build 18363).
The breakpoint trick also works in Rider (2019.3.3). However, only if you leave enough time for the main window to appear before resuming the program execution. This might also be the case in Visual Studio, but the IDE reacts too slowly for me to test.
I am guessing that the debugger is altering the behaviour of the program somehow; perhaps accidentally when listing all the process properties, or perhaps it has to do with the threads it uses. But how exactly? Could I replicate that same behaviour?
Some things that I already tried but did not work:
Increasing the number of attempts or the sleep time between attempts
Reordering the Refresh(), Thread.Sleep() and MainWindowHandle lines
Replacing the Thread.Sleep() call with await Task.Delay() (and making the function async)
Start the process with UseShellExecute set to true / false explicitly
Using [MTAThread] or [STAThread] attributes
Printing all the process properties via reflection after it is created / refreshed
Even if this did not work, perhaps the debugger reads those properties in a different way / order, so this might still be why debugging makes a difference.
As a bit of background, I came across this issue when I added UI tests to my WPF application using FlaUI (see the issue I created). I am quite sure now that it is not a problem of the library itself; it just happens to use Process.MainWindowHandle for some of its methods.
It turns out this was caused by an issue with .NET Core itself. The MainWindowHandle property would not be re-evaluated after the first attempt, no matter if it returned IntPtr.Zero.
When setting breakpoints, the only thing I was achieving was to delay the moment in which MainWindowHandle was being read. I could have achieved the same with a longer Thread.Sleep() call before that. In fact, 100 milliseconds was actually enough for Notepad in my case, but for the original WPF app I was testing I need about 1 second. Perhaps I was being too wary about debuggers in general.
I already submitted a pull request to fix this. In the meantime, if anybody is also affected by something similar, I would recommend replacing any Refresh() call with process = Process.GetProcessById(process.Id). This will return a new Process instance pointing to the same process, and hence the MainWindowHandle property can be re-evaluated without issues.
In my original example, it would look like this (reordered a bit to avoid an initial instance creation):
bool TestProcess()
{
var process = Process.Start("notepad");
try
{
for (var attempt = 0; attempt < 20; ++attempt)
{
if (process?.MainWindowHandle != IntPtr.Zero)
{
return true;
}
Thread.Sleep(100);
process = Process.GetProcessById(process.Id);
}
return false;
}
finally
{
process?.Kill();
}
}
My testing seems to suggest that Process.Start() at its bare bones costs ~15-27ms (depending on if you use CreateNoWindow, which ADDS ~10ms). In this question, I'm not trying to say the code I'm running is slow, but the actual act of starting a process, even if it does nothing.
I got these numbers by using a stopwatch on a console app that ran another console app literally just returning in its main method.
namespace RunNothing
{
class Program
{
static void Main(string[] args)
{
var startInfo = new ProcessStartInfo(#"C:\Users\Noggog\Documents\visual studio 2017\Projects\DoNothing\DoNothing\bin\Release\DoNothing.exe")
{
CreateNoWindow = true,
UseShellExecute = false,
};
var sw = new Stopwatch();
var proc = new Process()
{
StartInfo = startInfo,
EnableRaisingEvents = true
};
proc.Exited += (sender, a) =>
{
sw.Stop();
System.Console.WriteLine(sw.ElapsedMilliseconds);
};
sw.Start();
proc.Start();
System.Console.ReadLine();
}
}
}
So my question is whether anyone knows a good way to improve that startup time per Process.Start() call?
For background, I have an app that will be starting various .exes, most of which should do a quick bool check and short circuit out, asap. Every so often one of the calls will do a longer operation, but usually it will do nothing. Right now a ~15-27ms call per go is getting a bit heavy for my use case.
Edit:
Some people were asking for more details of the end usage. This is the project that drove the question. The end usage is an extension of git hooks to provide more hookable commands, and provide convenience features such as calling an exe in response to hooks being fired. One of the modes is that a single exe can handle multiple hooks, and decide which ones it would respond to. In that scenario, every git command would "check in" with the exe to see if it wanted to do any logic. This is where the Process.Start() time adds up. If you have 6 repos and your git client is initializing things by running several commands a pre and post hook combo can be 27ms (proc start time) * 2(pre/post) * X(commands) * 6(repos) = ~2+ seconds. Potentially none of these commands are ones needing response, but it's already added several seconds of sluggishness to the system.
I have a simple console app that writes out status of various resources in a nicely formatted way.
This app is using some 3rd party components to connect to places.
Those components unfortunately do a lot Console.Writes (or some sort of logging) which all ends up intertwined with my messages.
I tried redirecting Console output (in hopes that i could filter non-mine messages), but that seems to work ONLY on my messages.
var sb = new StringBuilder();
TextWriter tw = new StringWriter(sb);
Console.SetOut(tw);
So this works in redirecting console writes, but only on the ones i did.. output from 3rd party components is still streaming to my screen.. any other ways to supress that?
The reason you're not able to redirect the output of the third party components is because you're calling redirect on your process, not theirs. To do this, loop over the processes, find the ones that are writing to the console and redirect their output.
using System.Diagnostics;
Process[] processlist = Process.GetProcesses();
foreach(Process theprocess in processlist){
// you'll actually need to use something like the process name or id here.
if (theprocess == MyThirdPartyComponentsProcess)
{
theprocess.StartInfo.UseShellExecute = false;
theprocess.StartInfo.RedirectStandardOutput = true;
}
}
You'll have to be running as administrator for this work.
I've thought of one fairly hacky way:
Capture the existing Console.Out so you can reuse it in your "real" writer
Create your own TextWriter implementation, which...
... when it's asked to write, checks whether the calling code is in your assembly. See Assembly.GetCallingAssembly
So long as you don't have inlining problems, you should be able to check the context, and drop any output you don't want. You'd probably have to call Console.Out.WriteLine directly instead of Console.WriteLine, mind you - otherwise the caller will be the Console class.
A better solution would be to change your code to use log4net or something similar, so you could get your output in a more configurable way, not on the console - assuming your own use is also logging. (If it's for interactive output, something like the above may be the only way...)
Since, there doesn't seem to be a reasonable quick way to suppress these, I went with plan B, that is deferring my messages to the end. This way all the logging gets through, and than i write out all the information i am interested in.
I broke it down into Success/Warn/Error methods
public static void WriteError(string message, params string[] args) {
writerTasks.Add(() => {
var c = Console.ForegroundColor;
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine(message, args);
Console.ForegroundColor = c;
});
}
and at the end of the execution, i call this method to flush it out.
private static void WriteResults() {
foreach (var t in writerTasks)
t();
}
i do it this way, because i am changing text color depending on message type, which is contained within each action.
in my case this works great, since the information scrolls up hiding all the logging.. which in fact is also beneficial.. just didn't want it dispersed between the main output
I'm in to a loop defined dynamically the function that will run onClick of control.
the function is the following:
public static void TryOpenFile(string filename, EventHandler callback)
{
Process proc;
proc = Process.Start(filename);
if (callback != null)
{
proc.EnableRaisingEvents = true;
proc.Exited += (a, b) =>
{
callback(a, b);
};
}
}
And then:
for(int i = 0; i < numberOfControls; i++)
{
controlImg.SetFileToOpen(file,
delegate
{
//exited!
});
}
Looks like your filename points to an already running process. As per the MSDN documentation:
Return Value
Type: System.Diagnostics.Process
A new Process component
that is associated with the process resource, or null, if no process
resource is started (for example, if an existing process is reused).
Update: If your filename is a bad filename, it will obviously throw an Exception.
The documentation for this overload of Process.Start explains what is happening (emphasis mine):
Use this overload to start a process resource by specifying its file
name. The overload associates the resource with a new Process
component. If the process is already running, no additional process
resource is started. Instead, the existing process resource is reused
and no new Process component is created. In such a case, instead of
returning a new Process component, Start returns null to the calling
procedure.
A new process may not be started if you are using ShellExecute to start a file using it's association, rather than running an executable. For instance, if filename is (for example) "C:\Test.xls", it might start Excel. But if Excel was already running, it might open the file in the existing running instance, rather than starting a new process. In that case, the value of proc would be null.
Background
I have a Windows service that uses various third-party DLLs to perform work on PDF files. These operations can use quite a bit of system resources, and occasionally seem to suffer from memory leaks when errors occur. The DLLs are managed wrappers around other unmanaged DLLs.
Current Solution
I'm already mitigating this issue in one case by wrapping a call to one of the DLLs in a dedicated console app and calling that app via Process.Start(). If the operation fails and there are memory leaks or unreleased file handles, it doesn't really matter. The process will end and the OS will recover the handles.
I'd like to apply this same logic to the other places in my app that use these DLLs. However, I'm not terribly excited about adding more console projects to my solution, and writing even more boiler-plate code that calls Process.Start() and parses the output of the console apps.
New Solution
An elegant alternative to dedicated console apps and Process.Start() seems to be the use of AppDomains, like this: http://blogs.geekdojo.net/richard/archive/2003/12/10/428.aspx
I've implemented similar code in my application, but the unit tests have not been promising. I create a FileStream to a test file in a separate AppDomain, but don't dispose it. I then attempt to create another FileStream in the main domain, and it fails due to the unreleased file lock.
Interestingly, adding an empty DomainUnload event to the worker domain makes the unit test pass. Regardless, I'm concerned that maybe creating "worker" AppDomains won't solve my problem.
Thoughts?
The Code
/// <summary>
/// Executes a method in a separate AppDomain. This should serve as a simple replacement
/// of running code in a separate process via a console app.
/// </summary>
public T RunInAppDomain<T>( Func<T> func )
{
AppDomain domain = AppDomain.CreateDomain ( "Delegate Executor " + func.GetHashCode (), null,
new AppDomainSetup { ApplicationBase = Environment.CurrentDirectory } );
domain.DomainUnload += ( sender, e ) =>
{
// this empty event handler fixes the unit test, but I don't know why
};
try
{
domain.DoCallBack ( new AppDomainDelegateWrapper ( domain, func ).Invoke );
return (T)domain.GetData ( "result" );
}
finally
{
AppDomain.Unload ( domain );
}
}
public void RunInAppDomain( Action func )
{
RunInAppDomain ( () => { func (); return 0; } );
}
/// <summary>
/// Provides a serializable wrapper around a delegate.
/// </summary>
[Serializable]
private class AppDomainDelegateWrapper : MarshalByRefObject
{
private readonly AppDomain _domain;
private readonly Delegate _delegate;
public AppDomainDelegateWrapper( AppDomain domain, Delegate func )
{
_domain = domain;
_delegate = func;
}
public void Invoke()
{
_domain.SetData ( "result", _delegate.DynamicInvoke () );
}
}
The unit test
[Test]
public void RunInAppDomainCleanupCheck()
{
const string path = #"../../Output/appdomain-hanging-file.txt";
using( var file = File.CreateText ( path ) )
{
file.WriteLine( "test" );
}
// verify that file handles that aren't closed in an AppDomain-wrapped call are cleaned up after the call returns
Portal.ProcessService.RunInAppDomain ( () =>
{
// open a test file, but don't release it. The handle should be released when the AppDomain is unloaded
new FileStream ( path, FileMode.Open, FileAccess.ReadWrite, FileShare.None );
} );
// sleeping for a while doesn't make a difference
//Thread.Sleep ( 10000 );
// creating a new FileStream will fail if the DomainUnload event is not bound
using( var file = new FileStream ( path, FileMode.Open, FileAccess.ReadWrite, FileShare.None ) )
{
}
}
Application domains and cross-domain interaction is a very thin matter, so one should make sure he really understands how thing work before doing anything... Mmm... Let's say, "non-standard" :-)
First of all, your stream-creating method actually executes on your "default" domain (surprise-surprise!). Why? Simple: the method that you pass into AppDomain.DoCallBack is defined on an AppDomainDelegateWrapper object, and that object exists on your default domain, so that is where its method gets executed. MSDN doesn't say about this little "feature", but it's easy enough to check: just set a breakpoint in AppDomainDelegateWrapper.Invoke.
So, basically, you have to make do without a "wrapper" object. Use static method for DoCallBack's argument.
But how do you pass your "func" argument into the other domain so that your static method can pick it up and execute?
The most evident way is to use AppDomain.SetData, or you can roll your own, but regardless of how exactly you do it, there is another problem: if "func" is a non-static method, then the object that it's defined on must be somehow passed into the other appdomain. It may be passed either by value (whereas it gets copied, field by field) or by reference (creating a cross-domain object reference with all the beauty of Remoting). To do former, the class has to be marked with a [Serializable] attribute. To do latter, it has to inherit from MarshalByRefObject. If the class is neither, an exception will be thrown upon attempt to pass the object to the other domain. Keep in mind, though, that passing by reference pretty much kills the whole idea, because your method will still be called on the same domain that the object exists on - that is, the default one.
Concluding the above paragraph, you are left with two options: either pass a method defined on a class marked with a [Serializable] attribute (and keep in mind that the object will be copied), or pass a static method. I suspect that, for your purposes, you will need the former.
And just in case it has escaped your attention, I would like to point out that your second overload of RunInAppDomain (the one that takes Action) passes a method defined on a class that isn't marked [Serializable]. Don't see any class there? You don't have to: with anonymous delegates containing bound variables, the compiler will create one for you. And it just so happens that the compiler doesn't bother to mark that autogenerated class [Serializable]. Unfortunate, but this is life :-)
Having said all that (a lot of words, isn't it? :-), and assuming your vow not to pass any non-static and non-[Serializable] methods, here are your new RunInAppDomain methods:
/// <summary>
/// Executes a method in a separate AppDomain. This should serve as a simple replacement
/// of running code in a separate process via a console app.
/// </summary>
public static T RunInAppDomain<T>(Func<T> func)
{
AppDomain domain = AppDomain.CreateDomain("Delegate Executor " + func.GetHashCode(), null,
new AppDomainSetup { ApplicationBase = Environment.CurrentDirectory });
try
{
domain.SetData("toInvoke", func);
domain.DoCallBack(() =>
{
var f = AppDomain.CurrentDomain.GetData("toInvoke") as Func<T>;
AppDomain.CurrentDomain.SetData("result", f());
});
return (T)domain.GetData("result");
}
finally
{
AppDomain.Unload(domain);
}
}
[Serializable]
private class ActionDelegateWrapper
{
public Action Func;
public int Invoke()
{
Func();
return 0;
}
}
public static void RunInAppDomain(Action func)
{
RunInAppDomain<int>( new ActionDelegateWrapper { Func = func }.Invoke );
}
If you're still with me, I appreciate :-)
Now, after spending so much time on fixing that mechanism, I am going to tell you that is was purposeless anyway.
The thing is, AppDomains won't help you for your purposes. They only take care of managed objects, while unmanaged code can leak and crash all it wants. Unmanaged code doesn't even know there are such things as appdomains. It only knows about processes.
So, in the end, your best option remains your current solution: just spawn another process and be happy about it. And, I would agree with the previous answers, you don't have to write another console app for each case. Just pass a fully qualified name of a static method, and have the console app load your assembly, load your type, and invoke the method. You can actually package it pretty neatly in a very much the same way as you tried with AppDomains. You can create a method called something like "RunInAnotherProcess", which will examine the argument, get the full type name and method name out of it (while making sure the method is static) and spawn the console app, which will do the rest.
You don't have to create many console applications, you can create a single application that will receive as parameter the full qualified type name. The application will load that type and execute it.
Separating everything into tiny processes is the best method to really dispose all the resources. An application domain cannot do full resources disposing, but a process can.
Have you considered opening a pipe between the main application and the sub applications? This way you could pass more structured information between the two applications without parsing standard output.