My testing seems to suggest that Process.Start() at its bare bones costs ~15-27ms (depending on if you use CreateNoWindow, which ADDS ~10ms). In this question, I'm not trying to say the code I'm running is slow, but the actual act of starting a process, even if it does nothing.
I got these numbers by using a stopwatch on a console app that ran another console app literally just returning in its main method.
namespace RunNothing
{
class Program
{
static void Main(string[] args)
{
var startInfo = new ProcessStartInfo(#"C:\Users\Noggog\Documents\visual studio 2017\Projects\DoNothing\DoNothing\bin\Release\DoNothing.exe")
{
CreateNoWindow = true,
UseShellExecute = false,
};
var sw = new Stopwatch();
var proc = new Process()
{
StartInfo = startInfo,
EnableRaisingEvents = true
};
proc.Exited += (sender, a) =>
{
sw.Stop();
System.Console.WriteLine(sw.ElapsedMilliseconds);
};
sw.Start();
proc.Start();
System.Console.ReadLine();
}
}
}
So my question is whether anyone knows a good way to improve that startup time per Process.Start() call?
For background, I have an app that will be starting various .exes, most of which should do a quick bool check and short circuit out, asap. Every so often one of the calls will do a longer operation, but usually it will do nothing. Right now a ~15-27ms call per go is getting a bit heavy for my use case.
Edit:
Some people were asking for more details of the end usage. This is the project that drove the question. The end usage is an extension of git hooks to provide more hookable commands, and provide convenience features such as calling an exe in response to hooks being fired. One of the modes is that a single exe can handle multiple hooks, and decide which ones it would respond to. In that scenario, every git command would "check in" with the exe to see if it wanted to do any logic. This is where the Process.Start() time adds up. If you have 6 repos and your git client is initializing things by running several commands a pre and post hook combo can be 27ms (proc start time) * 2(pre/post) * X(commands) * 6(repos) = ~2+ seconds. Potentially none of these commands are ones needing response, but it's already added several seconds of sluggishness to the system.
Related
For reasons of memory isolation and stability I need to execute some methods/classes as sub-processes, not threads.
I found the following library that allows me to do just that: https://github.com/tmds/Tmds.ExecFunction.
However, I can't seem to find a way to get the process ID of the child process.
ExecFunction.Run(() => new NewSubProcessClass());
The code above will generate a new process and will even generate console output in a Linux Docker container. The problem is, that I have no process ID.
public static Process Start(Action action, Action<ExecFunctionOptions> configure = null);
The method I quoted above looked like it should do the trick, however, when I replace
ExecFunction.Run(() => new NewSubProcessClass());
with
Process process = ExecFunction.Start(() => new NewSubProcessClass());
I no longer get any console output.
UPDATE 0:
The suggestion from #bazzilic
process.WaitForExit();
is the solution to the problem.
If you examine the source code for Tmds.ExecFunction, both methods .Run(...) (here) and .Start(...) (here) under the hood call the same private method .Start(...) (source here). The two differences are:
the public .Start(...) method returns the process object from the tuple returned by the private .Start(...) method whereas .Run(...) does not;
the .Run(...) method also supplies waitForExit: true in the parameters.
In the code of the private .Start(...) method, there is this portion:
if (waitForExit)
{
process.WaitForExit();
}
Other than that, the two public methods are identical. The difference in your code behavior is most likely due to not waiting for the child process to finish in your parent process. Try this:
Process process = ExecFunction.Start(() => new NewSubProcessClass());
process.WaitForExit();
PS: Keep in mind that Process is IDisposable, so might be better to do
using ( Process process = ExecFunction.Start(() => new NewSubProcessClass()) ) {
process.WaitForExit();
}
I am writing an application to manage processes and handle failovers. This program is written in C# for .NET Core and will run on Ubuntu Server 16.04 x64.
I have this code to create processes and track them, with exit events and such
ProcessStartInfo psi = new ProcessStartInfo
{
WorkingDirectory = "/home/xyzserver/someprocess",
FileName = "mono",
Arguments = "someprocess.exe",
RedirectStandardOutput = true
};
_proc = Process.Start(psi);
_proc.EnableRaisingEvents = true;
_proc.Exited += ProcOnExited;
I understand from the docs here that calls to Console.WriteLine will block if the _proc.StandardOutput stream is full. I want to prevent this behavior and dispose all the output from the managed application, since it will also write to a physical log on its own.
In addition, I would like to avoid storing any of the output in any unused stream buffers since they will never be used. A preferred solution will not UseShellExecute.
I have considered adding these 2 lines in the hope that any received data will be disposed, but am unsure about correctness.
_proc.OutputDataReceived += (sender, eventArgs) => {};
_proc.BeginOutputReadLine();
Is there a better way to accomplish this? Thoughts or comments are appreciated.
I manually ran the test on .NET Core using 3 programs:
An HTTP server to track the TextOutputter program.
A TextOutputter program that prints 1000 characters and makes an HTTP request every second.
a ProgramRunner that runs one instance of TextOutputter.
Without the 2 lines, the buffer fills upto 64k and stalls. With the 2 lines, there is no stalling.
Im trying to implement a performance monitoring tool, I want to monitor basic things such as Memory and CPU.
I am attempting to do so by using Performance Counters as I believe this is what Task Manager is using behind the scenes too. I have no idea how Task Manager is able to do this however as to me it seems to take a VERY long time to retrieve process data using this method:
class Program
{
static void Main(string[] args)
{
while (true)
{
var pcs = Process.GetProcesses()
.Select(p => new PerformanceCounter("Process", "Working Set - Private", p.ProcessName));
var sw = Stopwatch.StartNew();
foreach (var pc in pcs)
pc.NextValue();
Console.WriteLine($"Time taken to read {pcs.Count()} performance counters: {sw.ElapsedMilliseconds}ms");
Thread.Sleep(1000);
}
}
}
Has anyone got any suggestions on how to do this or how even Task Manager or Process Explorer is able to do this?
How does Task Manager do it?
he used calls to ZwQuerySystemInformation, ZwQueryInformationProcess, ZwQueryInformationThread ..
Task Manager maintain database of active processes and periodically update this info by calling ZwQuerySystemInformation(SystemProcessInformation,) - so got array of SYSTEM_PROCESS_INFORMATION on exit.
add new entries if found new process, yet not in DB, remove entries for died processes, update info for lived
SYSTEM_PROCESS_INFORMATION already containing a lot information of process. additional information can be get by open process and call ZwQueryInformationProcess with appropriate info class
if you want implement a performance monitoring tool, without "quantum effect" (when the measurement affects the state itself) you need use this ntdll api. for definitions look at http://processhacker.sourceforge.net/doc/ntexapi_8h_source.html
despite this is undocumented, existing functions and structures not changed how minimum from win2000 (so ~17 years) - new version of windows add a lot new info classes, some fields which was spare/unused in old version - can become used, but old(legacy) not changed
I have the following code to launch and monitor a process:
Process process = new Process();
process.StartInfo.FileName = "foo.exe";
long maxMemoryUsage = 0;
process.Start()
while(!process.HasExited)
{
maxMemoryUsage = Math.Max(maxMemoryUsage, process.PrivateMemorySize64);
}
After using this code to run a large application that, according to the task manager used 328 MB at its peak (Memory "Private Working Set"). The value of maxMemoryUsage, and the value of process.PeakPagedMemorySize64 is 364544. According to MSDN this value should be interpreted as bytes, meaning it is a little over 300KB, a factor thousand away from the expected value. The other process.Peak...Memory properties also report extremely low values (all under a megabyte, except for PeakVirtualMemorySize64 which is 4MB which I think is the minimum value for this field).
I've tried launching different applications (C# and C++ based of which I have the source code) which I know to use very little or a lot of memory and the memory values where always very close to the values seen with this process. Apparently I'm doing something completely wrong.
So my question is: How can I measure the maximum memory usage of a process which I spawned from my C# application. (Note that I don't need to have the value ealtime as long as I know its value after the program exited, I also don't need it super precise as I don't care if it was 27.04MB or 30MB, but I do care if it was 30MB or 100MB).
Edit: here is a full reproducable test case
class Program
{
static void Main(string[] args)
{
Process process = new Process();
process.StartInfo.FileName = #"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv.exe";
long maxMemoryUsage = 0;
process.Start();
while(!process.HasExited)
{
maxMemoryUsage = Math.Max(maxMemoryUsage, process.PagedMemorySize64);
}
Console.Out.WriteLine("Memory used: " + (maxMemoryUsage / 1024.0) / 1024.0 + "MB");
Console.ReadLine();
}
}
According to the task manager Visual Studio uses 103MB. After closing Visual Studio the program reports 0.3984375MB.
Process class is heavily cached. You'll get only cached result, no matter how many times you read some property unless you throw a call to Refresh method. You need to call Process.Refresh to get the non cached result.
To quote from msdn
When a Process component is associated with a process resource, the
property values of the Process are immediately populated according to
the status of the associated process. If the information about the
associated process subsequently changes, those changes are not
reflected in the Process component's cached values. The Process
component is a snapshot of the process resource at the time they are
associated. To view the current values for the associated process,
call the Refresh method.
So, your code will become:
while(!process.HasExited)
{
process.Refresh();
maxMemoryUsage = Math.Max(maxMemoryUsage, process.PrivateMemorySize64);
}
Also you may consider looking at process.PeakXXX properties, that will help you I suppose.
I have a simple console app that writes out status of various resources in a nicely formatted way.
This app is using some 3rd party components to connect to places.
Those components unfortunately do a lot Console.Writes (or some sort of logging) which all ends up intertwined with my messages.
I tried redirecting Console output (in hopes that i could filter non-mine messages), but that seems to work ONLY on my messages.
var sb = new StringBuilder();
TextWriter tw = new StringWriter(sb);
Console.SetOut(tw);
So this works in redirecting console writes, but only on the ones i did.. output from 3rd party components is still streaming to my screen.. any other ways to supress that?
The reason you're not able to redirect the output of the third party components is because you're calling redirect on your process, not theirs. To do this, loop over the processes, find the ones that are writing to the console and redirect their output.
using System.Diagnostics;
Process[] processlist = Process.GetProcesses();
foreach(Process theprocess in processlist){
// you'll actually need to use something like the process name or id here.
if (theprocess == MyThirdPartyComponentsProcess)
{
theprocess.StartInfo.UseShellExecute = false;
theprocess.StartInfo.RedirectStandardOutput = true;
}
}
You'll have to be running as administrator for this work.
I've thought of one fairly hacky way:
Capture the existing Console.Out so you can reuse it in your "real" writer
Create your own TextWriter implementation, which...
... when it's asked to write, checks whether the calling code is in your assembly. See Assembly.GetCallingAssembly
So long as you don't have inlining problems, you should be able to check the context, and drop any output you don't want. You'd probably have to call Console.Out.WriteLine directly instead of Console.WriteLine, mind you - otherwise the caller will be the Console class.
A better solution would be to change your code to use log4net or something similar, so you could get your output in a more configurable way, not on the console - assuming your own use is also logging. (If it's for interactive output, something like the above may be the only way...)
Since, there doesn't seem to be a reasonable quick way to suppress these, I went with plan B, that is deferring my messages to the end. This way all the logging gets through, and than i write out all the information i am interested in.
I broke it down into Success/Warn/Error methods
public static void WriteError(string message, params string[] args) {
writerTasks.Add(() => {
var c = Console.ForegroundColor;
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine(message, args);
Console.ForegroundColor = c;
});
}
and at the end of the execution, i call this method to flush it out.
private static void WriteResults() {
foreach (var t in writerTasks)
t();
}
i do it this way, because i am changing text color depending on message type, which is contained within each action.
in my case this works great, since the information scrolls up hiding all the logging.. which in fact is also beneficial.. just didn't want it dispersed between the main output