We Keep Coding

Is there a way to use System.Diagnostics.Process in an IAsyncEnumerator?

First off, I am targeting .Net Core 3.1 and C#8.
I want something like this.
public static async Task<MyDataObj> GetData()
{
var dataObj = new MyDataObj();
var args = ArgsHelperFunction(...);
await foreach(string result in RunProcessAsync(args))
{
// Process result and store it in dataObj
}
return dataObj;
}
private static async IAsyncEnumerable<string> RunProcessAsync(string args)
{
await using (var myProcess = new Process())
{
myProcess.StartInfo.FileName = #"path\to\file.exe"
myProcess.StartInfo.Arguments = args;
myProcess.StartInfo.UseShellExecute = false;
myProcess.StartInfo.RedirectStandardError = true;
myProcess.StartInfo.CreateNoWindow = true;
myProcess.ErrorDataReceived += (s, e) =>
{
yield return e.Data;
}
myProcess.Start();
myProcess.BeginErrorReadLine();
process.WaitforExit();
}
}
When I try this setup I get an errors from await foreach(string result in RunProcessAsync(args))
CS8417 'Process': type used in an asynchronous using statement must be implicitly convertible to 'System.IAsyncDisposable' or implement a suitable 'DisposeAsync' method.
and this error from yield return e.Data;
CS1621 The yield statement cannot be used inside an anonymous method or lambda expression
The goal is this. I have an exe that does some stuff and writes information to the error output stream (not sure if that's its real name). I want to take in those writes as they are made, parse them for the information I want and store them in an object for later use.
I am a pretty novice coder and very new to asynchronous coding. I tested the functionality of RunProcessAsync but in a synchronous manner; where it was called and just wrote all the raw data to the output window without returning any of it to the calling method. That worked just fine. Also, I have gotten a test asyc stream working with IAsyncEnumerable, but it just used Task.Delay and returned some integers. Now I am trying to combine the to things and my lack of experience is getting in my way.
Thank you for any help you all might give and for helping to increase skill and knowledge of C#.

Without a minimal, reproducible example, it will be impossible to address your concern completely. But we can deal with the two specific issues you've raised.
First, if your object (such as Process) doesn't support IAsyncDisposable, then just don't use that. Use the synchronous using statement instead.
As far as the yield return in the method goes, if you take a moment you'll probably see that what you tried to write doesn't make any sense. How would the event handler, which is a completely different method, be able to cause the current method to yield a new value? You need the event handler to signal to the current method when the event occurs. You can do this in a variety of ways, but SemaphoreSlim is one of the more straightforward ways.
Putting those together, you might get something like this:
private static async IAsyncEnumerable<string> RunProcessAsync(string args)
{
using (var myProcess = new Process())
{
myProcess.StartInfo.FileName = #"path\to\file.exe";
myProcess.StartInfo.Arguments = args;
myProcess.StartInfo.UseShellExecute = false;
myProcess.StartInfo.RedirectStandardError = true;
myProcess.StartInfo.CreateNoWindow = true;
ConcurrentQueue<string> dataQueue = new ConcurrentQueue<string>();
SemaphoreSlim dataSemaphore = new SemaphoreSlim(0);
myProcess.ErrorDataReceived += (s, e) =>
{
dataQueue.Enqueue(e.Data);
dataSemaphore.Release();
}
myProcess.Start();
myProcess.BeginErrorReadLine();
while (true)
{
await dataSemaphore.WaitAsync();
// Only one consumer, so this will always succeed
dataQueue.TryDequeue(out string data);
if (data == null) break;
yield return data;
}
}
}
Since you didn't provide an actual MCVE, it's not feasible for me to try to reconstruct your scenario from scratch. So the above isn't compiled, never mind tested. But it should show the gist.
Which is, you need to keep your iterator method asynchronous (which means you can't block on a call to WaitForExit()), and you need to somehow move the data received by the ErrorDataReceived event handler back to the iterator method. In the above, I use a thread-safe queue object in conjunction with a semaphore.
The semaphore count gets increased (via Release()) in the event handler each time a line of data is received, which is then consumed by the iterator method by decreasing the semaphore count (via WaitAsync()) and returning the line received.
There are lots of other mechanisms one could use for the producer/consumer aspect here. There's a well-received Q&A here that discusses async-compatible mechanisms, including a custom version of BlockingCollection<T> that supports async operations, and a mention of the BufferBlock<T> class from TPL Dataflow.
Here is an example that uses BufferBlock<T> (which has semantics very similar to BlockingCollection<T> but includes async handling of the consuming code):
static async IAsyncEnumerable<string> RunProcessAsync(string args)
{
using (var process = new Process())
{
myProcess.StartInfo.FileName = #"path\to\file.exe";
process.StartInfo.Arguments = args;
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardError = true;
process.StartInfo.CreateNoWindow = true;
BufferBlock<string> dataBuffer = new BufferBlock<string>();
process.ErrorDataReceived += (s, e) =>
{
if (e.Data != null)
{
dataBuffer.Post(e.Data);
}
else
{
dataBuffer.Complete();
}
};
process.Start();
process.BeginErrorReadLine();
while (await dataBuffer.OutputAvailableAsync())
{
yield return dataBuffer.Receive();
}
}
}

Something like this?
using CliWrap;
using CliWrap.EventStream;
var cmd = Cli.Wrap("foo").WithArguments("bar");
await foreach (var cmdEvent in cmd.ListenAsync())
{
switch (cmdEvent)
{
case StartedCommandEvent started:
_output.WriteLine($"Process started; ID: {started.ProcessId}");
break;
case StandardOutputCommandEvent stdOut:
_output.WriteLine($"Out> {stdOut.Text}");
break;
case StandardErrorCommandEvent stdErr:
_output.WriteLine($"Err> {stdErr.Text}");
break;
case ExitedCommandEvent exited:
_output.WriteLine($"Process exited; Code: {exited.ExitCode}");
break;
}
}
This is available in CliWrap.

Call a process asynchronously via WebAPI

I am in a scenario wherein I need to call an exe process via an API endpoint (fire and forget). However, the exe that I will call is a long running one but I need to wait for it to finish to execute further codes. This will happen in the background and the client shouldn't really wait for it. When I'm trying the code below, the call is waiting for the exe process to finish. I've also tried making the method "async" but still same result.
[HttpPost]
public async Task<bool> ToggleCarWeaverService(string command)
{
try
{
var fileName = #"SomeExe.exe";
await _service.RunProcessAsync(command, fileName);
return true;
}
catch (Exception ex)
{
throw new HttpResponseException(HttpStatusCode.InternalServerError);
}
}
public Task<int> RunProcessAsync(string command, string fileName)
{
// Use ProcessStartInfo class
ProcessStartInfo startInfo = new ProcessStartInfo
{
CreateNoWindow = false,
UseShellExecute = true,
FileName = fileName,
WindowStyle = ProcessWindowStyle.Normal,
Arguments = command
};
try
{
var tcs = new TaskCompletionSource<int>();
var process = new Process
{
StartInfo = startInfo,
EnableRaisingEvents = true
};
process.Exited += (sender, args) =>
{
tcs.SetResult(process.ExitCode);
//Do more code
process.Dispose();
};
process.Start();
return tcs.Task;
}
catch (Exception ex)
{
// Log error.
throw ex;
}
}

Do not await the async task.
Use Task.Run to wrap and continue
[HttpPost]
public IHttpActionResult ToggleCarWeaverService(string command) {
Task.Run(async () => {
var fileName = #"SomeExe.exe";
await _service.RunProcessAsync(command, fileName);
//...other code after process finish
});
return Ok();
}

The answers given here will, correctly, answer your question - that is, by not awaiting the result, your call will return immediately. However, there is no guarantee that the process will continue to run. Once you've left the function you, effectively, have a rouge process that may or may not complete.
Most Cloud providers have a solution to this (e.g. Azure WebJobs), or you could roll your own using something like HangFire.

Then you shouldn't await the call further at all and thus change the line where you await the call RunProcessAsync to
_service.RunProcessAsync(command, fileName);

How to run a process(Task) remotely and wait for it to finish?

I have created a simple "job scheduler" specific to my needs. It looks for job specs written to a file, and then starts multiple tasks to run jobs in parallel.
It waits for each task to finish before checking if there are more jobs to be completed.
This works very well for me, but I would now like to use a second server to run some of the jobs. Is there a way to start a job on a remote machine AND somehow receive notification of when it has finished? I know WMI is an option, but wondered if WMI (or something else) would fit easily in to the structure below.
Here is how I do it (very simplified) on a single machine:
List<Taskdata> TaskList = new List<Taskdata>();
string[] JobsToAdd = GetListFromSomewhere();
foreach (var jobString in JobsToAdd)
{
var LastTask = new Task(RunProc, jobString);
LastTask.Start();
TaskList.Add(new Taskdata(LastTask, DateTime.UtcNow));
}
if (TaskList.Any())
{
Task.WaitAny(TaskList.Select(x => x.Task).ToArray());
}
public void RunProc(object state)
{
var process = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = Path.Combine(#"H:\Projects\Simulator\Simulator\bin\Debug\", "Simulator001.exe"),
Arguments = (string)state,
WindowStyle = ProcessWindowStyle.Normal,
CreateNoWindow = false
}
};
process.Start();
process.WaitForExit();
}

How can I stop async Process by CancellationToken?

I found beneath code for execute some process without freezing UI. This code is executed when 'Start Work' button is pressed. And I think users would stop this work by 'Stop' button. So I found this article at MSDN.. https://msdn.microsoft.com/en-us/library/jj155759.aspx . But, It was hard that applying this CancellationToken at this code.. Anyone can help this problem?
I use public static async Task<int> RunProcessAsync(string fileName, string args) method only.
Code (From https://stackoverflow.com/a/31492250):
public static async Task<int> RunProcessAsync(string fileName, string args)
{
using (var process = new Process
{
StartInfo =
{
FileName = fileName, Arguments = args,
UseShellExecute = false, CreateNoWindow = true,
RedirectStandardOutput = true, RedirectStandardError = true
},
EnableRaisingEvents = true
})
{
return await RunProcessAsync(process).ConfigureAwait(false);
}
}
// This method is used only for internal function call.
private static Task<int> RunProcessAsync(Process process)
{
var tcs = new TaskCompletionSource<int>();
process.Exited += (s, ea) => tcs.SetResult(process.ExitCode);
process.OutputDataReceived += (s, ea) => Console.WriteLine(ea.Data);
process.ErrorDataReceived += (s, ea) => Console.WriteLine("ERR: " + ea.Data);
bool started = process.Start();
if (!started)
{
//you may allow for the process to be re-used (started = false)
//but I'm not sure about the guarantees of the Exited event in such a case
throw new InvalidOperationException("Could not start process: " + process);
}
process.BeginOutputReadLine();
process.BeginErrorReadLine();
return tcs.Task;
}
Usage :
var cancelToken = new CancellationTokenSource();
int returnCode = async RunProcessAsync("python.exe", "foo.py", cancelToken.Token);
if (cancelToken.IsCancellationRequested) { /* something */ }
When the start button clicked, it starts some python script.
When script is running and user wants to stop it, user presses stop button.
Then program executes below code.
cancelToken.Cancel();
Thank you very much for reading this question.

The simple answer is that you can just call process.Kill() when the token is canceled:
cancellationToken.Register(() => process.Kill());
But there are two problems with this:
If you attempt to kill a process that doesn't exist yet or that has already terminated, you get an InvalidOperationException.
If you don't Dispose() the CancellationTokenRegistration returned from Register(), and the CancellationTokenSource is long-lived, you have a memory leak, since the registrations will stay in memory as long as the CancellationTokenSource.
Depending on your requirements, and your desire for clean code (even at the cost of complexity) it may be okay to ignore problem #2 and work around problem #1 by swallowing the exception in a catch.

It's quite simple now:
process.WaitForExitAsync(token);

c# ProcessStartInfo.Start - reading output but with a timeout

If you want to start another process and wait (with time out) to finish you can use the following (from MSDN).
//Set a time-out value.
int timeOut=5000;
//Get path to system folder.
string sysFolder=
Environment.GetFolderPath(Environment.SpecialFolder.System);
//Create a new process info structure.
ProcessStartInfo pInfo = new ProcessStartInfo();
//Set file name to open.
pInfo.FileName = sysFolder + #"\eula.txt";
//Start the process.
Process p = Process.Start(pInfo);
//Wait for window to finish loading.
p.WaitForInputIdle();
//Wait for the process to exit or time out.
p.WaitForExit(timeOut);
//Check to see if the process is still running.
if (p.HasExited == false)
//Process is still running.
//Test to see if the process is hung up.
if (p.Responding)
//Process was responding; close the main window.
p.CloseMainWindow();
else
//Process was not responding; force the process to close.
p.Kill();
MessageBox.Show("Code continuing...");
If you want to start another process and read its output then you can use the following pattern (from SO)
// Start the child process.
Process p = new Process();
// Redirect the output stream of the child process.
p.StartInfo.UseShellExecute = false;
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.FileName = "Write500Lines.exe";
p.Start();
// Do not wait for the child process to exit before
// reading to the end of its redirected stream.
// p.WaitForExit();
// Read the output stream first and then wait.
string output = p.StandardOutput.ReadToEnd();
p.WaitForExit();
How can you combine the two to read all input, not get stuck in deadlock and have a timeout if the running process goes awry?

This technique will hang if the output buffer is filled with more that 4KB of data. A more foolproof method is to register delegates to be notified when something is written to the output stream. I've already suggested this method before in another post:
ProcessStartInfo processInfo = new ProcessStartInfo("Write500Lines.exe");
processInfo.ErrorDialog = false;
processInfo.UseShellExecute = false;
processInfo.RedirectStandardOutput = true;
processInfo.RedirectStandardError = true;
Process proc = Process.Start(processInfo);
// You can pass any delegate that matches the appropriate
// signature to ErrorDataReceived and OutputDataReceived
proc.ErrorDataReceived += (sender, errorLine) => { if (errorLine.Data != null) Trace.WriteLine(errorLine.Data); };
proc.OutputDataReceived += (sender, outputLine) => { if (outputLine.Data != null) Trace.WriteLine(outputLine.Data); };
proc.BeginErrorReadLine();
proc.BeginOutputReadLine();
proc.WaitForExit();

You don't have to combine the two - the Process class has an event that fires when output is sent to the StandardOutput - OutputDataReceived.
If you subscribe to the event, you will be able to read output as it arrives and in your main program loop you can still timeout.

you can try modifying the first method to something like this
Process p = Process.Start(pInfo);
string output = string.Empty;
Thread t = new Thread(() => output = p.StandardOutput.ReadToEnd() );
t.Start();
//Wait for window to finish loading.
p.WaitForInputIdle();
//Wait for the process to exit or time out.
p.WaitForExit(timeOut);

void OpenWithStartInfo()
{
ProcessStartInfo startInfo = new ProcessStartInfo("IExplore.exe", "Default2.aspx");
startInfo.WindowStyle = ProcessWindowStyle.Minimized;
Process p = Process.Start(startInfo);
p.WaitForInputIdle();
//p.WaitForExit(2);
p.Kill();
}

You could also use the APM, like this:
Define a delegate for the ReadToEnd call:
private delegate string ReadToEndDelegate();
Then use the delegate to call the method like this:
ReadToEndDelegate asyncCall = reader.ReadToEnd;
IAsyncResult asyncResult = asyncCall.BeginInvoke(null, null);
asyncResult.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(10));
asyncCall.EndInvoke(asyncResult);
EDIT: Error handling removed for clarity.

Just add everything from the first example below the WaitForExit() call to the second example.

None of the above answers work for me when dealing with interactive promts. (My command sometimes promts a question to the user and that should also be covered by timeout).
This is my solution.
A disadvantage is that i don't get any output if we run in a timeout.
ReadToEnd() blocks the execution so we have to run it in another thread and kill this thread if the process runs into the specified timeout.
public static Tuple<string, string> ExecuteCommand(string command)
{
// prepare start info
var procStartInfo = new ProcessStartInfo("cmd", "/c " + command)
{
ErrorDialog = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
WorkingDirectory = #"C:\",
CreateNoWindow = true
};
// start process
var proc = new Process {StartInfo = procStartInfo};
proc.Start();
var error = "";
var output = "";
// read stdout and stderr in new thread because it is blocking
Thread readerThread = new(() =>
{
try
{
error = proc.StandardError.ReadToEnd().Trim();
output = proc.StandardOutput.ReadToEnd().Trim();
}
catch (ThreadInterruptedException e)
{
Debug.WriteLine("Interrupted!!");
}
});
readerThread.Start();
// wait for max 6 seconds
if (proc.WaitForExit(6_000))
{
// if command runs to an enc => wait for readerThread to collect error/output stream
readerThread.Join();
}
else
{
// if process takes longer than 6 seconds => kill reader thread and set error to timeout
readerThread.Interrupt();
error = "Timeout!";
}
// return output and error
return new Tuple<string, string>(output, error);
}