We Keep Coding

Missing Output when Redirecting Standard Error in C# / .NET Process

I'm working with a 3rd party, command-line tool called "sam-ba" v3.5 (available here for free). It's a C++ / QML command line tool that interfaces with a hardware module to read/write data. Output from commands, in most cases, is sent to Standard Error.
I have a C# / .NET application that creates a Process object to execute the sam-ba tool and run commands. Executing the commands works as expected. What doesn't always work is the redirect of the Standard Error output. In some commands, part or all of the output is not received by the C# application. For example, here is the execution of a command using the sam-ba tool directly at the Windows 10 command line:
C:\Temp\Stuff\sam-ba_3.5>sam-ba -p serial:COM5 -d sama5d3 -m version
Error: Cannot open invalid port 'COM5'
Cannot open invalid port 'COM5'
Here is some simple code from a C# application to create a Process object to execute the sam-ba tool with the same command:
Process p = new Process
{
StartInfo = new ProcessStartInfo("sam-ba.exe", "-p serial:COM5 -d sama5d3 -m version")
{
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
CreateNoWindow = true
}
};
p.Start();
string output = p.StandardOutput.ReadToEnd();
string error = p.StandardError.ReadToEnd();
p.WaitForExit();
Console.WriteLine("Standard Out: " + output);
Console.WriteLine("Standard Error: " + error);
The Output of the C# application:
Standard Out:
Standard Error: Cannot open invalid port 'COM5'
In this simple example, only 1 of the output lines is redirected to Standard Error while the other is not. I've tried many different commands and results are mixed. Sometimes I get everything, sometimes partial output, sometimes no output.
Now ... here's the real issue. The following is a python script (v3.8) that does exactly what the C# application is doing:
import subprocess
import sys
result = subprocess.run("sam-ba.exe -p serial:COM5 -d sama5d3 -m version", capture_output=True, text=True)
print("stdout:", result.stdout)
print("stderr:", result.stderr)
This script always returns the correct output to standard error. BUT ... when I run this script from the C# app to create a chain of C# -> python -> sam-ba, I get the same issue of output missing from the stream.
This has led me to 2 conclusions:
Something in that sam-ba tool is different about the way it is outputting its text. Format, content, ... something. There's an inconsistency going on somewhere in that code
Something is different about the environment created by the C# Process object when executing external applications that doesn't happen when the external application is run directly. Otherwise, why would the python script get all the output when run directly, but not when run through the C# Process object?
It is #2 that has brought me here. I'm looking for any insight on how to diagnose this. Something I'm doing wrong, settings I can try within the Process object, thoughts on how data can go into a stream and not come out on redirect, or if anyone has ever seen something like this before and how they resolved it.
UPDATE
Got a hold of the sam-ba tool's source code. The output the C# app is not capturing is coming from the QML files. They are using this 'print()' method that I can't really find any details on. The output the C# app can capture is being delivered back to the C++ side via signals and then sent to standard error. This feeds back into my conclusion #1 where they have inconsistencies in their code.
Still, this potentially means there is a conflict between C# and QT/QML, which would explain why the Python script gets the QML output, but the C# app does not.

The following uses ShellExecute instead of CreateProcess when running
process. When using ShellExecute one can't re-direct StandardOutput and/or StandardError for Process. To work around this, both StandardOutput and StandardError are re-directed to a temp file, and then the data is read from the temp file--which seems to result in the same output that one sees when running from a cmd window.
Note: In the following code it's necessary to use %windir%\system32\cmd.exe (ex: C:\Windows\system32\cmd.exe) with the /c option. See the usage section below.
Add using statement: using System.Diagnostics;
Then try the following:
public string RunProcess(string fqExePath, string arguments, bool runAsAdministrator = false)
{
string result = string.Empty;
string tempFilename = System.IO.Path.Combine(System.IO.Path.GetTempPath(), "tempSam-ba.txt");
string tempArguments = arguments;
if (String.IsNullOrEmpty(fqExePath))
{
Debug.WriteLine("fqExePath not specified");
return "Error: fqExePath not specified";
}
//redirect both StandardOutput and StandardError to a temp file
if (!arguments.Contains("2>&1"))
{
tempArguments += String.Format(" {0} {1} {2}", #"1>", tempFilename, #"2>&1");
}
//create new instance
ProcessStartInfo startInfo = new ProcessStartInfo(fqExePath, tempArguments);
if (runAsAdministrator)
{
startInfo.Verb = "runas"; //elevates permissions
}//if
//set environment variables
//pStartInfo.EnvironmentVariables["SomeVar"] = "someValue";
startInfo.RedirectStandardError = false;
startInfo.RedirectStandardOutput = false;
startInfo.RedirectStandardInput = false;
startInfo.UseShellExecute = true; //use ShellExecute instead of CreateProcess
startInfo.CreateNoWindow = false;
startInfo.WindowStyle = ProcessWindowStyle.Hidden;
startInfo.ErrorDialog = false;
startInfo.WorkingDirectory = System.IO.Path.GetDirectoryName(fqExePath);
using (Process p = Process.Start(startInfo))
{
//start
p.Start();
//waits until the process is finished before continuing
p.WaitForExit();
}
//read output from temp file
//file may still be in use, so try to read it.
//if it is still in use, sleep and try again
if (System.IO.File.Exists(tempFilename))
{
string errMsg = string.Empty;
int count = 0;
do
{
//re-initialize
errMsg = string.Empty;
try
{
result = System.IO.File.ReadAllText(tempFilename);
Debug.WriteLine(result);
}
catch(System.IO.IOException ex)
{
errMsg = ex.Message;
}
catch (Exception ex)
{
errMsg = ex.Message;
}
System.Threading.Thread.Sleep(125);
count += 1; //increment
} while (!String.IsNullOrEmpty(errMsg) && count < 10);
//delete temp file
System.IO.File.Delete(tempFilename);
}
return result;
}
Usage:
RunProcess(#"C:\Windows\system32\cmd.exe", #"/c C:\Temp\sam-ba_3.5\sam-ba.exe -p serial:COM5 -d sama5d3 -m version");
Note: /c C:\Temp\sam-ba_3.5\sam-ba.exe -p serial:COM5 -d sama5d3 -m version is the value of the process "Argument" property.
Update:
Option 2:
Here's a solution that uses named pipes. Process is used to redirect the output to a named pipe instead of a file. One creates a named pipe "server" which listens for a connection from a client.Then System.Diagnostics.Process is used to run the desired command and redirect the output to the named pipe server. The "server" reads the output, and then raises event "DataReceived" which will return the data to any subscribers.
The named pipe server code is from here, however I've modified it. I've added numerous events--which can be subscribed to. I've also added the ability for the server to shut itself down after it's finished reading the data by setting "ShutdownWhenOperationComplete" to "true".
Create a class named: HelperNamedPipeServer.cs
HelperNamedPipeServer.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO.Pipes;
using System.IO;
using System.Diagnostics;
using System.Threading;
using System.Security.Principal;
namespace ProcessTest
{
public class HelperNamedPipeServer : IDisposable
{
//delegates
public delegate void EventHandlerClientConnected(object sender, bool e);
public delegate void EventHandlerDataReceived(object sender, string data);
public delegate void EventHandlerOperationCompleted(object sender, bool e);
public delegate void EventHandlerMessageComplete(object sender, bool e);
public delegate void EventHandlerReadComplete(object sender, bool e);
public delegate void EventHandlerServerShutdown(object sender, bool e);
public delegate void EventHandlerServerStarted(object sender, bool e);
//event that subscribers can subscribe to
public event EventHandlerClientConnected ClientConnected;
public event EventHandlerDataReceived DataReceived;
public event EventHandlerMessageComplete MessageReadComplete;
public event EventHandlerOperationCompleted OperationCompleted;
public event EventHandlerReadComplete ReadComplete;
public event EventHandlerServerShutdown ServerShutdown;
public event EventHandlerServerStarted ServerStarted;
public bool IsClientConnected
{
get
{
if (_pipeServer == null)
{
return false;
}
else
{
return _pipeServer.IsConnected;
}
}
}
public string PipeName { get; set; } = string.Empty;
public bool ShutdownWhenOperationComplete { get; set; } = false;
//private int _bufferSize = 4096;
private int _bufferSize = 65535;
//private volatile NamedPipeServerStream _pipeServer = null;
private NamedPipeServerStream _pipeServer = null;
public HelperNamedPipeServer()
{
PipeName = "sam-ba-pipe";
}
public HelperNamedPipeServer(string pipeName)
{
PipeName = pipeName;
}
private NamedPipeServerStream CreateNamedPipeServerStream(string pipeName)
{
//named pipe with security
//SecurityIdentifier sid = new SecurityIdentifier(WellKnownSidType.BuiltinAdministratorsSid, null); //member of Administrators group
//SecurityIdentifier sid = new SecurityIdentifier(WellKnownSidType.WorldSid, null); //everyone
//SecurityIdentifier sid = new SecurityIdentifier(WellKnownSidType.BuiltinUsersSid, null); //member of Users group
//PipeAccessRule rule = new PipeAccessRule(sid, PipeAccessRights.ReadWrite, System.Security.AccessControl.AccessControlType.Allow);
//PipeSecurity pSec = new PipeSecurity();
//pSec.AddAccessRule(rule);
//named pipe - with specified security
//return new NamedPipeServerStream(PipeName, PipeDirection.InOut, NamedPipeServerStream.MaxAllowedServerInstances, PipeTransmissionMode.Byte, PipeOptions.Asynchronous, _bufferSize, _bufferSize, pSec);
//named pipe - access for everyone
//return new System.IO.Pipes.NamedPipeServerStream(pipeName, PipeDirection.InOut, NamedPipeServerStream.MaxAllowedServerInstances, PipeTransmissionMode.Message, PipeOptions.Asynchronous);
return new System.IO.Pipes.NamedPipeServerStream(pipeName, PipeDirection.InOut, NamedPipeServerStream.MaxAllowedServerInstances, PipeTransmissionMode.Byte, PipeOptions.Asynchronous);
}
public void Dispose()
{
Shutdown();
}
private void OnClientConnected()
{
LogMsg("OnClientConnected");
//raise event
if (ClientConnected != null)
ClientConnected(this, true);
}
private void OnDataReceived(string data)
{
LogMsg("OnClientConnected");
//raise event
if (DataReceived != null && !String.IsNullOrEmpty(data))
{
if (DataReceived != null)
DataReceived(this, data);
}
}
private void OnMessageReadComplete()
{
LogMsg("OnMessageReadComplete");
//raise event
if (MessageReadComplete != null)
MessageReadComplete(this, true);
}
private void OnOperationCompleted()
{
LogMsg("OnOperationCompleted");
//raise event
if (OperationCompleted != null)
OperationCompleted(this, true);
}
private void OnReadComplete()
{
LogMsg("OnReadComplete");
//raise event
if (ReadComplete != null)
ReadComplete(this, true);
}
private void OnServerShutdown()
{
LogMsg("OnServerShutdown");
//raise event
if (ServerShutdown != null)
ServerShutdown(this, true);
}
private void OnServerStarted()
{
LogMsg("OnServerStarted");
//raise event
if (ServerStarted != null)
ServerStarted(this, true);
}
private async void DoConnectionLoop(IAsyncResult result)
{ //wait for connection, then process the data
if (!result.IsCompleted) return;
if (_pipeServer == null) return;
//IOException = pipe is broken
//ObjectDisposedException = cannot access closed pipe
//OperationCanceledException - read was canceled
//accept client connection
try
{
//client connected - stop waiting for connection
_pipeServer.EndWaitForConnection(result);
OnClientConnected(); //raise event
}
catch (IOException) { RebuildNamedPipe(); return; }
catch (ObjectDisposedException) { RebuildNamedPipe(); return; }
catch (OperationCanceledException) { RebuildNamedPipe(); return; }
while (IsClientConnected)
{
if (_pipeServer == null) break;
try
{
// read from client
string clientMessage = await ReadClientMessageAsync(_pipeServer);
OnDataReceived(clientMessage); //raise event
}
catch (IOException) { RebuildNamedPipe(); return; }
catch (ObjectDisposedException) { RebuildNamedPipe(); return; }
catch (OperationCanceledException) { RebuildNamedPipe(); return; }
}
//raise event
OnOperationCompleted();
if (!ShutdownWhenOperationComplete)
{
//client disconnected. start listening for clients again
if (_pipeServer != null)
RebuildNamedPipe();
}
else
{
Shutdown();
}
}
private void LogMsg(string msg)
{
//ToDo: log message
string output = String.Format("{0} - {1}", DateTime.Now.ToString("yyyy/MM/dd HH:mm:ss"), msg);
//ToDo: uncomment this line, if desired
//Debug.WriteLine(output);
}
private void RebuildNamedPipe()
{
Shutdown();
_pipeServer = CreateNamedPipeServerStream(PipeName);
_pipeServer.BeginWaitForConnection(DoConnectionLoop, null);
}
private async Task<string> ReadClientMessageAsync(NamedPipeServerStream stream)
{
byte[] buffer = null;
string clientMsg = string.Empty;
StringBuilder sb = new StringBuilder();
int msgIndex = 0;
int read = 0;
LogMsg("Reading message...");
if (stream.ReadMode == PipeTransmissionMode.Byte)
{
LogMsg("PipeTransmissionMode.Byte");
//byte mode ignores message boundaries
do
{
//create instance
buffer = new byte[_bufferSize];
read = await stream.ReadAsync(buffer, 0, buffer.Length);
if (read > 0)
{
clientMsg = Encoding.UTF8.GetString(buffer, 0, read);
//string clientMsg = Encoding.Default.GetString(buffer, 0, read);
//remove newline
//clientMsg = System.Text.RegularExpressions.Regex.Replace(clientString, #"\r\n|\t|\n|\r|", "");
//LogMsg("clientMsg [" + msgIndex + "]: " + clientMsg);
sb.Append(clientMsg);
msgIndex += 1; //increment
}
} while (read > 0);
//raise event
OnReadComplete();
OnMessageReadComplete();
}
else if (stream.ReadMode == PipeTransmissionMode.Message)
{
LogMsg("PipeTransmissionMode.Message");
do
{
do
{
//create instance
buffer = new byte[_bufferSize];
read = await stream.ReadAsync(buffer, 0, buffer.Length);
if (read > 0)
{
clientMsg = Encoding.UTF8.GetString(buffer, 0, read);
//string clientMsg = Encoding.Default.GetString(buffer, 0, read);
//remove newline
//clientMsg = System.Text.RegularExpressions.Regex.Replace(clientString, #"\r\n|\t|\n|\r|", "");
//LogMsg("clientMsg [" + msgIndex + "]: " + clientMsg);
sb.Append(clientMsg);
msgIndex += 1; //increment
}
} while (!stream.IsMessageComplete);
//raise event
OnMessageReadComplete();
} while (read > 0);
//raise event
OnReadComplete();
LogMsg("message completed");
}
return sb.ToString();
}
private void Shutdown()
{
LogMsg("Shutting down named pipe server");
if (_pipeServer != null)
{
try { _pipeServer.Close(); } catch { }
try { _pipeServer.Dispose(); } catch { }
_pipeServer = null;
}
}
public void StartServer(object obj = null)
{
LogMsg("Info: Starting named pipe server...");
_pipeServer = CreateNamedPipeServerStream(PipeName);
_pipeServer.BeginWaitForConnection(DoConnectionLoop, null);
}
public void StopServer()
{
Shutdown();
OnServerShutdown(); //raise event
LogMsg("Info: Server shutdown.");
}
}
}
Next, I've created a "Helper" class that contains the code to start the named pipe server, run the command using Process, and return the data. There are three ways to get the data. It's returned by the method, one can subscribe to the "DataReceived" event, or once the method completes, the data will be in property "Data".
Helper.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.IO.Pipes;
using System.IO;
using System.Threading;
namespace ProcessTest
{
public class Helper : IDisposable
{
public delegate void EventHandlerDataReceived(object sender, string data);
//event that subscribers can subscribe to
public event EventHandlerDataReceived DataReceived;
private StringBuilder _sbData = new StringBuilder();
private HelperNamedPipeServer _helperNamedPipeServer = null;
private bool _namedPipeServerOperationComplete = false;
public string Data { get; private set; } = string.Empty;
public Helper()
{
}
private void OnDataReceived(string data)
{
if (!String.IsNullOrEmpty(data) && DataReceived != null)
{
DataReceived(this, data);
//Debug.Write("Data: " + data);
}
}
public void Dispose()
{
ShutdownNamedPipeServer();
}
public async Task<string> RunSambaNamedPipesAsync(string fqExePath, string arguments, string pipeName = "sam-ba-pipe", string serverName = ".", bool runAsAdministrator = false)
{
string result = string.Empty;
string tempArguments = arguments;
//re-initialize
_namedPipeServerOperationComplete = false;
_sbData = new StringBuilder();
Data = string.Empty;
if (String.IsNullOrEmpty(fqExePath))
{
Debug.WriteLine("fqExePath not specified");
return "fqExePath not specified";
}
//create new instance
_helperNamedPipeServer = new HelperNamedPipeServer(pipeName);
_helperNamedPipeServer.ShutdownWhenOperationComplete = true;
//subscribe to events
_helperNamedPipeServer.DataReceived += HelperNamedPipeServer_DataReceived;
_helperNamedPipeServer.OperationCompleted += HelperNamedPipeServer_OperationCompleted;
//start named pipe server on it's own thread
Thread t = new Thread(_helperNamedPipeServer.StartServer);
t.Start();
//get pipe name to use with Process
//this is where output from the process
//will be redirected to
string fqNamedPipe = string.Empty;
if (String.IsNullOrEmpty(serverName))
{
fqNamedPipe = String.Format(#"\\{0}\pipe\{1}", serverName, pipeName);
}
else
{
fqNamedPipe = String.Format(#"\\{0}\pipe\{1}", ".", pipeName);
}
//redirect both StandardOutput and StandardError to named pipe
if (!arguments.Contains("2>&1"))
{
tempArguments += String.Format(" {0} {1} {2}", #"1>", fqNamedPipe, #"2>&1");
}
//run Process
RunProcess(fqExePath, tempArguments, runAsAdministrator);
while (!_namedPipeServerOperationComplete)
{
await Task.Delay(125);
}
//set value
Data = _sbData.ToString();
return Data;
}
public void RunProcess(string fqExePath, string arguments, bool runAsAdministrator = false)
{
if (String.IsNullOrEmpty(fqExePath))
{
Debug.WriteLine("fqExePath not specified");
throw new Exception( "Error: fqExePath not specified");
}
//create new instance
ProcessStartInfo startInfo = new ProcessStartInfo(fqExePath, arguments);
if (runAsAdministrator)
{
startInfo.Verb = "runas"; //elevates permissions
}//if
//set environment variables
//pStartInfo.EnvironmentVariables["SomeVar"] = "someValue";
startInfo.RedirectStandardError = false;
startInfo.RedirectStandardOutput = false;
startInfo.RedirectStandardInput = false;
startInfo.UseShellExecute = true; //use ShellExecute instead of CreateProcess
startInfo.WindowStyle = ProcessWindowStyle.Hidden;
startInfo.ErrorDialog = false;
startInfo.WorkingDirectory = System.IO.Path.GetDirectoryName(fqExePath);
using (Process p = Process.Start(startInfo))
{
//start
p.Start();
//waits until the process is finished before continuing
p.WaitForExit();
}
}
private void HelperNamedPipeServer_OperationCompleted(object sender, bool e)
{
//Debug.WriteLine("Info: Named pipe server - Operation completed.");
//set value
Data = _sbData.ToString();
//set value
_namedPipeServerOperationComplete = true;
}
private void HelperNamedPipeServer_DataReceived(object sender, string data)
{
Debug.WriteLine("Info: Data received from named pipe server.");
if (!String.IsNullOrEmpty(data))
{
//append
_sbData.Append(data.TrimEnd('\0'));
//send data to subscribers
OnDataReceived(data);
}
}
private void ShutdownNamedPipeServer()
{
Debug.WriteLine("Info: ShutdownNamedPipeServer");
try
{
if (_helperNamedPipeServer != null)
{
//unsubscribe from events
_helperNamedPipeServer.DataReceived -= HelperNamedPipeServer_DataReceived;
_helperNamedPipeServer.OperationCompleted -= HelperNamedPipeServer_OperationCompleted;
_helperNamedPipeServer.Dispose();
_helperNamedPipeServer = null;
}
}
catch (Exception ex)
{
}
}
}
}
Usage:
private async void btnRunUsingNamedPipes_Click(object sender, EventArgs e)
{
//Button name: btnRunUsingNamedPipes
using (Helper helper = new Helper())
{
//subscribe to event
helper.DataReceived += Helper_DataReceived;
var result = await helper.RunSambaNamedPipesAsync(#"C:\Windows\system32\cmd.exe", #"/c C:\Temp\sam-ba_3.5\sam-ba.exe -p serial:COM5 -d sama5d3 -m version");
Debug.WriteLine("Result: " + result);
//unsubscribe from event
helper.DataReceived -= Helper_DataReceived;
}
}
private void Helper_DataReceived(object sender, string data)
{
//System.Diagnostics.Debug.WriteLine(data);
//RichTextBox name: richTextBoxOutput
if (richTextBoxOutput.InvokeRequired)
{
richTextBoxOutput.Invoke((MethodInvoker)delegate
{
richTextBoxOutput.Text = data;
richTextBoxOutput.Refresh();
});
}
}
Resources:
When do we need to set ProcessStartInfo.UseShellExecute to True?
Redirecting error messages from Command Prompt: STDERR/STDOUT
PipeTransmissionMode.Message: How do .NET named pipes distinguish between messages?
WellKnownSidType Enum

Get data from program that launched from vscode

I have a vscode extension that launches an executable, i know how to pass data from vscode to my program but not the other way around.
// class that launches the exe
class Execute {
constructor(private _extensionPath: string) {}
public Launch() {
console.log('213');
Process.exec(
`WpfApp1.exe true`,
{ cwd: this._extensionPath },
(error: Error, stdout: string, stderr: string) => {
if (stdout.length === 0) {
return;
}
}
);
}
}
// calling the class
let exe = new Execute(
vscode.extensions.getExtension('author.extension').extensionPath
);
exe.Launch();
c# receiving data
void App_Startup(object sender, StartupEventArgs e)
{
try
{
test_p = e.Args[0];
if (test_p == "true")
{
}
}
catch { MessageBox.Show("fail"); }
}
how can i send data from the c# application to the vscode extension?
calling a function in vscode would be even better.

you can also run a executable with c#:
public static string[] Cmd(bool xWaitForExecution, params string[] xCommands)
{
//PROCESS CMD
if (xCommands == null || xCommands.Length == 0) return null;
ProcessStartInfo info = new ProcessStartInfo("cmd.exe");
info.CreateNoWindow = true;
info.UseShellExecute = false;
info.RedirectStandardInput = true; //STD INPUT
info.RedirectStandardOutput = true; //STD OUTPUT
info.RedirectStandardError = true; //STD ERROR
Process process = Process.Start(info);
//WRITE COMMANDS
using (StreamWriter sw = process.StandardInput)
if (sw.BaseStream.CanWrite)
foreach (string cmd in xCommands)
sw.WriteLine(cmd);
//GET OUTPUT & ERROR
if (!xWaitForExecution) return null;
string output = process.StandardOutput.ReadToEnd(); //OUTPUT
string error = process.StandardError.ReadToEnd(); //ERROR
string exit = process.ExitCode.ToString(); //EXIT CODE
process.Close();
return new string[] { output, error, exit };
}
The function runs cmd64.exe and it should use like:
//Call Cmd, true means the c# application will wait for the complete execute of your executable (needed to obtain output values)
string[] ret = Cmd(true, "\\mypath\\my.exe -Argument1 -Argument2"); //Passing arguments depends on your executable
string output = ret[0];
Console.WriteLine(ret[0]) //printed arguments from your executable (for instance python: print("Argument1"))
It is not completly clear why you need the VS Code extension to execute an executable. This is a working alternative to run executables on windows from c#.

Send Data:
private void AnyEvent(object sender, MouseEventArgs e)
{
const String output = "testOutput,testOutput2";
Console.Write(output);
}
Receive Data:
import * as Process from 'child_process';
// put the code below in a class or function.
Process.exec(
`app.exe input`,
{ cwd: "Path to folder of the executable" },
(error, stdout: string, stderr: string) => {
const output = stdout.split(','); //it only outputs data after the exe is closed!?
if (output[0] === 'testOutput') {
console.log('Output: == "testOutput"');
}
}
);
Example Project should work after running npm i if not open an issue or comment below.

Keeping console app alive and event-driven while processes run in the background

I am trying to write a console application that acts as a "job manager" by running processes in the background. These processes would be running JScript files with arguments passed in. This console application will be distributed across many machines, and will pull from a centralized source (ie. database) to get jobs. The purpose of this application is to eliminate the need for individualized batch files on all of these machines.
I am having trouble keeping the application alive. In the code that I included, you can see in my main function that I am making an initial call to the JobManger's StartNewJobs() method. After this initial call to this method, I'd like my application to then be event-driven, only waking up and running when a process has exited, allowing me to start a new process. The problem I am running into is that once the main() function finishes (when the initial StartNewJobs() method finishes) the console closes and the program ends.
My question is what is the proper way to keep my console application alive and allow it to be event-driven rather than procedural? I know I can probably throw in a while(true) at the end of the main function, but that seems sloppy and incorrect.
Batch file we are trying to replace:
C:\Windows\SysWOW64\cscript.exe c:\temp\somejscriptfile.js 49f1bdd8-5e6b-40cc-92bc-eb20c237a959
C:\Windows\SysWOW64\cscript.exe c:\temp\somejscriptfile.js 654e3783-a1b6-43be-8027-c7d060bf131f
...
Program.cs:
using DistributedJobs.Data;
using DistributedJobs.Logging;
using DistributedJobs.Models;
using Microsoft.Practices.EnterpriseLibrary.Common.Configuration;
using Microsoft.Practices.EnterpriseLibrary.ExceptionHandling;
using System;
namespace DistributedJobs
{
class Program
{
static void Main(string[] args)
{
//Get intial objects/settings
ILogger logger = new Logger(Properties.Settings.Default.LoggingLevel, EnterpriseLibraryContainer.Current.GetInstance<ExceptionManager>());
IDataProvider dataProvider = new SQLDataProvider();
DMSPollingJobType availableJobTypes = DMSPollingJobType.FlatFile;
if (Properties.Settings.Default.SupportsVPN)
{
availableJobTypes |= DMSPollingJobType.VPN;
}
String executableLocation = Properties.Settings.Default.ExecutableLocation;
String jsLocation = Properties.Settings.Default.JSLocation;
Int32 maxProcesses = Properties.Settings.Default.MaxProcesses;
//Create job manager and start new processes/jobs
DMSJobManager jobManager = new DMSJobManager(logger, dataProvider, availableJobTypes, executableLocation, jsLocation, maxProcesses);
jobManager.StartNewJobs();
}
}
}
JobManager.cs:
using DistributedJobs.Models;
using System.Diagnostics;
using System;
using System.Collections.Generic;
using DistributedJobs.Logging;
namespace DistributedJobs.Data
{
public class JobManager
{
private IDataProvider DataProvider;
private ILogger Logger;
private Dictionary<Job, Process> RunningProcesses;
private JobType AvailableJobTypes;
private String ExecutableLocation;
private String JSLocation;
private Int32 MaxProcesses;
public Boolean CanStartNewJob
{
get
{
Boolean canStartNewJob = false;
if (RunningProcesses.Count < MaxProcesses)
{
canStartNewJob = true;
}
foreach (KeyValuePair<Job, Process> entry in RunningProcesses)
{
if (entry.Key.JobType != JobType.FlatFile)
{
canStartNewJob = false;
break;
}
}
return canStartNewJob;
}
}
public JobManager(ILogger logger, IDataProvider dataProvider, JobType availableJobTypes, String executableLocation, String jsLocation, Int32 maxProcesses)
{
Logger = logger;
DataProvider = dataProvider;
RunningProcesses = new Dictionary<Job, Process>();
AvailableJobTypes = availableJobTypes;
ExecutableLocation = executableLocation;
JSLocation = jsLocation;
MaxProcesses = maxProcesses;
}
public void StartNewJobs()
{
while (CanStartNewJob)
{
Job newJob = DataProvider.GetNextScheduledJob(AvailableJobTypes);
if (newJob != null)
{
Process newProcess = CreateNewProcess(newJob);
RunningProcesses.Add(newJob, newProcess);
newProcess.Start();
}
}
}
public Process CreateNewProcess(Job job)
{
ProcessStartInfo startInfo = new ProcessStartInfo();
startInfo.FileName = ExecutableLocation;
startInfo.Arguments = JSLocation + " " + job.JobID.ToString();
startInfo.UseShellExecute = false;
Process retProcess = new Process()
{
StartInfo = startInfo,
EnableRaisingEvents = true
};
retProcess.Exited += new EventHandler(JobFinished);
return retProcess;
}
public void JobFinished(object sender, EventArgs e)
{
Job finishedJob = null;
foreach (KeyValuePair<Job, Process> entry in RunningProcesses)
{
if ((Process)sender == entry.Value)
{
finishedJob = entry.Key;
break;
}
}
if (finishedJob != null)
{
RunningProcesses.Remove(finishedJob);
StartNewJobs();
}
}
}
}

You could try using Application.Run()(System.Windows.Forms). This will start a standard message loop.
So at the end of your Main method just add a Application.Run():
static void Main(string[] args)
{
//Get intial objects/settings
ILogger logger = new Logger(Properties.Settings.Default.LoggingLevel, EnterpriseLibraryContainer.Current.GetInstance<ExceptionManager>());
IDataProvider dataProvider = new SQLDataProvider();
DMSPollingJobType availableJobTypes = DMSPollingJobType.FlatFile;
if (Properties.Settings.Default.SupportsVPN)
{
availableJobTypes |= DMSPollingJobType.VPN;
}
String executableLocation = Properties.Settings.Default.ExecutableLocation;
String jsLocation = Properties.Settings.Default.JSLocation;
Int32 maxProcesses = Properties.Settings.Default.MaxProcesses;
//Create job manager and start new processes/jobs
DMSJobManager jobManager = new DMSJobManager(logger, dataProvider, availableJobTypes, executableLocation, jsLocation, maxProcesses);
jobManager.StartNewJobs();
// start message loop
Application.Run();
}

How to wait for the remote exe to complete - c#

The code looks lengthy but it's a simple program.
I have built a console app (TakeScreenshots) that will take website screenshots from firefox, chrome & ie in that order & save them in a folder. When I manually run TakeScreenshots.exe, all 3 screenshots are saved.
Now, I have built another console app (MyApp) that will execute TakeScreenshots.exe. But in this way, only the firefox screenshot is saved and not of the other 2. There are no exceptions. It just says "Process Complete". I guess, MyApp is not waiting for the TakeScreenshots to complete.
How can I fix this.
[TakeScreenshots will later be placed in few remote computers & run by MyApp]
TakeScreenshots code:
private static string[] WebDriversList = ["firefox","chrome","internetexplorer"];
private static void TakeAPic()
{
string url = "http://www.google.com";
string fileNamePrefix = "Test";
string snapSavePath = "D:\\Pics\\";
foreach (string wd in WebDriversList)
{
IWebDriver NewDriver = null;
switch (wd.ToLower())
{
case "firefox":
using (NewDriver = new FirefoxDriver())
{
if (NewDriver != null)
{
CaptureScreenshot(NewDriver, url, fileNamePrefix, snapSavePath);
}
}
break;
case "chrome":
using (NewDriver = new ChromeDriver(WebDriversPath))
{
if (NewDriver != null)
{
CaptureScreenshot(NewDriver, url, fileNamePrefix, snapSavePath);
}
}
break;
case "internetexplorer":
using (NewDriver = new InternetExplorerDriver(WebDriversPath))
{
if (NewDriver != null)
{
CaptureScreenshot(NewDriver, url, fileNamePrefix, snapSavePath);
}
}
break;
}
if (NewDriver != null)
{
NewDriver.Quit();
}
}
}
private static void CaptureScreenshot(IWebDriver driver,string url,string fileNamePrefix,
string snapSavePath)
{
driver.Navigate().GoToUrl(url);
Screenshot ss = ((ITakesScreenshot)driver).GetScreenshot();
ICapabilities capabilities = ((RemoteWebDriver)driver).Capabilities;
ss.SaveAsFile(snapSavePath + fileNamePrefix + "_" + capabilities.BrowserName + ".png",
ImageFormat.Png);
}
MyApp code:
static void Main(string[] args)
{
ExecuteTakeScreenshot();
Console.WriteLine("PROCESS COMPLETE");
Console.ReadKey();
}
private static void ExecuteTakeScreenshot()
{
ProcessStartInfo Psi = new ProcessStartInfo("D:\\PsTools\\");
Psi.FileName = "D:\\PsTools\\PsExec.exe";
Psi.Arguments = "/C \\DESK101 D:\\Release\\TakeScreenshots.exe";
Psi.UseShellExecute = false;
Psi.RedirectStandardOutput = true;
Psi.RedirectStandardInput = true;
Process.Start(Psi).WaitForExit();
}
Update:
It was my mistake. Initially WebDriversPath was assigned "WebDrivers/". When I changed it to the actual path "D:\WebDrivers\", it worked. But I still dont understand how it worked when TakeScreenshots.exe was run manually and it doesn't when run from another console

In similar problems I have had success with waiting for input idle first. Like this:
Process process = Process.Start(Psi);
process.WaitForInputIdle();
process.WaitForExit();
You could try this. For me it was needed to print a pdf using Adobe Reader and not close it to early afterwards.
Example:
Process process = new Process();
process.StartInfo.FileName = DestinationFile;
process.StartInfo.Verb = "print";
process.Start();
// In case of Adobe Reader the following statement is needed:
process.WaitForInputIdle();
process.WaitForExit(2000);
process.WaitForInputIdle();
process.Kill();

How to pass parameter to cmd.exe and get the result back into C# Windows application

GUI of my program.
I want to pass parameter as dir to cmd.exe with .NET Process class and get correct output into the c# program.
Source:
using System;
using System.Diagnostics;
using System.Threading;
using System.Windows.Forms;
namespace MyCmd
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
comboBox1.Items.Add("cmd.exe");
comboBox1.SelectedIndex = 0;
}
private Thread t;
private void button1_Click(object sender, EventArgs e)
{
object[] param = new object[] { comboBox1.Text, txtParams.Text };
if (param.Length > 0 && comboBox1.Text != "")
{
t = new Thread(() => doTask(param));
t.IsBackground = true;
t.Start();
}
else
{
MessageBox.Show("Invalid parameters!");
}
}
private void doTask(object[] param)
{
Process proc = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = param[0].ToString(),
Arguments = param[1].ToString(),
UseShellExecute = false,
RedirectStandardOutput = true,
CreateNoWindow = true
}
};
proc.Start();
while (!proc.StandardOutput.EndOfStream)
{
string line = proc.StandardOutput.ReadLine();
this.Invoke((MethodInvoker)delegate
{
txtResponse.Text += line + Environment.NewLine;
});
}
proc.WaitForExit();
}
}
}
This is output
I want to launch cmd.exe and then enter dir then, it list the directory info into my application.
UPDATED:
Source updated:
using System;
using System.Diagnostics;
using System.Threading;
using System.Windows.Forms;
namespace MyCmdV2
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
comboBox1.Items.Add("cmd.exe");
comboBox1.SelectedIndex = 0;
proc = new Process();
proc.StartInfo = new ProcessStartInfo
{
FileName = "cmd.exe",
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardInput = true,
CreateNoWindow = true
};
proc.Start();
}
private Process proc;
private Thread t;
private void button1_Click(object sender, EventArgs e)
{
t = new Thread(() => doIt());
t.IsBackground = true;
t.Start();
}
private void doIt()
{
if (txtParams.Text.ToLower() == "cls")
{
txtResponse.Text = "";
}
else
{
proc.StandardInput.WriteLine(#txtParams.Text);
while (!proc.StandardOutput.EndOfStream)
{
string line = proc.StandardOutput.ReadLine();
this.Invoke((MethodInvoker)delegate
{
txtResponse.Text += line + Environment.NewLine;
});
}
proc.WaitForExit();
}
}
}
}
This is an example query only.I need to run any exe program with parameters.
List all my program in to ComboBox and then pass the parameters in TextBox.
Some time i need to pass parameter as IP address like this. \\192.168.1.2
Ex: psinfo.exe \\192.168.1.2
But C# it's store as \\\\192.168.1.2. How to remove that escape sequence characters in parameter.

Your code looks fine as it did pass "dir" as an argument. Your code did the same as executing "cmd.exe dir". Are you wanting to launch cmd.exe and then enter dir so it lists the directory structure? If so, you'll need to redirect the standard input like you did the standard output and then write dir to the input stream.
UPDATE
Code to execute your command:
private void doTask(object[] param)
{
Process proc = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = param[0].ToString(),
//remove this line, it's not needed
//Arguments = param[1].ToString(),
UseShellExecute = false,
RedirectStandardOutput = true,
//Add this line so you can write commands
RedirectStandardInput = true,
CreateNoWindow = true
}
};
proc.Start();
//now write your command with WriteLine so it mimics an enter press
proc.StandardInput.WriteLine(param[1].ToString());
while (!proc.StandardOutput.EndOfStream)
{
string line = proc.StandardOutput.ReadLine();
this.Invoke((MethodInvoker)delegate
{
txtResponse.Text += line + Environment.NewLine;
});
}
proc.WaitForExit();
}

Seems that standard input and output are already correctly set. If you take a look in the manual here, you should get a better picture of what to do. I performed two tests using the Windows run feature. The first one was cmd dir, the same as yours and it failed miserably. The second one was cmd /K dir, which produced what I think is the intended output. The used switch leaves the cmd window open, check the manual for other switches that close the process after the command is done.
One more thing to note, while most of the things run in the cmd are programs, commands like "dir" and "cd" are just that, commands, and you won't find dir.exe or cd.exe anywhere.

When you invoke cmd.exe, you need to use the /c parameter. Go to your command prompt and type this:
cmd.exe dir
This is essentially what you do in your application, which gives to the same output as in your application.
Now try:
cmd.exe /c dir
This is the output you are looking for. So, you tell cmd.exe to run the command dir by using the /c param.
So, your ProcessStartInfo.Arguments would look like this:
private void doTask(object[] param)
{
Process proc = new Process
{
StartInfo = new ProcessStartInfo
{
FileName = param[0].ToString(),
Arguments = "/c " + param[1].ToString(),
UseShellExecute = false,
RedirectStandardOutput = true,
CreateNoWindow = true
}
};
proc.Start();
I have recreated your solution, and this works as expected.