My C# program needs to send data to a 3rd-party program via its standard input. However, the program waits for the input stream to reach EOF before processing. Here my code:
// Starts the process.
var process = new Process();
process.StartInfo.CreateNoWindow = true;
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardInput = true;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.FileName = "foo.exe";
process.Start();
// Sends data to child process.
var input = process.StandardInput;
input.WriteLine("eval('2 * PI')");
input.Flush();
input.Close();
// Reads the result.
var output = process.StandardOutput;
var result = output.ReadLine();
The child program won't do anything and my C# code becomes stuck at output.ReadLine() call. However if I kill the C# process, then the child starts to work exactly on the data I've sent. How can I make the child encounter an EOF while I'm still alive?
StreamWriter might not be sending an actual eof when it closes the stream. You could try writing your own to the stream just before you close it. Something like this might work:
input.Write((char)26);
You may have to find out what the process expects for eof.
Related
I am redirecting Process.StandardOutput and Process.StandardError from a System.Diagnostics.Process that uses 7zip to extract and zip archives and am unable to read the progress from the process.
It appears, 7Zip like some other applications, emit are backspace and delete characters to partially write a line data and then and delete the written characters using backspace and delete in order o show progress. I am trying to read those partial line outputs from the target process am unable to do so. However, I may be wrong in this assumption.
What I have tried:
var process = new Process
{
StartInfo =
{
FileName = command,
Arguments = arguments,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true
}
};
process.Start();
After the above code block, I have tried using various methods of reading the data.
I have tried the async event handlers:
process.OutputDataReceived += (sender, args) => { Console.WriteLine(args.Data); };
process.ErrorDataReceived += (sender, args) => { Console.WriteLine(args.Data); };
process.BeginOutputReadLine();
process.BeginErrorReadLine();
I have tried using the async methods of the StandardOutput:
while (!process.StandardOutput.EndOfStream)
{
char[] buffer = new char[256];
int read = process.StandardOutput.ReadAsync(buffer, 0, buffer.Length).Result;
Console.Write(buffer, 0, read);
}
and
process.StandardOutput.BaseStream.CopyToAsync(Console.OpenStandardOutput());
And have tried using the async methods of the underlying base stream.
while (!process.StandardOutput.EndOfStream)
{
byte[] buffer = new byte[256];
int read = process.StandardOutput.BaseStream.ReadAsync(buffer, 0, buffer.Length).Result;
string data = Encoding.UTF8.GetString(buffer, 0, read);
Console.Write(data);
}
As an example, run 7Zip from the terminal with the following command:
"c:\program files\7-zip\7z.exe" x -o"C:\target" "K:\Disk_23339.secure.7z"
This shows progress output when running directly in a command prompt, with each success progress incrementing overwriting the previous:
It then uses backspace chars to overwrite the previous progress.
Running the same command and arguments using Process.Start()
var process = new Process
{
StartInfo =
{
FileName = "c:\program files\7-zip\7z.exe",
Arguments = 'x -o"C:\target" \"K:\Disk_23339.secure.7z\"",
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true
}
};
process.Start();
process.StandardOutput.BaseStream.CopyToAsync(Console.OpenStandardOutput());
process.WaitForExit();
When running this and attempting to read the redirected standard output of the process characters that are not emitted from the source process that do not contain a new line (either by linefeed or carriage return + line feed) are output to the standard output or standard error of System.Diagnostics.Process and hence never written to the console.
7zip of course is just one example. This issue also occurs with numerous PowerShell and Python scripts.
Is there anyway to read these characters from Process.StandardOutput and Process.StandardError.
I am not sure but I think the issue is the underlying stream reader reads one line at a time and never returns these partial lines because they never include line ending characters.
Though the post had been months, in case your problem (not able to get progress from 7zip when execute in command line) still exists, use the "-bsp1" switch in command line argument.
I was also looking for the solution of same issue and just got it (tested successfully in my case) right. By redirect StandardOutput, repeatedly do cmd.StandardOutput.ReadLine() (the Synchronous method, I tested this method instead of asynchronous method that use EventHandler, but I think async method would work too), and use RegExp to detect the progress to update my UI.
My command (run in .NET)
C:\Program Files\7-Zip\7z.exe a -t7z -mx=9 -bsp1 "G:\Temp\My7zTest_Compressed.7z" "G:\SourceFolder"
Credit to #justintjacob
How to show extraction progress of 7zip inside cmd?
I have graphviz dot.exe file that I call with parameter -Tpng (output type is png, but I don't care if it is in png, bmp, or any other). I start it in C# code:
ProcessStartInfo psi = new ProcessStartInfo();
psi.FileName = path;
psi.UseShellExecute = false;
psi.Arguments = "-Tpng";
psi.RedirectStandardInput = true;
psi.RedirectStandardOutput = true;
psi.CreateNoWindow = true;
Process p = Process.Start(psi);
Then, I write input
p.StandardInput.WriteLine(input);
input is defined before, it's a string. Input is valid, tested manually.
Then, I need to read output that graphviz prints into standart output and parse it to Image.
I've tried to read memory stream, but I was either unable to read it, or, after reading, the memory stream was locked (threw exception when tried Image.FromStream(myMemoryStream);).
I was able to load output to string
string output = "";
while (true)
{
string newOutput = p.StandardOutput.ReadLine();
output += newOutput;
if (newOutput == String.Empty)
break;
}
I've tried to parse this string as described in this answer, but it threw exception (string is not valid).
How can I get Image from the dot.exe output?
From the comments it seems the program is expecting the StandardInput to be finished before returning the content. Close the StandardInput to achieve it:
p.StandardInput.WriteLine(input);
p.StandardInput.BaseStream.Close();
I'm trying to do something that seems like it should be relatively simple: Call jpegoptim from C#.
I can get it to write to disk fine, but getting it to accept a stream and emit a stream has so far eluded me - I always end up with 0 length output or the ominous "Pipe has been ended."
One approach I tried:
var processInfo = new ProcessInfo(
jpegOptimPath,
"-m" + quality + " -T1 -o -p --strip-all --all-normal"
);
processInfo.CreateNoWindow = true;
processInfo.WindowStyle = ProcessWindowStyle.Hidden;
processInfo.UseShellExecute = false;
processInfo.RedirectStandardInput = true;
processInfo.RedirectStandardOutput = true;
processInfo.RedirectStandardError = true;
using(var process = Process.Start(processInfo))
{
await Task.WhenAll(
inputStream.CopyToAsync(process.StandardInput.BaseStream),
process.StandardOutput.BaseStream.CopyToAsync(outputStream)
);
while (!process.HasExited)
{
await Task.Delay(100);
}
// Do stuff with outputStream here - always length 0 or exception
}
I've also tried this solution:
http://alabaxblog.info/2013/06/redirectstandardoutput-beginoutputreadline-pattern-broken/
using (var process = new Process())
{
process.StartInfo.UseShellExecute = false;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.RedirectStandardError = true;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.FileName = fileName;
process.StartInfo.Arguments = arguments;
process.Start();
//Thread.Sleep(100);
using (Task processWaiter = Task.Factory.StartNew(() => process.WaitForExit()))
using (Task<string> outputReader = Task.Factory.StartNew(() => process.StandardOutput.ReadToEnd()))
using (Task<string> errorReader = Task.Factory.StartNew(() => process.StandardError.ReadToEnd()))
{
Task.WaitAll(processWaiter, outputReader, errorReader);
standardOutput = outputReader.Result;
standardError = errorReader.Result;
}
}
Same problem. Output length 0. If I let jpegoptim run without the output redirect I get what I'd expect - an optimized file - but not when I run it this way.
There's gotta be a right way to do this?
Update: Found a clue - don't I feel sheepish - jpegoptim never supported piping to stdin until an experimental build in 2014, fixed this year. The build I have is from an older library, dated 2013. https://github.com/tjko/jpegoptim/issues/6
A partial solution - see deadlock issue below. I had multiple problems in my original attempts:
You need a build of jpegoptim that will read and write to pipes instead of files-only. As mentioned builds prior to mid-2014 can't do it. The github "releases" of jpegoptim are useless zips of source, not built releases, so you'll need to look elsewhere for actual built releases.
You need to call it properly, passing --stdin and --stdout, and depending on how you'll be responding to it, avoid parameters that might cause it to write nothing, like -T1 (which will, when optimization is going to only be 1% or less, cause it to emit nothing to stdout).
You need to perform the non-trivial task of: Redirecting both input and output on the Process class
and avoiding a Buffer overflow on the input side that will get you 0 output once again - the obvious stream.CopyToAsync() overruns Process's very limited 4096 byte (4K) buffer and gets you nothing.
So many routes to nothing. None signalling why.
var processInfo = new ProcessInfo(
jpegOptimPath,
"-m" + quality + " --strip-all --all-normal --stdin --stdout",
);
processInfo.CreateNoWindow = true;
processInfo.WindowStyle = ProcessWindowStyle.Hidden;
processInfo.UseShellExecute = false;
processInfo.RedirectStandardInput = true;
processInfo.RedirectStandardOutput = true;
processInfo.RedirectStandardError = true;
using(var process = new Process())
{
process.StartInfo = processInfo;
process.Start();
int chunkSize = 4096; // Process has a limited 4096 byte buffer
var buffer = new byte[chunkSize];
int bufferLen = 0;
var inputStream = process.StandardInput.BaseStream;
var outputStream = process.StandardOutput.BaseStream;
do
{
bufferLen = await input.ReadAsync(buffer, 0, chunkSize);
await inputStream.WriteAsync(buffer, 0, bufferLen);
inputStream.Flush();
}
while (bufferLen == chunkSize);
do
{
bufferLen = await outputStream.ReadAsync(buffer, 0, chunkSize);
if (bufferLen > 0)
await output.WriteAsync(buffer, 0, bufferLen);
}
while (bufferLen > 0);
while(!process.HasExited)
{
await Task.Delay(100);
}
output.Flush();
There are some areas for improvement here. Improvements welcome.
Biggest problem: On some images, this deadlocks on the outputStream.ReadAsync line.
It all belongs in separate methods to break it up - I unrolled a bunch of methods to keep this example simple.
There are a bunch of flushes that may not be necessary.
The code here is meant to handle anything that streams in and out. The 4096 is a hard limit that any Process will deal with, but the assumption that all the input goes in, then all the output comes out is likely a bad one and based on my research could result in a deadlock for other types of process. It appears that jpegoptim behaves in this (very buffered, very unpipe-like...) way when passed --stdin --stdout however, so, this code copes well for this specific task.
I have tried calling a Process(console application) using the following code:
ProcessStartInfo pi = new ProcessStartInfo();
pi.UseShellExecute = false;
pi.RedirectStandardOutput = true;
pi.CreateNoWindow = true;
pi.FileName = #"C:\fakepath\go.exe";
pi.Arguments = "FOO BAA";
Process p = Process.Start(pi);
StreamReader streamReader = p.StandardOutput;
char[] buf = new char[256];
string line = string.Empty;
int count;
while ((count = streamReader.Read(buf, 0, 256)) > 0)
{
line += new String(buf, 0, count);
}
It works for only some cases.
The file that does not work has a size of 1.30 mb,
I don't know if that is the reason for it not working correctly.
line returns an empty string.
I hope this is clear.
Can someone point out my error? Thanks in advance.
A couple thoughts:
The various Read* methods of streamreader require you to ensure that your app has completed before they run, otherwise you may get no output depending on timing issues. You may want to look at the Process.WaitForExit() function if you want to use this route.
Also, unless you have a specific reason for allocating buffers (pain in the butt IMO) I would just use readline() in a loop, or since the process has exited, ReadToEnd() to get the whole output. Neither requires you to have to do arrays of char, which opens you up to math errors with buffer sizes.
If you want to go asynchronous and dump output as you run, you will want to use the BeginOutputReadLine() function (see MSDN)
Don't forget that errors are handled differently, so if for any reason your app is writing to STDERR, you will want to use the appropriate error output functions to read that output as well.
I have a (C#) console application which maintains a state. The state can be altered by feeding the application with various input through the console. I need to be able to both feed the application with a bit of input, then read the output rinse and repeat.
I create a new process and do all of the normal work of redirecting the input/output. The problem is that after I've sent input and call ReadLine() on the standard output it does not return a value before I call Close() on the standard input after which I cannot write anymore to the input stream.
How can I keep open the input stream while still receiving output?
var process = new Process
{
StartInfo =
{
FileName =
#"blabal.exe",
RedirectStandardInput = true,
RedirectStandardError = true,
RedirectStandardOutput = true,
UseShellExecute = false,
CreateNoWindow = true,
ErrorDialog = false
}
};
process.EnableRaisingEvents = false;
process.Start();
var standardInput = process.StandardInput;
standardInput.AutoFlush = true;
var standardOutput = process.StandardOutput;
var standardError = process.StandardError;
standardInput.Write("ready");
standardInput.Close(); // <-- output doesn't arrive before after this line
var outputData = standardOutput.ReadLine();
process.Close();
process.Dispose();
The console application I'm redirecting IO from is very simple. It reads from the console using Console.Read() and writes to it using Console.Write(). I know for certain that this data is readable, since I have another application that reads from it using standard output / input (not written in .NET).
That is happening because of you are using Write("ready") which is will append a string to the text, instead use WriteLine("ready"). that simple :).