We Keep Coding

Execute all .bat files in the current folder and delete them after execution

I am a newbie programmer wanting to make averaging videos. I have made a program to create n .bat files doing the average of n images, now I would like to execute them as fast as possible.
The .bat files are independent.
I am in a Windows environment.
I have looked at C# multi threading (threadpool, parrallel.for, parralel.foreach etc), but none of the functions there seems to work. I have no illusion that it's me who's doing something wrong though.
Powershell has a function doing what I want, but only for other powershell commands.
The code I have now that mostly works is:
(complete solution at https://github.com/Madsfoto/ParallelExecutionForEach )
var paths = Directory.GetFiles(Directory.GetCurrentDirectory(), "*.bat"); // have a list of all .bat files in the current directory
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.StartInfo.CreateNoWindow = true;
proc.StartInfo.UseShellExecute = false; // above is to not see the cmd window
proc.StartInfo.WorkingDirectory = Directory.GetCurrentDirectory(); // It's easier than having to specify where this program will be run.
Parallel.ForEach(paths, new ParallelOptions { MaxDegreeOfParallelism = 4 }, currentFile => // 4 is set because I have 4 cores to use
{
proc.StartInfo.FileName = currentFile; // Set the currentfile as the one being executed. currentFile is the name of the .bat file to execute
proc.Start(); // execute the .bat file
proc.WaitForExit();
File.Delete(currentFile);
});
I get System.InvalidOperationException: No process is associated with this object and System.UnauthorizedAccessException’s when I run more than 3-4 processes at the same time.
I suspect that it is the WaitForExit() that is giving me problems, but do not have the skills to debug it.
I have looked at Threading.Task as well, but my skill is not good enough to use it.
So the solution I am after is as follows:
Execute either 1 input file with x lines of independent action or x files with 1 action, with a limit of y processes at the same time either set at compile or runtime.
Programming language is not important to me, although my preference is understandable C#.
(The result is something like https://www.youtube.com/watch?v=ph6-6bYTgs0, with n frames averaged together)

The solution was to move the
proc.Start(); // execute the .bat file
proc.WaitForExit();
try
{
File.Delete(FileName);
}
catch
{
}
code into it's own function (with the bookkeeping stuff (defining proc etc)).
The File.Delete() was the culprit, it turns out that there might be a bug in the Parallel.ForEach, but I have been unable to reproduce it reliably (experimenting gives errors ~0.01% of the time), but this way it works. It does require people to rerun the executable, but that is a burden I can justify pushing to the user.
The github link has been updated with a working version.

Execute BATCH script in a programs memory

I am currently faced with a problem, I need to execute a batch script within a programs memory (so it does not have to extract the batch file to a temporary location).
I am open to solutions in C# and C++
Any help would be appreciated

cmd.exe won't run a script from the memory of your process. The options which seem most obvious to me are:
Relax the constraint that stops you extracting the script to a temporary file.
Compress your script into a single line and use cmd.exe /C to execute it. You'll need to use the command separator &&.
Write your own batch command interpreter.
Use a different scripting language.
Options 3 and 4 aren't really very attractive! Option 1 looks pretty good to me but I don't know what's leading to your constraint.

Open a pipe to the command shell and write the program code into that pipe. Here is an example: http://support.microsoft.com/default.aspx?scid=kb;en-us;190351

In C# it's an easy way to use System.Diagnostics for the job.
How!?
Basically, every batch command is an .exe file so you can start it in a separate process.
Some code:
using System.Diagnostics;
static void Main()
{
Process batch;
batch = Process.Start("ping.exe", "localhost");
batch.WaitForExit();
batch.Close();
batch = Process.Start("choice.exe", "");
batch.WaitForExit();
batch.Close();
batch = Process.Start("ping.exe", "localhost -n 10");
batch.WaitForExit();
batch.Close();
}
If you don't want to start every command in a separate process the solution is with a simple stream redirection.
ProcessStartInfo startInfo = new ProcessStartInfo();
startInfo.FileName = #"cmd.exe"; // Specify exe name.
startInfo.UseShellExecute = false;
startInfo.ErrorDialog = false;
startInfo.RedirectStandardInput = true;
//
// Start the process.
//
Process process = Process.Start(startInfo);
string[] batchFile = {"ping localhost", "ping google.com -n 10", "exit"};
int cmdIndex = 0;
while (!process.HasExited)
{
if (process.Threads.Count == 1 && cmdIndex < batchFile.Length)
{
process.StandardInput.WriteLine(batchFile[cmdIndex++]);
}
}

What's a good way to write batch scripts in C#?

strange behaviour on another process via Process.Start(startInfo)

Our C# (V3.5) application needs to call another C++ executable which is from another company. we need to pass a raw data file name to it, it will process that raw data (about 7MB) file and generate 16 result files (about 124K for each).
The code to call that executable is this:
ProcessStartInfo startInfo = new ProcessStartInfo();
startInfo.CreateNoWindow = true;
startInfo.UseShellExecute = false;
startInfo.RedirectStandardError = true;
startInfo.RedirectStandardOutput = true;
startInfo.WindowStyle = ProcessWindowStyle.Hidden;
startInfo.FileName = exePath;
startInfo.Arguments = rawDataFileName;
try
{
Process correctionProcess = Process.Start(startInfo);
correctionProcess.WaitForExit();
}
catch(nvalidOperationException ex)
{
....
}
catch(...)
...
It works fine. Now we have new raw data. After replace the old raw data with the new raw data file. That executable process never return to us. It will hang forever. If we kill our C# application, those result files will be generated in the target directoy. It looks like the executable does create those result files but has issue to write to the disk and return to us until the process is terminated.
It is NOT like this with the old raw data file.
When we run the executable with the new raw data directly (no from our C# app call), it works fine. This means this executable has no problem with the new raw data.
My question 1: what's the possible causes for this behaviour?
Now I change our code with startInfo.UseShellExecute = true; and add startInfo.WorkingDirectory= ..., and disabled
//startInfo.RedirectStandardError = true;
//startInfo.RedirectStandardOutput = true;
Then it works.
My question 2: why use Windows Shell solve this issue?
My question 3: why it works before without using Shell?
My question 4: when we should use Shell and When shouldn't?
thanks,

Several possibilities:
You are redirecting output and error but not reading it. The process will stall when its stdout or stderr buffer fills up to capacity.
The program might be displaying an error message and waiting for a keypress. You are not redirecting input nor check stderr, that keypress will never come.
Some programs, xcopy.exe is a very good example, require stdin to be redirected when you redirect stdout. Although the failure mode for xcopy.exe is an immediate exit without any diagnostic.
Seeing it fixed when you kill your C# program makes the first bullet the likeliest reason.

I know this, it is a very common problem. I has to do with the output, which must be handled asynchronously. You just can't WaitForExit when output exceeds certain amount of data.
You need to add
myStdErr= correctionProcess.StandardError.ReadToEnd();
Only once usually works, if you want to overkill this works ("P" being my Process)
while (!P.HasExited)
stdErr+= P.StandardError.ReadToEnd();

If you don't need the stdout/stderr, just turn the Redirect* properties to false.

How to send series of commands to a command window process?

We have a few commands(batch files/executables) on our network path which we have to call to initialize our 'development environment' for that command window. It sets some environmental variables, adds stuff to the Path etc. (Then only whatever working commands we type will be recognized & I don't know what goes inside those initializing commands)
Now my problem is, I want to call a series of those 'working commands' using a C# program, and certainly, they will work only if the initial setup is done. How can I do that? Currently, I'm creating a batch file by scratch from the program like this for example:
file.Writeline("InitializationStep1.bat")
file.Writeline("InitializeStep2.exe")
file.Writeline("InitializeStep3.exe")
Then the actual commands
file.Writeline("Dowork -arguments -flags -blah -blah")
file.Writeline("DoMoreWork -arguments -flags -blah -blah")
Then finally close the file writer, and run this batch file.
Now if I directly execute this using Process.<strike>Run</strike>Start("cmd.exe","Dowork -arguments"); it won't run.
How can I achieve this in a cleaner way, so that I have to run the initialization commands only once? (I could run cmd.exe each time with all three initializers, but they take a lot of time so I want to do it only once)

As #Hakeem has pointed out, System.Diagnostic.Process does not have a static Run method. I think you are referring to the method Start.
Once you have completed building the batch file, then simply execute it using the following code,
Process p = new Process();
p.StartInfo.FileName = batchFilePath;
p.StartInfo.Arguments = #"-a arg1 -b arg2";
p.Start();
Note that the # symbol is required to be prefixed to the argument string so that escape sequence characters like \ are treated as literals.
Alternative code
Process.Start(batchFilePath, #"-a arg1 -b arg2");
or
ProcessStartInfo processStartInfo = new ProcessStartInfo();
processStartInfo.FileName = batchFilePath;
processStartInfo.Arguments = #"-a arg1 -b arg2";
Process.Start(processStartInfo);
More information
Process.Start method
Example of multi command batch file
dir /O
pause
dir
pause
Save this file as .bat and then execute using the Start method. In this case you can specify the argument with the command in the batch file itself (in the above example, the /O option is specified for the dir command.
I suppose you already have done the batch file creation part, now just append the arguments to the commands in the batch file.
Redirecting Input to a process
Since you want to send multiple commands to the same cmd process, you can redirect the standard input of the process to the take the input from your program rather than the keyboard.
Code is inspired from a similar question at: Execute multiple command lines with the same process using C#
private string ProcessRunner()
{
ProcessStartInfo processStartInfo = new ProcessStartInfo("cmd.exe");
processStartInfo.RedirectStandardInput = true;
processStartInfo.RedirectStandardOutput = true;
processStartInfo.UseShellExecute = false;
Process process = Process.Start(processStartInfo);
if (process != null)
{
process.StandardInput.WriteLine("dir");
process.StandardInput.WriteLine("mkdir testDir");
process.StandardInput.WriteLine("echo hello");
//process.StandardInput.WriteLine("yourCommand.exe arg1 arg2");
process.StandardInput.Close(); // line added to stop process from hanging on ReadToEnd()
string outputString = process.StandardOutput.ReadToEnd();
return outputString;
}
return string.Empty;
}
The method returns the output of the command execution. In a similar fashion, you could also redirect and read the StandardOuput stream of the process.

The Process.Run method that you mentioned, is that from the Process class in System.Diagnostics namespace? AFAIK, the Process type doesn't have either a static or instance method named Run. If you haven't already I'd try with the Start method on Process, either instance or static

c# redirect (pipe) process output to another process

I am trying to run a process in c# using the Process class.
Process p1 = new process();
p1.startinfo.filename = "xyz.exe";
p1.startinfo.arguments = //i am building it based on user's input.
p1.start();
So based on user input i am building the argument value. Now i have a case where i have to pipe the output of p1 to another process say grep. so i basically tried this
p1.startinfo.arguments = "-info |grep 1234" ;
what i intended is something like xyz.exe -info|grep 1234
but this doesn't seem to work in .net .. I can actually create another process variable and run "grep" as a separate process.. But i was wondering if there is any way to do as iam trying out above..

The much easier way would be to do just use cmd as your process.
Process test = new Process();
test.StartInfo.FileName = "cmd";
test.StartInfo.Arguments = #"/C ""echo testing | grep test""";
test.Start();
You can capture the output or whatever else you want like any normal process then. This was just a quick test I built, but it works outputting testing to the console so I would expect this would work for anything else you plan on doing with the piping. If you want the command to stay open then use /K instead of /C and the window will not close once the process finishes.