I have a WinForms application that runs multiple processes which run as background workers. I create a background worker for each new Process
BackgroundWorker background = new BackgroundWorker();
background.DoWork += new DoWorkEventHandler(bkWorker_DoWork);
background.RunWorkerCompleted += new RunWorkerCompletedEventHandler(bkWorker_Complete);
background.WorkerReportsProgress = false;
background.WorkerSupportsCancellation = true;
My Dowork code looks like this:
private void bkWorker_DoWork(object sender, DoWorkEventArgs e) {
WorkerArgument obj = (WorkerArgument) e.Argument;
BackgroundWorker worker = (BackgroundWorker) sender;
Process proc = new Process();
proc.StartInfo.FileName = "cmdprocess"; //"cmd.exe";
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.CreateNoWindow = true;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.RedirectStandardInput = true;
proc.StartInfo.RedirectStandardError = true;
proc.OutputDataReceived += new DataReceivedEventHandler(SortOutputHandler);
proc.ErrorDataReceived += new DataReceivedEventHandler(ErrorOutputHandler);
proc.Start();
process.Add(obj.Row, proc.Id);
proc.BeginOutputReadLine();
proc.BeginErrorReadLine();
proc.WaitForExit();
proc.Close();
}
As you may notice that I am using an event handler to sort the output given by the cmdProcess (SortOutputHandler) same as Error output.
I am showing this output in a text box
private void SortOutputHandler(object sendingProcess, DataReceivedEventArgs outLine) {
if (!String.IsNullOrEmpty(outLine.Data)) {
if (txtLog.InvokeRequired) {
this.Invoke((MethodInvoker) delegate {
string[] lines = txtLog.Lines;
if (lines.Length > 200) {
string[] newlines = lines.Skip(200) as string[];
txtLog.Lines = newlines;
}
txtLog.AppendText(Environment.NewLine + outLine.Data);
});
} else {
string[] lines = txtLog.Lines;
if (lines.Length > 200) {
string[] newlines = lines.Skip(200) as string[];
txtLog.Lines = newlines;
}
txtLog.AppendText(Environment.NewLine + outLine.Data);
}
}
}
The Error Handler is as follows:
private void ErrorOutputHandler(object sendingProcess, DataReceivedEventArgs outLine) {
if (!String.IsNullOrEmpty(outLine.Data)) {
if (txtLog.InvokeRequired) {
this.Invoke((MethodInvoker) delegate {
string[] lines = txtLog.Lines;
if (lines.Length > 200) {
string[] newlines = lines.Skip(200) as string[];
txtLog.Lines = newlines;
}
txtLog.AppendText(Environment.NewLine + outLine.Data);
});
} else {
string[] lines = txtLog.Lines;
if (lines.Length > 200) {
string[] newlines = lines.Skip(200) as string[];
txtLog.Lines = newlines;
}
txtLog.AppendText(Environment.NewLine + outLine.Data);
}
}
}
The problem is that when the number of processes increases the log starts running rapidly.
I have created a new Form frmLog with a multi-line textbox inside it, Can you tell me How I can use it to get Output from one process? Like there can be a "View Log" Button and when I click it only the log of that process show.
Also is it possible to write the output of all the processes to unique files separately?
First of all, consider please as an example following type to hold process output
interface IProcessOutput
{
IReadOnlyCollection<string> Cerr { get; }
IReadOnlyCollection<string> Cout { get; }
void OnError(string text);
void OnOutput(string text);
}
Then, starting a new process, instantiate a type that implements this interface and "bind" events to corresponding objects.
I encourage you use Queue (as IReadOnlyCollection) if you want to roll your logs (with 2000 or whatever limit) because Queue holds head and tail internally so it does not allocate/move memory if you keep capacity of some limit. It is fast enough for it.
Change GUI so that ListBox with log output can belong to any of running processes (kind of master/detail). So, once you select a process, you will see corresponding log lines.
Instead of textbox, please use ListBox. Use BeginUpdate() and EndUpdate() while loading strings into it. If your output is huge, use self-drawing in listbox so you do not need to store strings into its' Items collection rather access queue objects directly.
PS: Be ready to receive strings that form not a complete line (without new line char). The event with output is internally called when process flushes file handle (cerr, cout are also file handles). So if process flushes it in the middle of a string - you will get half of the string.
Related
I have built a winform interface for my python program, my python program is a real time voice assistant, what I need is the interface should response instantly when the python gives outputs. I need to display standard output to the interface instantly. the below program is what I made.
in this code, the interface is not responding properly. python program executes in background continuously and not responding to the voice. i need a program that execute my python program and display the standard output to the winform interface.
namespace #interface
{
public partial class Form1 : Form
{
public static string text;
public Form1()
{
InitializeComponent();
}
private void pictureBox1_Click(object sender, EventArgs e)
{
}
private async void start_button_Click(object sender, EventArgs e)
{
string line;
int counter=0;
msg.Text = "Hey, Tell me something!";
Task task = new Task(Execute);
task.Start();
}
public void Execute()
{
// full path of python interpreter
string python = #"C:/Users/Jayasooryan/AppData/Local/Programs/Python/Python36-32/python.exe";
// python app to call
string myPythonApp = #"C:/Users/Jayasooryan/AppData/Local/Programs/Python/Python36-32/Avira.py";
// Create new process start info
ProcessStartInfo myProcessStartInfo = new ProcessStartInfo(python);
// make sure we can read the output from stdout
myProcessStartInfo.UseShellExecute = false;
myProcessStartInfo.RedirectStandardOutput = true;
myProcessStartInfo.CreateNoWindow = true;
// start python app with 3 arguments
// 1st arguments is pointer to itself,
// 2nd and 3rd are actual arguments we want to send
myProcessStartInfo.Arguments = myPythonApp;
Process myProcess = new Process();
// assign start information to the process
myProcess.StartInfo = myProcessStartInfo;
// start the process
myProcess.Start();
// Read the standard output of the app we called.
// in order to avoid deadlock we will read output first
// and then wait for process terminate:
StreamReader myStreamReader = myProcess.StandardOutput;
string myString = myStreamReader.ReadLine();
text = myString;
//Console.WriteLine(myString);
/*if you need to read multiple lines, you might use:
string myString = myStreamReader.ReadToEnd() */
// wait exit signal from the app we called and then close it.
myProcess.WaitForExit();
myProcess.Close();
// write the output we got from python app
//Console.ReadLine();
}
private void textBox1_TextChanged(object sender, EventArgs e)
{
}
}
}
First: you should not try to read a single line from the python process, but rather use the OutputDataReceived event to be triggered when the process writes new data.
Second: since the output is buffered, you probably want to flush it after writing in your python process.
So here's a simple python script that keeps writing to the standard output (note how stdout.flush is called):
import random
import time
import sys
while True:
rand = random.randint(1, 10)
time.sleep(1)
print rand
sys.stdout.flush()
if rand in (9, 10):
break
And here's a simple Form that reads the output of that very script:
var f = new Form();
var t = new TextBox();
t.Dock = DockStyle.Fill;
t.Multiline = true;
f.Controls.Add(t);
f.Load += (s, e) => {
Process process = new Process();
process.StartInfo.FileName = "python";
process.StartInfo.Arguments = #"d:\script.py";
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardOutput = true;
process.OutputDataReceived += (s2, e2) => {
t.Text += e2.Data + Environment.NewLine;
};
process.Start();
process.BeginOutputReadLine();
};
f.ShowDialog();
I want to process a large ammount of data stored in a text file. Here is the code I use to make it work faster:
var result = File
.ReadLines(textBox1.Text)
.AsParallel()
.WithDegreeOfParallelism(100)
.Select(line => ProcessLine(line));
The method ProcessLine gets the line then processes it and add it to an ArrayList.
After all the processing is done I load the ArrayList into a Datagrid,
but sometimes it completes all the lines and sometimes it hangs, I don't know why.
Any suggestions ?
Update
Here is the Method ProcessLine
private string ProcessLine(string domain)
{
ProcessStartInfo cmdinfo = new ProcessStartInfo();
cmdinfo.FileName = "cmd.exe";
cmdinfo.Arguments = "/c nslookup";
cmdinfo.RedirectStandardInput = true;
cmdinfo.RedirectStandardOutput = true;
cmdinfo.CreateNoWindow = true;
cmdinfo.UseShellExecute = false;
cmdinfo.RedirectStandardError = false;
Process cmdd = new Process();
cmdd = Process.Start(cmdinfo);
string spf = "none";
createproc:
try
{
cmdd.StandardInput.WriteLine("set q=txt");
cmdd.StandardInput.Flush();
cmdd.StandardInput.WriteLine(domain);
cmdd.StandardInput.WriteLine("exit");
cmdd.StandardInput.WriteLine("exit");
StreamReader r = cmdd.StandardOutput;
//cmdd.WaitForExit();
cmdd.Close();
spf = "";
string rdl = string.Empty;
bool spffound = false;
while (rdl != null)
{
try
{
rdl = r.ReadLine();
if (rdl.Contains("v=spf"))
{
spffound = true;
spf = rdl.Trim();
this.Invoke(new MethodInvoker(delegate
{
textBox2.AppendText("domain found : " + domain + Environment.NewLine + "SPF = " + spf + Environment.NewLine);
textBox2.Update();
}));
break;
}
}
catch (Exception)
{
}
}
if (!spffound)
spf = "none";
nbrDoms++;
this.Invoke(new MethodInvoker(delegate
{
DomsElapsed.Text = nbrDoms + " Domains Elapsed";
DomsElapsed.Update();
}));
SPFRecord srx = new SPFRecord((string)spf.Clone(), (string)domain.Clone());
if (srx == null)
{
cmdd.Kill();
cmdinfo = new ProcessStartInfo();
cmdinfo.FileName = "cmd.exe";
cmdinfo.Arguments = "/c nslookup";
cmdinfo.RedirectStandardInput = true;
cmdinfo.RedirectStandardOutput = true;
cmdinfo.CreateNoWindow = true;
cmdinfo.UseShellExecute = false;
cmdinfo.RedirectStandardError = false;
cmdd = new Process();
cmdd.StartInfo = cmdinfo;
cmdd.Start();
goto createproc;
}
lock (pageManager)
{
pageManager.AddRecord(srx);
}
//this.Invoke(new MethodInvoker(delegate
//{
//}));
}
catch(Exception exc)
{
cmd.Kill();
cmdinfo = new ProcessStartInfo();
cmdinfo.FileName = "cmd.exe";
cmdinfo.Arguments = "/c nslookup";
cmdinfo.RedirectStandardInput = true;
cmdinfo.RedirectStandardOutput = true;
cmdinfo.CreateNoWindow = true;
cmdinfo.UseShellExecute = false;
cmdinfo.RedirectStandardError = false;
cmdd = new Process();
cmdd.StartInfo = cmdinfo;
cmdd.Start();
Thread.Sleep(10);
goto createproc;
}
return "";
}
Read the lines of text into a string, with something like file.readalllines(psudo code)
Basically each thread is locking the other, are you trying this for speed or because the file is too large to fit into memory?
Ok, a few things to mention:
Do not use goto statements - it's hard to understand what your method does. Just move the creation of the Process into a separate method and call that method instead of using goto
Processes do take time to load and quite a lot for what you want to do. To avoid this load penalty, instead of creating and calling a process try to replace that with a method which does the same. There's an example of nslookup without calling the process. Try adapting it to your needs
Remove the locks - if your application somehow gets to use 100 threads the lock is a waste of time. You'll have 99 threads waiting for the other single thread to push its data to the pageManager. As #Mrinal Kamboj pointed out, you can use a thread-safe collection. In this case, use a BlockingCollection<T> and add the results there. At the other end of the queue have the pageManager listening and consuming each item as it arrives.
The UI needs its separate cycles to refresh which also takes time. If pageManager.AddRecord() somehow has to refresh the UI then the other threads won't wait just for the add operation.
UI updates must be done in the thread that created the controls and that thread can't update the UI if it's waiting for another thread.
The overall algorithm should look like this:
public class Engine
{
private readonly BlockingCollection<string> _messagePipeline = new BlockingCollection<string>();
public BlockingCollection<string> MessagePipeline
{
get { return _messagePipeline; }
}
public void Process(string file)
{
File.ReadLines(file)
.AsParallel()
.ForAll(line =>
{
var nsLookupResult = NsLookupMethod(line);
if(nsLookupResult.HasInfoYouNeed)
_messagePipeline.Add(nsLookupResult.DisplayInfo);
});
}
}
public class MainForm : Form
{
private readonly Engine _engine; // ...
private void OnStartButtonClick(object sender, EventArgs e)
{
var cts = new CancellationTokenSource();
_engine.Process(textbox1.Text);
Task.Factory.StartNew(()=>
{
foreach(var message in _engine.MessagePipeline.GetConsumingEnumerable())
{
// show the message
Application.DoEvents(); // allow the app to process other events not just pushing messages.
}
}, cts.Token,
TaskCreationOptions.PreferFairness,
// Specify that you want UI updates to be done on the UI thread
// and not on any other thread
TaskScheduler.FromCurrentSynchronizationContext());
}
}
And that should do it. I do have a (more or less academic) example of this kind of logic in action. The UI update logic is in the MainForm of the application and the processing logic is in an Engine class; have a look there.
I'm writing an application that creates a batch file and then run :
I know I can create a Batch file and run it
no problem with that .
What I want to do is :
once I have created the string that makes the file , Is there is any way to execute the string as a Batch file ?
something like
string BatchFile = "echo \"bla bla\" \n iperf -c 123 ... ... .. "
Diagnostics.Process.Start(BatchFile);
You can run CMD.EXE with /c as an executable and have the rest as arguments :
Process.Start("cmd.exe", "/c echo \"bla bla\" \n iperf -c 123 ... ... .. ");
for me, I am using this code:
Process process;
private void button1_Click(object sender, EventArgs e)
{
process = new Process();
process.StartInfo.UseShellExecute = false;
process.StartInfo.FileName = "cmd";
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardInput = true;
process.Start();
BackgroundWorker worker = new BackgroundWorker();
worker.DoWork += new DoWorkEventHandler(worker_DoWork);
worker.RunWorkerAsync();
process.StandardInput.WriteLine("cd d:/tempo" );
process.StandardInput.WriteLine("dir");
}
void worker_DoWork(object sender, DoWorkEventArgs e)
{
string line;
while (!process.StandardOutput.EndOfStream)
{
line = process.StandardOutput.ReadLine();
if (!string.IsNullOrEmpty(line))
{
SetText(line);
}
}
}
delegate void SetTextCallback(string text);
private void SetText(string text)
{
if (this.textBox1.InvokeRequired)
{
SetTextCallback d = new SetTextCallback(SetText);
this.Invoke(d, new object[] { text });
}
else
{
this.textBox1.Text += text + Environment.NewLine;
}
}
private void FrmMain_FormClosing(object sender, FormClosingEventArgs e)
{
process.StandardInput.WriteLine("exit");
process.Close();
}
You may create your Batch "file" as a long string with lines terminated in \n, exactly as you shown in your example, and then execute that string (I called it a "NotBatch-text") executing cmd.exe and redirecting such string into its Stdin standard handle. This way, your "NotBatch-text" may use a large number of Batch features, like expansion of %variables%, IF and FOR commands nested at any level, and many more. You may also use delayed !variable! expansion if you execute cmd.exe with /V:ON switch. Really, the only things that don't works in the NotBatch-text are: parameters and SHIFT command, and GOTO/CALL :label commands; further details at this post.
If you want to execute a more advanced "NotBatch-text" string, you may even simulate GOTO and CALL :label commands with the aid of a third party program, as described at this post.
For all too long, I have been trying to run an external .bat file (calls an R script for some statistical processing), and have the console redirect to the U.I.
I think I am close, but just as I have gotten it to work I have run into a sizable problem! That is: it only bloody works once the main thread has ended (via: return;), and not during Thread.Sleep, or .WaitOne() or etc.
Here is my code in the main thread.
string batLoc = ALLRG___.RSCRBIN_LOC + "current.bat";
BackgroundWorker watchboxdWorker1 = new BackgroundWorker();
watchboxdWorker1.DoWork += frmC.WatchboxWorker1_WatchExt;
frmC.wbResetEvent = new AutoResetEvent(false);
watchboxdWorker1.RunWorkerAsync(batLoc);
//Thread.Sleep(1000*20);
//frmC.wbResetEvent.WaitOne();
return;
Note the commented out Sleep and/or WaitOne() instructions. If I try and use these the BackgroundWorker DOES execute, but the 'events' which update the U.I do not.
The code in my form (frmC above) is as follows,
public void WatchboxWorker1_WatchExt(object sender, DoWorkEventArgs e)
{
string exeLoc = (string) e.Argument;
string arg1 = exeLoc;
string arg2 = "";
ProcessStartInfo pStartInfo = new ProcessStartInfo();
pStartInfo.FileName = exeLoc;
pStartInfo.Arguments = string.Format("\"{0}\" \"{1}\"", arg1, arg2);
pStartInfo.WorkingDirectory = Path.GetDirectoryName(exeLoc);
pStartInfo.CreateNoWindow = true;
pStartInfo.UseShellExecute = false;
pStartInfo.RedirectStandardInput = true;
pStartInfo.RedirectStandardOutput = true;
pStartInfo.RedirectStandardError = true;
Process process1 = new Process();
process1.EnableRaisingEvents = true;
process1.OutputDataReceived += new DataReceivedEventHandler(wbOutputHandler);
process1.ErrorDataReceived += new DataReceivedEventHandler(wbErrorHandler);
process1.StartInfo = pStartInfo;
process1.SynchronizingObject = rtbWatchbox;
process1.Start();
process1.BeginOutputReadLine();
process1.BeginErrorReadLine();
process1.StandardInput.Close();
process1.WaitForExit();
wbResetEvent.Set();
}
public void wbOutputHandler(Object source, DataReceivedEventArgs outLine)
{
int x = 0;
if (!String.IsNullOrEmpty(outLine.Data))
{
rtbWatchbox.AppendText(outLine.Data);
}
}
public void wbErrorHandler(Object source, DataReceivedEventArgs outLine)
{
int x = 0;
if (!String.IsNullOrEmpty(outLine.Data))
{
rtbWatchbox.AppendText(outLine.Data);
}
}
My problem is --
The wbOutputHandler and wbErrorHandler get fired as the console updates nicely - but only when the main thread has exited (using the return;).... if I use the Thread.Sleep or .WaitOne() in the main thread to pass control to the BackgroundWorker (WatchboxWorker1_WatchExt), then the code runs successfully, but the wbOutputHandler and wbErrorHandler methods do not get triggered at all.
In fact, if I do the Thread.Sleep(10*1000), then the external program starts running as planned, 10 seconds pass, then when the main UI thread exits I get a whole big enormous update all at once.
I don't want to have my main thread closed, I want to keep doing stuff there after the Worker is finished!
[ of course happy for alternate methods that are a better approach ]
"Help me Stack Overflow, you are my only hope!"
The answer was to put a backgroundWorker within another backgroundWorker, which is created for the UI Thread. I thought quite complicated given the reletivly simple requirement of printing a console output to the UI!
I now call my functions from the UI as follows -
private void btInsertBCModls_Click(object sender, EventArgs e)
{
BackgroundWorker bw = new BackgroundWorker();
bw.DoWork += RC2___Scratchpad4.BC_RunExistingBCModel;
bw.RunWorkerAsync(this);
}
Next I use the delegate & Invoke method on any richTextBox I need to update from another thread -
delegate void UpdateWriteboxCallback(String str);
public void wbWriteBox(string WriteString)
{
if (!String.IsNullOrEmpty(WriteString))
{
if (rtbWatchbox.InvokeRequired)
{
UpdateWriteboxCallback at = new UpdateWriteboxCallback(wbWriteBox);
this.Invoke(at, new object[] { WriteString });
}
else
{
// append richtextbox as required
}
}
}
Then from within my function I use another BackgroundWorker to run the console stuff -
public static void BC_RunExistingBCModel(object sender, DoWorkEventArgs e)
{
RC2___RhinegoldCoreForm frmC = e.Argument as RC2___RhinegoldCoreForm;
BackgroundWorker watchboxWorker = new BackgroundWorker();
watchboxWorker.DoWork += frmC.WatchboxWorker_RunProc;
watchboxWorker.RunWorkerAsync(batLoc);
while (watchboxWorker.IsBusy)
Thread.Sleep(50);
frmC.UpdateRGCoreStatusBox4("Executed script " + m + "... ");
}
Which in turn, in the DoWork function, calls the wbWriteBox function above.
public void WatchboxWorker_RunProc(object sender, DoWorkEventArgs e)
{
string exeLoc = (string) e.Argument;
string arg1 = exeLoc;
string arg2 = "";
ProcessStartInfo pStartInfo = new ProcessStartInfo();
pStartInfo.FileName = exeLoc;
pStartInfo.Arguments = string.Format("\"{0}\" \"{1}\"", arg1, arg2);
pStartInfo.WorkingDirectory = Path.GetDirectoryName(exeLoc);
pStartInfo.CreateNoWindow = true;
pStartInfo.UseShellExecute = false;
pStartInfo.RedirectStandardInput = true;
pStartInfo.RedirectStandardOutput = true;
pStartInfo.RedirectStandardError = true;
Process process1 = new Process();
process1.EnableRaisingEvents = true;
process1.OutputDataReceived += (s, e1) => this.wbWriteBox(e1.Data);
process1.ErrorDataReceived += (s, e1) => this.wbWriteBox(e1.Data);
process1.StartInfo = pStartInfo;
process1.SynchronizingObject = rtbWatchbox;
process1.Start();
process1.BeginOutputReadLine();
process1.BeginErrorReadLine();
process1.StandardInput.Close();
process1.WaitForExit();
//wbResetEvent.Set();
}
Phew! A tricky solution to an easily defined problem. If someone has a better way, let me know.
And thanks to Carsten for all the help - magnificent.
I need to run PLink as a process as part of WinForm application
this is my code
public void RunProcess(string FileName, string Arguments, bool EventWhenExit , bool IsWaitBeforeStart = true )
{
process = new Process();
process.OutputDataReceived += new DataReceivedEventHandler(OnDataReceivedEvent);//**
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;
process.StartInfo.RedirectStandardInput = true;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.UseShellExecute = false;
process.StartInfo.FileName = FileName; // Gets or sets the application or document to start.
process.StartInfo.Arguments = Arguments;//Gets or sets the set of command-line arguments to use when starting the application
if (IsWaitBeforeStart) Thread.Sleep(5000);
if (EventWhenExit)
{
process.EnableRaisingEvents = true;
process.Exited += new EventHandler(myprocess_Exited);
}
process.Start();
process.BeginOutputReadLine();
PID = process.Id;
ProcessTimeOut.Enabled = true;
ProcessInputStream = process.StandardInput;
ProcessTimeOut.Enabled = false;
}
private void OnDataReceivedEvent(object sender, DataReceivedEventArgs e)
{
//prints to screen using control.invoke
//add data to a string list
}
My setup consist of a telnet server that I need to run few command on it and parse the result
if I run the application from cmd it prints the result under one sec (it is about 50 rows)
but if I run it using my code it takes up to almost 7 sec !
From my understanding process.start() and running via cmd should be the same
so the problem should be somewhere in my code or logic
what can be the problem ?
Ok so with help from Vajura comment I've made a simple(vary) buffer to implement simple consumer/producer pattern
inside RunProcess :
public void RunProcess(string FileName, string Arguments, bool EventWhenExit , bool IsWaitBeforeStart = true )
{
//... same code as before
PollingService();
}
second changing the event DataReceivedEventHandler
to store data to buffer (and stop invoking the print to UI)
the code is something like ProcessLog.Add(e.Data);
now for the second Thread to run over the buffer :
private void PollingService()
{
var T = new Thread (()=>
{
while (true)
{
if (ProcessLogIndex < ProcessLog.Count)
{
lock (this)
{
var tempList = ProcessLog.GetRange(ProcessLogIndex, ProcessLog.Count - ProcessLogIndex);
ProcessLogIndex = ProcessLog.Count;
foreach (var cell in tempList)
{
string ToSend = !string.IsNullOrEmpty(cell) ? (cell.Contains('$') ? cell.Substring(cell.LastIndexOf('$')) : cell) : "";
onDataOutputFromProcess(this, ToSend, Proc.ToString());
}
}
}
Thread.Sleep(1000);
}
});
T.IsBackground = true;
T.Start();
}