We Keep Coding

How to write to output pane from a background task

I am writing a C# extension for Visual Studio. I only have a basic understanding of the async/await mechanism and tasks usage. I am thus a bit puzzled when it comes to running some program on the background and echo its standard output to the Visual Studio output pane.
Indeed, I have written a small wrapper around the IVsOutputWindowPane object and the IDE recommends to add ThreadHelper.ThrowIfNotOnUIThread() or ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync() calls when writing to the Visual Studio output pane. I can understand that since it is a part of the UI.
I have tried the following:
private void MenuItemCallback(object sender, EventArgs evt)
{
var dte = (DTE2)ServiceProvider.GetService(typeof(EnvDTE.DTE));
var outputWindow = (IVsOutputWindow)ServiceProvider.GetService(typeof(SVsOutputWindow));
var outputWriter = new OutputWindowMgr(dte, outputWindow);
outputWriter.PrintLine("START");
// Run extern program in background
var analysis = System.Threading.Tasks.Task.Run(() => {
try
{
using (Process process = new Process())
{
process.StartInfo.FileName = bla;
process.StartInfo.Arguments = bla;
process.StartInfo.WorkingDirectory = bla;
process.StartInfo.RedirectStandardError = true;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.UseShellExecute = false;
process.Start();
while (!process.StandardOutput.EndOfStream)
{
outputWriter.PrintLine(process.StandardOutput.ReadLine());
}
}
}
catch (Exception e)
{
outputWriter.PrintLine(e.Message);
}
});
// It looks like waiting for the task to finish while still allowing
// the UI thread to process events will unlock UI thread
while (!analysis.Wait(10))
{
Application.DoEvents();
}
outputWriter.PrintLine("STOP");
}
If I keep the recommended ThreadHelper.ThrowIfNotOnUIThread() or ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync() calls in the outputWriter.PrintLine method, it will crash.
If I ignore them (comment them out), it will work however the order is broken: the "STOP" message will appear before the outside program has ever the chance to begin. Moreover, I am not confident it will not crash also in other situations.
What are the best practices / patterns for writing a background program's standard output (which is obviously NOT on the UI thread) to the Visual Studio output pane (which obviously IS on the UI thread)?
Thanks!

Your version of the MenuItemCallback looks like the older, 2015-style. Mine, using Visual Studio 2017, does all this on async TAB functions. It adds a little complexity, but I believe this solution would work for you as well.
The key for me was using the OutputDataReceived callback on the process, and in it, creating a task that requests the UI thread to add text to the output window. I imagine there's smarter ways of doing a bunch of these things, but this works, gives progressive output, and doesn't block the UI at any point.
private async Task ClearOutputWindowAsync()
{
await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(package.DisposalToken);
var dte = await package.GetServiceAsync(typeof(EnvDTE.DTE));
if (dte == null)
return;
EnvDTE80.DTE2 dte2 = dte as EnvDTE80.DTE2;
EnvDTE.OutputWindowPanes panes = dte2.ToolWindows.OutputWindow.OutputWindowPanes;
try
{
mOutputPane = panes.Item("My VS Extension");
}
catch (ArgumentException)
{
mOutputPane = panes.Add("My VS Extension");
}
mOutputPane.Clear();
mOutputPane.Activate();
}
private async Task AddOutputStringAsync(string msg)
{
await ThreadHelper.JoinableTaskFactory.SwitchToMainThreadAsync(package.DisposalToken);
mOutputPane.OutputString(msg);
mOutputPane.OutputString("\n");
}
private void StandardOutReceived(object proc, System.Diagnostics.DataReceivedEventArgs ev)
{
package.JoinableTaskFactory.RunAsync(() => AddOutputStringAsync(ev.Data));
}
private async Task<System.Diagnostics.Process> GetBuildProcessAsync()
{
// this is async only because I needed to access the Solution in
// establishing my process. It can only be read on the main thread
// which this isn't by default
Process process = new Process();
process.StartInfo.FileName = bla;
process.StartInfo.Arguments = bla;
process.StartInfo.WorkingDirectory = bla;
process.StartInfo.RedirectStandardError = true;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.UseShellExecute = false;
return process;
}
private async void Execute(object sender, EventArgs e)
{
if (mWorking)
return;
System.Diagnostics.Process buildProcess = await package.JoinableTaskFactory.RunAsync<System.Diagnostics.Process>(() => GetBuildProcessAsync());
if (buildProcess == null)
{
return;
}
mWorking = true;
await package.JoinableTaskFactory.RunAsync(() => ClearOutputWindowAsync());
string errorOut = "";
await System.Threading.Tasks.Task.Run(() => {
try
{
buildProcess.OutputDataReceived += StandardOutReceived;
buildProcess.Start();
buildProcess.BeginOutputReadLine();
errorOut = buildProcess.StandardError.ReadToEnd();
buildProcess.WaitForExit();
}
catch (Exception ex)
{
errorOut = ex.Message;
}
});
await package.JoinableTaskFactory.RunAsync(() => AddOutputStringAsync(errorOut));
mWorking = false;
}

C# FFMPEG Process and Multiple files

I am working on a C# Form tool that will help me convert all of my phone and DSLR video to HEVC, currently, i have a program that uploads the photos and videos to different directories in my home server each time i connect to the WiFi. Once a month or so, i manually convert all the videos, but thought I would automate the process.. I have the Form working perfectly for processing 1 file. but get into trouble when processing a Directory (with possible sub-directories) all at once..
Sorry, this is long, just want to be thorough. here is the button calls
private void processFile_Click(object sender, EventArgs e)
{
OpenFileDialog file = new OpenFileDialog();
file.InitialDirectory = baseMediaDirectory;
if (file.ShowDialog() == DialogResult.OK)
{
ProcessSinlgeFile(file.FileName);
}
}
(above)for one file and (below) for a directory
private void processDirectory_Click(object sender, EventArgs e)
{
FolderBrowserDialog file = new FolderBrowserDialog();
file.SelectedPath = baseMediaDirectory;
if(file.ShowDialog() == DialogResult.OK)
{
ProcessDirectoryOfFiles(file.SelectedPath);
}
}
private void ProcessDirectoryOfFiles(string selectedPath)
{
List<string> listOfFiles = GetAllFiles(selectedPath);
foreach (string s in listOfFiles)
{
ProcessSinlgeFile(s);
}
}
both ultimately call this method, to do some checks and setup
private void ProcessSinlgeFile(string fileName)
{
if (IsAcceptableMediaFile(fileName))
{
outputWindow.AppendText("File to Process: " + fileName);
processMediaFile =
new MediaFileWrapper(this.outputWindow, new MediaFile(fileName), new NReco.VideoInfo.FFProbe());
if (processMediaFile.OkToProcess)
{
int initialCRFValue = 15;
//ultrafast superfast veryfast faster fast medium slow slower veryslow placebo
string intialSpeed = "veryfast";
try {
ConvertToMPEG(processMediaFile.getFFMPEGCommand(initialCRFValue, intialSpeed), processMediaFile);
}
catch
{
// at somepoint, we'll catch a bad file size (or compression)
// then change the CRF value and/or compression speed
}
}
}
}
ultimately I get to this Method and run into trouble.
private async void ConvertToMPEG(string arguments, MediaFileWrapper processMediaFile)
{
startTime = DateTime.Now;
watch = new Stopwatch();
watch.Start();
progressBar1.Minimum = 0;
progressBar1.Maximum = processMediaFile.GetTotalMilliseconds();
// Start the child process.
p = new Process();
//Setup filename and arguments
outputWindow.AppendText("ffmpeg " + arguments);
p.StartInfo.Arguments = arguments;
p.StartInfo.FileName = "ffmpeg.exe";
p.StartInfo.UseShellExecute = false;
// Redirect the output stream of the child process.
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.RedirectStandardError = true;
p.StartInfo.RedirectStandardInput = true;
// capture the date for stdout and std error
// note FFMPEG uses Stderr exclusively
p.ErrorDataReceived += new DataReceivedEventHandler(ErrorDataReceived);
p.OutputDataReceived += new DataReceivedEventHandler(OutputDataReceived);
// Hide Console Window
p.StartInfo.CreateNoWindow = true;
p.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
p.Start();
p.BeginErrorReadLine();
p.BeginOutputReadLine();
await p.WaitForExitAsync();
}
and WaitForExitAsync is in another class because in can not be in here with a Form
public static Task WaitForExitAsync(this Process process,
CancellationToken cancellationToken = default(CancellationToken))
{
var tcs = new TaskCompletionSource<object>();
process.EnableRaisingEvents = true;
process.Exited += (sender, args) => tcs.TrySetResult(null);
if (cancellationToken != default(CancellationToken))
cancellationToken.Register(tcs.SetCanceled);
return tcs.Task;
}
however, single files work fine, when I call a directory through, it continuously starts processes for each file, trying to run them all at the same time. You can see I tried implementing this
process.WaitForExit() asynchronously
with no luck.

process output is much slower than with cmd

I need to run PLink as a process as part of WinForm application
this is my code
public void RunProcess(string FileName, string Arguments, bool EventWhenExit , bool IsWaitBeforeStart = true )
{
process = new Process();
process.OutputDataReceived += new DataReceivedEventHandler(OnDataReceivedEvent);//**
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;
process.StartInfo.RedirectStandardInput = true;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.UseShellExecute = false;
process.StartInfo.FileName = FileName; // Gets or sets the application or document to start.
process.StartInfo.Arguments = Arguments;//Gets or sets the set of command-line arguments to use when starting the application
if (IsWaitBeforeStart) Thread.Sleep(5000);
if (EventWhenExit)
{
process.EnableRaisingEvents = true;
process.Exited += new EventHandler(myprocess_Exited);
}
process.Start();
process.BeginOutputReadLine();
PID = process.Id;
ProcessTimeOut.Enabled = true;
ProcessInputStream = process.StandardInput;
ProcessTimeOut.Enabled = false;
}
private void OnDataReceivedEvent(object sender, DataReceivedEventArgs e)
{
//prints to screen using control.invoke
//add data to a string list
}
My setup consist of a telnet server that I need to run few command on it and parse the result
if I run the application from cmd it prints the result under one sec (it is about 50 rows)
but if I run it using my code it takes up to almost 7 sec !
From my understanding process.start() and running via cmd should be the same
so the problem should be somewhere in my code or logic
what can be the problem ?

Ok so with help from Vajura comment I've made a simple(vary) buffer to implement simple consumer/producer pattern
inside RunProcess :
public void RunProcess(string FileName, string Arguments, bool EventWhenExit , bool IsWaitBeforeStart = true )
{
//... same code as before
PollingService();
}
second changing the event DataReceivedEventHandler
to store data to buffer (and stop invoking the print to UI)
the code is something like ProcessLog.Add(e.Data);
now for the second Thread to run over the buffer :
private void PollingService()
{
var T = new Thread (()=>
{
while (true)
{
if (ProcessLogIndex < ProcessLog.Count)
{
lock (this)
{
var tempList = ProcessLog.GetRange(ProcessLogIndex, ProcessLog.Count - ProcessLogIndex);
ProcessLogIndex = ProcessLog.Count;
foreach (var cell in tempList)
{
string ToSend = !string.IsNullOrEmpty(cell) ? (cell.Contains('$') ? cell.Substring(cell.LastIndexOf('$')) : cell) : "";
onDataOutputFromProcess(this, ToSend, Proc.ToString());
}
}
}
Thread.Sleep(1000);
}
});
T.IsBackground = true;
T.Start();
}

open new Thread inside for statement bu wait until my Thread will finish before open new one

i try to my operation inside for statement in different Thread but i want to wait until my Tread will finish before open new Thread:
public class Play
{
private string _filePath;
private int _deviceNumber;
public Play(string filePath, int deviceNumber)
{
_filePath = filePath;
_deviceNumber = deviceNumber;
}
public void start()
{
Thread thread = new Thread(send);
thread.IsBackground = true;
thread.Start();
}
private void send()
{
ProcessStartInfo processStartInfo = new ProcessStartInfo(#"D:\SendQueue\SendQueue\bin\Debug\Send.exe");
processStartInfo.Arguments = string.Format("{0} {2}{1}{2}", (_deviceNumber).ToString(), _filePath, "\"");
processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
processStartInfo.RedirectStandardOutput = true;
processStartInfo.RedirectStandardError = true;
processStartInfo.CreateNoWindow = true;
processStartInfo.UseShellExecute = false;
processStartInfo.ErrorDialog = false;
using (Process process = Process.Start(processStartInfo))
{
process.WaitForExit();
}
}
}
from start button click i am play all the file within my Listbox:
for (int i = 0; i < ListBox.Items.Count && shouldContinue; i++)
{
PlayFile play = new PlayFile ((string)ListBox.Items[i], add);
playCapture.start();
}

Task taskA = Task.Factory.StartNew(() => send());
taskA.Wait();
But this will block the caller thread of the task (The work will be done in a second thread but actually, the caller thread will be blocked during the execution) you will not benefit from doing the work in another thread!
This link will help you
If you need to do all of the work in another thread, and keep the main stream going without block, you can put your for statement in a thread, this will work perfectly.

open differents threads using BackgroundWorker

i build an application how take Pcap file (wireshark file) and play the packets, the play operation is with exe file who get file path and interface int.
private void btnStart_Click(object sender, EventArgs e)
{
shouldContinue = true;
btnStart.Enabled = false;
btnStop.Enabled = true;
groupBoxAdapter.Enabled = false;
groupBoxRootDirectory.Enabled = false;
string filePath = string.Empty;
ThreadPool.QueueUserWorkItem(delegate
{
for (int i = 0; i < lvFiles.Items.Count && shouldContinue; i++)
{
this.Invoke((MethodInvoker)delegate { filePath = lvFiles.Items[i].Tag.ToString(); });
pcapFile = new PcapFile();
pcapFile.sendQueue(filePath, adapter);
}
this.Invoke((MethodInvoker)delegate
{
btnStart.Enabled = true;
btnStop.Enabled = false;
groupBoxAdapter.Enabled = true;
groupBoxRootDirectory.Enabled = true;
});
});
}
the sendQueue code:
public void sendQueue(string filePath, int deviceNumber)
{
ProcessStartInfo processStartInfo = new ProcessStartInfo(#"D:\Downloads\SendQueue\sendQueue.exe");
processStartInfo.Arguments = string.Format("{0} {2}{1}{2}", (deviceNumber).ToString(), filePath, "\"");
processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
processStartInfo.RedirectStandardOutput = true;
processStartInfo.RedirectStandardError = true;
processStartInfo.CreateNoWindow = true;
processStartInfo.UseShellExecute = false;
processStartInfo.ErrorDialog = false;
using (Process process = Process.Start(processStartInfo))
{
process.WaitForExit();
}
}

Doen't look that you need Background worker.
List<string> tags = new List<string>();
foreach (object item in lvFiles.Items)
{
tags.Add(item.tag.ToString());
}
ThreadPool.QueueUserWorkItem(delegate
{
for (int i = 0; i < tags.Count && shouldContinue; i++)
{
sendQueue(tags[i], adapter);
}
//...
}

Your UI thread is most likely blocked because the pcapFile.sendQueue is synchronous. This means even though your async loop queues the play files the UI thread is busy 99.99% of the time playing the file's content. This may or may not be the case since you haven't posted PcapFile's source.
The task to make your UI responsive is a bit more involving, you need to restructure PcapFile to load up a frame (audio? video?) at a time and let the UI thread run the rest of the time or even to work completely in the background.
The form design should also rely on events from PcapFile instead of trying to run it in a BackgroundWorker