We Keep Coding

Why sometimes when watching a file/directory for changes it's not changing the flag to true?

In the constructor i'm calling WatchDirectory method:
private void WatchDirectory()
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = userVideosDirectory;
watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.Size;
watcher.Filter = "*.mp4";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
}
Then the event OnChanged:
private void OnChanged(object source, FileSystemEventArgs e)
{
try
{
var info = new FileInfo(e.FullPath);
fileforupload = info.FullName;
if (e.ChangeType == WatcherChangeTypes.Changed)
{
var theSize = info.Length;
label2.BeginInvoke((Action)(() =>
{
label2.Text = theSize.ToString();
}));
}
dirchanged = true;
}
catch (Exception ee)
{
string err = ee.ToString();
}
}
Then i'm using a while loop to check when dirchange flag is true:
WatchDirectory();
while (dirchanged == false)
{
if (dirchanged == true)
{
Youtube_Uploader youtubeupload = new
Youtube_Uploader(fileforupload);
break;
}
}
The problem is that sometimes it's never changes the dirchanged to true on the OnChanged event. Not sure why. It seems to fire the OnChanged event but sometimes it doesn't execute the dirchanged = true;
Therefore inside the while loop dirchanged flag remains false all the time.
I added now a new method i called it IsFileLocked:
protected virtual bool IsFileLocked(FileInfo file)
{
FileStream stream = null;
try
{
stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
}
catch (IOException)
{
return true;
}
finally
{
if (stream != null)
stream.Close();
}
return false;
}
And i use this in the event OnChanged:
private void OnChanged(object source, FileSystemEventArgs e)
{
try
{
var info = new FileInfo(e.FullPath);
fileforupload = info.FullName;
IsFileLocked(info);
if (e.ChangeType == WatcherChangeTypes.Changed)
{
var theSize = info.Length;
label2.BeginInvoke((Action)(() =>
{
label2.Text = theSize.ToString();
}));
}
dirchanged = true;
}
catch (Exception ee)
{
string err = ee.ToString();
}
}
And in the method IsFileLocked i'm getting exception:
The process cannot access the file 'C:\Users\bout0_000\Videos\My Great Game - My Great Capture - 2015-08-10 14-22-52.mp4' because it is being used by another process.
I'm using external program that create the file and since the program still working on creating the file the watcher can't get to it.
So i have a confilct here from one side i want to know to watch when the file is ready finished created but on the other side i can't know since the external program still working on it.
So how can i find out when the external program finished working on the file and the file is ready ?
This is the whole part of the code of the while:
if (request.QueryString[0] == "stop")
{
dirchanged = false;
StartRecrod();
result = "Recording stopped and preparing the file to be shared on youtube";
WatchDirectory();
while (dirchanged == false)
{
if (dirchanged == true)
{
string ttttt = "ok";
break;
}
}
}
I added a string ttttt just for testing.
Sometimes it's getting to the string ttttt when using a break point and sometimes not.
In my program when i touch my android screen it send command to the pc web server and it's getting here but someting is wrong with the while loop and the flag dirchanged sometimes it does enter the while and the IF and does the string ttttt and sometimes it dosen't.
This is what i did now with the await:
TaskCompletionSource<bool> sy;
public async void SendResponse(HttpListenerRequest request)
{
string result = "";
string key = request.QueryString.GetKey(0);
if (key == "cmd")
{
if (request.QueryString[0] == "nothing")
{
return "Connection Success";
}
if (request.QueryString[0] == "start")
{
StartRecrod();
result = "Recording started";
}
if (request.QueryString[0] == "stop")
{
dirchanged = false;
StartRecrod();
result = "Recording stopped and preparing the file to be shared on youtube";
sy = new TaskCompletionSource<bool>();
WatchDirectory();
await sy.Task;
Youtube_Uploader youtubeupload = new Youtube_Uploader(fileforupload);
}
}
else
{
result = "Nothing have been done";
}
if (Youtube_Uploader.fileuploadedsuccess != null && Youtube_Uploader.fileuploadedsuccess != "")
{
result = Youtube_Uploader.fileuploadedsuccess;
}
return result;
}
But some problems.
First i'm getting errors over all the returns.
Error 2 Since 'Automatic_Record.Form1.SendResponse(System.Net.HttpListenerRequest)' returns void, a return keyword must not be followed by an object expression
And error when init my web server:
WebServer ws = new WebServer(SendResponse, "http://+:8098/");
On the SendResponse i'm getting:
Error 1 'void Automatic_Record.Form1.SendResponse(System.Net.HttpListenerRequest)' has the wrong return type
This errors happen now when changed the method to async.
This is my WebServer method that i get error when init it since it should get something else then async:
public WebServer(Func<HttpListenerRequest, string> method, params string[] prefixes)
: this(prefixes, method) { }
public void Run()
{
ThreadPool.QueueUserWorkItem((o) =>
{
Console.WriteLine("Webserver running...");
try
{
while (_listener.IsListening)
{
ThreadPool.QueueUserWorkItem((c) =>
{
var ctx = c as HttpListenerContext;
try
{
string rstr = _responderMethod(ctx.Request);
System.Diagnostics.Trace.Write(ctx.Request.QueryString);
//ctx.Request.QueryString
byte[] buf = Encoding.UTF8.GetBytes(rstr);
ctx.Response.ContentLength64 = buf.Length;
ctx.Response.OutputStream.Write(buf, 0, buf.Length);
System.Data.SqlClient.SqlConnectionStringBuilder builder = new System.Data.SqlClient.SqlConnectionStringBuilder();
}
catch { } // suppress any exceptions
finally
{
// always close the stream
ctx.Response.OutputStream.Close();
}
}, _listener.GetContext());
}
}
catch { } // suppress any exceptions
});
}

This code is horribly broken. Yes, dirchanged is always false inside the while loop, because if it becomes true you won't be in the while loop any longer.
In addition, your code blocks events from occurring, which may block the file watcher event itself, and also is not optimization safe. Use proper synchronization, here's an example:
TaskCompletionSource<bool> sy;
private void OnChanged(object source, FileSystemEventArgs e)
{
sy.SetResult(true);
}
and wait with
sy = new TaskCompletionSource<bool>();
WatchDirectory();
await sy.Task; // or sy.Task.Wait()
(You'll need to use the async keyword on the method containing that code).
This fixes all the problems you had before -- it doesn't burn CPU cycles, it continues processing Windows messages, and it won't break if the compiler chooses to cache variables in registers.

dirchanged could be getting set to true just after evaluating the inner if block. Then, next loop it breaks out without ever running your uploader.

So you have two main questions?
1.) Why is dirchanged not being set to true?
and the apparent cause...
2.) How do you use FileSystemWatcher to only act on a file that's available for edit?
FileSystemWatcher is known for being a little touchy, and I agree with your diagnosis that file access is probably the culprit. An unpredictable file access error is exactly what I would expected from a FileSystemWatcher trying to do something with a file that was just modified. Can you edit the code that's creating the file? If so, one method I've used with FileSystemWatcher is to have it only watch for file creation of a fictitious file type such as ".fsw". The program creating the file will then rename it to ".fsw" whenever it is done editing it, that way the FileSystemWatcher only gets called when it has a file available to act upon, and it can then rename the file to it's actual type. Also, if you can edit the creation code, make sure that you are doing everything you can to release the file from there. I've seen this behavior before because I forgot to close a TextWriter.
Also, I would move the line
dirchanged = true;
Outside of the try statement. Why have it in there since it definitely wont throw an error? Also, your catch statement isn't really doing error handling. Any error in your try statement and you get ejected before reaching the dirchanged = true line without being alerted that this is what happened. Have you tested your delegate code on its own? Is there a need to have the if statement for type = changed right there? If you're troubleshooting, I would consider limiting your try statement content or moving it to after the while loop as much as possible.
Also, wouldn't this be a lot more simple for your while statement?
while (dirchanged == false){}
Youtube_Uploader youtubeupload = new Youtube_Uploader(fileforupload);
It's not the most elegant solution, but one work around is to simply wait if you know the program creating/editing the file is going to close it very soon...
while (dirchanged == false){}
System.Threading.Thread.Sleep(1000);
Youtube_Uploader youtubeupload = new Youtube_Uploader(fileforupload);
EDIT: Better yet, rather than a while statement use Ben Voigt's suggestion of a TaskCompletionSource. You'll still have to deal with the file being locked but you should be able to do that after the "task" has been flagged as completed.

Process.ErrorDataReceived fired after Process is disposed?

I have my code below and sometimes get ObjectDisposedException at errorWaitHandle.Set();.
How could this happen when my process instance is disposed?
System.ObjectDisposedException: Safe handle has been closed
public static int Execute(string filename, string arguments, out string output, out string error, int timeoutInMilliSeconds = Timeout.Infinite)
{
using (AutoResetEvent outputWaitHandle = new AutoResetEvent(false), errorWaitHandle = new AutoResetEvent(false))
{
// separate using for process to ensure this is disposed before handles above.
using (System.Diagnostics.Process process = new System.Diagnostics.Process())
{
process.StartInfo.FileName = filename;
process.StartInfo.Arguments = arguments;
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = true;
StringBuilder outputSB = new StringBuilder();
StringBuilder errorSB = new StringBuilder();
process.OutputDataReceived += (sender, e) =>
{
if (e.Data == null)
{
outputWaitHandle.Set();
}
else
{
outputSB.AppendLine(e.Data);
}
};
process.ErrorDataReceived += (sender, e) =>
{
if (e.Data == null)
{
errorWaitHandle.Set();
}
else
{
errorSB.AppendLine(e.Data);
}
};
process.Start();
// See http://stackoverflow.com/questions/139593/processstartinfo-hanging-on-waitforexit-why
// for why we need to read output and error asynch
process.BeginOutputReadLine();
process.BeginErrorReadLine();
if (!process.WaitForExit(timeoutInMilliSeconds) ||
!outputWaitHandle.WaitOne(timeoutInMilliSeconds) ||
!errorWaitHandle.WaitOne(timeoutInMilliSeconds))
{
throw new TimeoutException(
string.Format("Executing [{0}] with argument [{1}] didn't finish within timeout {2} milliseconds", filename, arguments, timeoutInMilliSeconds));
}
output = outputSB.ToString();
error = errorSB.ToString();
return process.ExitCode;
}
}
}

I've found that the Process events can fire in unexpected orders because of their asynchronous nature (i.e. "Exited" being fired BEFORE "ErrorDataReceived").
You also don't know how those events are being wired up under the covers of the Process class so the various object(s) lifetimes are not really known to you. By the time your handler gets called, the Process object could have been (and apparently is) disposed.
I tried to approach this problem almost identically as you; by using AutoResetEvent's and building up the Error / Data strings from within their respective event handlers.
The way I ended up fixing this is by calling Process.WaitForExit() twice:
System.Diagnostics.Process process = new System.Diagnostics.Process()
// Process setup code
if(process.WaitForExit(timeout)){
process.WaitForExit(); // Note the lack of a timeout parameter
// By now all your events should have fired and your strings built
string errorString = errorSB.ToString();
}
The excerpt from MSDN states:
When standard output has been redirected to asynchronous event
handlers, it is possible that output processing will not have
completed when this method returns. To ensure that asynchronous event
handling has been completed, call the WaitForExit() overload that
takes no parameter after receiving a true from this overload. To help
ensure that the Exited event is handled correctly in Windows Forms
applications, set the SynchronizingObject property.
Source: https://msdn.microsoft.com/en-us/library/ty0d8k56(v=vs.110)

The solution is to subscribe to OutputDataReceived and ErrorDataReceived events to actual methods instead of anonymous methods. This way you can unsubscribe in Dispose() method.
See full code here:
https://github.com/borismod/OsTestFramework/blob/master/OsTestFramework/ProcessExecutor.cs

Manipulate the output receiving from command RUN and act accordingly - C#

I guess you all misunderstood my question and closed it at How to run a Command in C# and retrieve data from it?
I had said in that post also :-
want to run a Command from command promt and want its output and maipulate its output. If required, want to close the process and display error or appropriate message. To stop the process, I have to press "F4' key on command prompt. Till the process is stopeed or killed, it has to be alive only.
I have created a class to handle running the cmd. And I keep getting the output. But on reading the output's each line I want to stop or throw exception if found anything improper in the output.
I am tying to connect to server via cmd. Server keeps on giving output. Suppose the server gave output as :
Trying to start .....
Cananot load file
.....
Exiting
While retrieving the output, I want to check for lines like "Cannot find file", "Connected Successfully, etc and set properties ccordingly (like connected = true, errorMsg = "Cannot find file". Where I am calling the class, I can take care of those proeprties and stop if found connected == true or errorMsg.length > 0. With this inform the user that "Connection is achieved or error msg stating regarding "Cannot load file" and disconnect the server if errorMsg found.
I didn't find anywhere doing any manipulation on the output receving and that's where I find myself stuck. I found a lot on internet. Am stuck and trying to figre out this part from last 3-4 days. Then have posted here.
I need help in that. Please help me. If requiried I will psot code snippets. But please help me. AND don't close this thread as answered ithout understanding my question fully. This is no duplicate.
My code is class :
public int ConnectToServer()
{
int error = 0;
connected = false;
try
{
process = Process.Start(processInfo);
process.BeginOutputReadLine();
process.OutputDataReceived += new DataReceivedEventHandler(Process_OutputDataReceived);
//if (errorMsg.Length > 0)
// throw new Exception(errorMsg);
}
catch (Exception e)
{
Console.WriteLine("Error Processing ConnectToServer : " + e.Message);
connected = false;
errorMsg = e.Message;
error = -1;
return error;
}
return error;
}
private void Process_OutputDataReceived(object sender, DataReceivedEventArgs e)
{
errorMsg = "";
connected = false;
string d = e.Data;
if (!string.IsNullOrEmpty(d))
{
if (sb != null)
sb.Append(d + "\n");
Console.WriteLine("LINE = " + d);
if (d.IndexOf("Initialization Completed") > 0)
{
connected = true;
Console.WriteLine("********* Connected = " + connected);
}
else if (isInValidLine(d))
{
//throw new Exception(d);
connected = false;
errorMsg = d;
return;
}
}
return;
}
private bool isInValidLine(string line)
{
if (line.IndexOf("Cannot load file") > 0)
{
errorMsg = line;
return true;
}
return false;
}
IS THE ABOVE CLASS CODE CORRECT WITH MY REQUIREMENTS ?
In impementation :
while (!oc.Connected)
{
timepassed = (int)(DateTime.Now - start).TotalMilliseconds;
if (timepassed > timeout)
{
oc.DisconnectServer();
connectedToVpn = false;
throw new Exception("NotConnectedException");
} else if (oc.ErrorMessage.Length > 0)
{
oc.DisconnectServer();
connectedToVpn = false;
throw new Exception(oc.ErrorMessage);
}
Thread.Sleep(100);
}
Here what I am doing is, when I get the output line, I check if it states as Conneced or is invalid. If its invalid, I set the line as the errorMsg. In my while loop I keep chekcing for Connected and errorMessage, but the value of errorMessage stays as "" only. It never gets updated, which tell me that the processing output code is never executed. Nor in debug mode I find the cursor at that line, but the Line = is displayed proeprly in Console. So, don't understand what's going wrong and where.
Hope this helps you more understand.
Thanks

once you have redirected the standard output of the process you have executed you could parse what you receive as it arrives, I believe also line by line, then you can post commands to control your process.
to read output you have redirected the standard output, to send input you should also redirect the standard input of the process.

Killing Java Process from C# Console App

I posted about this a little while ago, but I resolved the other issue and ran into one more. I am about to deploy this program to 28 hosting machines so I want to make sure this is working before I do so.
I wrote a little c# NET application that is basically a wrapper for a Java application, when my app starts, the Java app starts, when my app closes, it closes, and so on.
Everything works properly except that when I close my application, the Java application continues to run. When I create the process, I store the Process var in a variable outside of the methods, and then use that when my application goes to shutdown. For whatever reason though it is not terminating the Java application.
class Program
{
private static Process minecraftProcess;
public static void LaunchMinecraft(String file, String memoryValue)
{
String memParams = "-Xmx" + memoryValue + "M" + " -Xms" + memoryValue + "M ";
String args = memParams + "-jar " + file + " nogui";
ProcessStartInfo processInfo = new ProcessStartInfo("java.exe", args);
processInfo.CreateNoWindow = true;
processInfo.UseShellExecute = false;
try
{
//using (Process minecraftProcess = Process.Start(processInfo))
using (minecraftProcess = Process.Start(processInfo))
{
minecraftProcess.WaitForExit();
}
}
catch
{
// Log Error
}
}
static void Main(string[] args)
{
Arguments CommandLine = new Arguments(args);
// Hook ProcessExit Event
AppDomain.CurrentDomain.ProcessExit += new EventHandler(Current_ProcessExit);
if (CommandLine["file"] != null && CommandLine["memory"] != null)
{
// Launch the Application (Command Line Parameters)
LaunchMinecraft(CommandLine["file"], CommandLine["memory"]);
}
else
{
// Launch the Application (Default Parameters)
LaunchMinecraft("minecraft_server.jar", "1024");
}
}
static void Current_ProcessExit(object sender, EventArgs e)
{
System.Threading.Thread.Sleep(10000);
// If we have an active Minecraft Service, Shut it down
if (minecraftProcess != null)
{
minecraftProcess.Kill();
}
}
}

You can't Sleep in a ProcessExit handler.
The documentation states:
The total execution time of all
ProcessExit event handlers is limited,
just as the total execution time of
all finalizers is limited at process
shutdown. The default is two seconds.
An unmanaged host can change this
execution time by calling the
ICLRPolicyManager::SetTimeout method
with the OPR_ProcessExit enumeration
value.

Nevermind, I just realized the minecraftProcess variable is static.

Don't know if you did not solve this issue by yourself but:
You should be aware that there are Start methods for instances (returning bool) and static (returning a object).
You should not use using with something other than using-local variables!
Just this should work fine:
minecraftProcess = Process.Start(processInfo)
minecraftProcess.WaitForExit();

C# Read stdout of child process asynchronously

I am working with C# and having trouble understanding how to read stdout asynchronously from a child process. What I want to do is create a child process that executes an application and then present whatever is received from that process' stdout in a textbox. I need to see every output character from the child process immediately and can not wait for a line to complete, therefore I don't think that the Process.OutputDataReceived event suits my purpose. Can you tell me a sane way to accomplish this?
I have tried calling Process.StandardOutput.BaseStream.BeginRead() and passing a call-back function to it, but in this call-back function I receive an exception from Process.StandardOutput.BaseStream.EndRead().
My code looks like this (the child process is a script engine - abbreviated "SE" - for verifying the functionality of an external device. Scripts are executed in sequence and each script requires one instance of the SE application)
private bool startScript()
{
// Starts the currently indexed script
if (null != scriptList)
{
if (0 == scriptList.Count || scriptListIndexer == scriptList.Count)
{
// If indexer equals list count then there are no active scripts in
// the list or the last active script in the list has just finished
return false; // ## RETURN ##
}
if (ScriptExecutionState.RUNNING == scriptList[scriptListIndexer].executionState)
{
return false; // ## RETURN ##
}
if (0 == SE_Path.Length)
{
return false; // ## RETURN ##
}
SE_Process = new Process();
SE_Process.StartInfo.FileName = SE_Path;
SE_Process.StartInfo.CreateNoWindow = true;
SE_Process.StartInfo.UseShellExecute = false;
SE_Process.StartInfo.RedirectStandardError = true;
SE_Process.StartInfo.RedirectStandardOutput = true;
SE_Process.EnableRaisingEvents = true;
SE_Process.StartInfo.Arguments = scriptList[scriptListIndexer].getParameterString();
// Subscribe to process exit event
SE_Process.Exited += new EventHandler(SE_Process_Exited);
try
{
if (SE_Process.Start())
{
// Do stuff
}
else
{
// Do stuff
}
}
catch (Exception exc)
{
// Do stuff
}
// Assign 'read_SE_StdOut()' as call-back for the event of stdout-data from SE
SE_Process.StandardOutput.BaseStream.BeginRead(SE_StdOutBuffer, 0, SE_BUFFERSIZE, read_SE_StdOut, null);
return true; // ## RETURN ##
}
else
{
return false; // ## RETURN ##
}
}
private void read_SE_StdOut(IAsyncResult result)
{
try
{
int bytesRead = SE_Process.StandardOutput.BaseStream.EndRead(result); // <-- Throws exceptions
if (0 != bytesRead)
{
// Write the received data in output textbox
...
}
// Reset the callback
SE_Process.StandardOutput.BaseStream.BeginRead(SE_StdOutBuffer, 0, SE_BUFFERSIZE, read_SE_StdOut, null);
}
catch (Exception exc)
{
// Do stuff
}
}
void SE_Process_Exited(object sender, EventArgs e)
{
// Keep track of whether or not the next script shall be started
bool continueSession = false;
switch (SE_Process.ExitCode)
{
case 0: // PASS
{
// Do stuff
}
...
}
SE_Process.Dispose(); // TODO: Is it necessary to dispose of the process?
if (continueSession)
{
ts_incrementScriptListIndexer();
if (scriptListIndexer == scriptList.Count)
{
// Last script has finished, reset the indexer and re-enable
// 'scriptListView'
...
}
else
{
if (!startScript())
{
// Do stuff
}
}
}
else
{
ts_resetScriptListIndexer();
threadSafeEnableScriptListView();
}
}
What happens is that after one SE process finishes I get an exception of type InvalidOperationException that says
StandardOut has not been redirected or
the process hasn't started yet.
from the call to SE_Process.StandardOutput.BaseStream.EndRead(). I do not understand why because I have set SE_Process.StartInfo.RedirectStandardOutput before start of every new process. It appears to me as if the stdout stream of an exited process calls my read_SE_StdOut() function after the process has been disposed, is that possible?
Thank you for reading!

The exception you get is normal. One BeginRead() call can never succeed: the last one, just after the process terminates. You'd normally avoid calling BeginRead() again if you know the process is completed so you don't get the exception. However, you'd rarely know. Just catch the exception. Or use BeginOutputReadLine() instead, it will catch it for you.
I'm guessing that you also redirect stderr and that the tool uses it to output "X". There is no way to keep output on both stderr and stdout synchronized after it is buffered and redirected.