Hi I have a problem using Filesystemwatcher & BackgroundWorker process.
I have a windows forms application that checks for new text files on a folder, it process them and creates xml files from them.
I´m using FSW to monitor for new txt files on a folder, the app works fine but when the folder receives a large amount of files (let's say 1000), the app freezes because it's processing all of them.
it occurred to me to add a backgroundworker, so the FSW calls it everytime a new file is created, this way we can process the file on the background without freezing the UI.
This idea did not work because for every file that is created, I try to call the RunWorkerAsync() method, so if it's busy processing a file and I try to process a new one it will throw the following error:
"This BackgroundWorker is currently busy and cannot run multiple tasks concurrently."
So I tried to loop the method with a while til it gets available but, infinite exception is thrown.
this is the simplified version of my code:
private void fileSystemWatcher1_Created(object sender, System.IO.FileSystemEventArgs e)
{
readFile();
}
private void readFile()
{
while (backgroundWorker1.IsBusy)
{
readFile();
}
backgroundWorker1.RunWorkerAsync(idx);
}
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
int i = (int)e.Argument;
i += 1;
e.Result = i;
}
private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
label1.Text = "Processing...";
this.Refresh();
}
private void backgroundWorker1_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
label1.Text = "Completed...";
this.Refresh();
idx = (int)e.Result;
}
The exception thrown says “An unhandled exception of type 'System.StackOverflowException' occurred in WindowsFormsApplication2.exe, make sure you do not have an infinite loop or recursion”
Of course I could remove the FSW, but I'd like to know if there's a way to make them work together, any ideas?
What you have is a classic Producer/Consumer problem.
Solve it with a System.Collections.Concurrent.ConcurrentQueue<string>.
On the FSW event, add the filename to the Queue.
Start 1 or 2 BackgroundWorkers to process the queue.
And this is the code that overflows your stack in a hurry:
private void readFile()
{
while (backgroundWorker1.IsBusy)
{
readFile(); // the recursive call, will fail quickly
}
backgroundWorker1.RunWorkerAsync(idx);
}
Not only does this cause an SO exception, it also blocks your main thread.
You need a better way to wait, and the ConcurrentQueue gives you that.
Instantiating new BackgroundWorkers would do the trick, as would Henk's solution above.
Or, you can do it without changing your code too much just using the ThreadPool.
private void fileSystemWatcher1_Created(object sender, System.IO.FileSystemEventArgs e)
{
ThreadPool.QueueUserWorkItem(o => readFile(e));
}
public void readFile(System.IO.FileSystemEventArgs e)
{
this.BeginInvoke(new MethodInvoker(() =>
{
label1.Text = "Processing...";
this.Refresh(); //you shouldn't need this
}));
//your long running read/processing... doing something event args
this.BeginInvoke(new MethodInvoker(() =>
{
label1.Text = "Completed...";
this.Refresh();
idx = (int) e.Result;
}));
}
Why not instantiate a new BackgroundWorker in readFile instead of reusing?
Related
I have made a simple gui in C# wpf, (sorry I can't show the GUI because my reputation is below 10)
It's consist of richtextbox and some other controls. Umm... this application will read a file then display the file contents to richtextbox line by line while reading the file using background worker. The function that read the file is like this :
public int parse_persoFile2(string fname, BackgroundWorker worker, DoWorkEventArgs e)
{
if (fname == null) return -1;
System.IO.StreamReader ifs;
ifs = new System.IO.StreamReader(fname);
int max = (int)e.Argument;
int p = 0;
while (ifs.Peek() != -1)
{
string tempData = ifs.ReadLine();
if (tempData.Contains("CMD=5107") || tempData.Contains("CMD=5106") || tempData.Contains("CMD=5102"))
{
//field.AppendText(tempData.Remove(tempData.LastIndexOf('\\')).Remove(0, 4) + "\r\n");
//field.AppendText("--------\r\n");
//System.Threading.Thread.Sleep(500);
string data = tempData.Remove(tempData.LastIndexOf('\\')).Remove(0, 4) + "\r\n";
worker.ReportProgress(p, data);
}
p++;
}
worker.ReportProgress(100);
return 0;
}
As we can see, I'm using backgroundworker in this function to get the string readed from file then send that string to reportprogress in order to be displayed in richtextbox. As a note that persoFile2 function is made from another object in my program... :-)
Then for the rest, I have made the doWork function, worker_progressChanged, and worker_RunWorkerCompleted to make backroundWorker works correctly. Those codes are like this :
private void doWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker wrk = sender as BackgroundWorker;
parser.parse_persoFile2(fileName, wrk, e);
}
private void proggChanged(object sender, ProgressChangedEventArgs e)
{
if(e.UserState != null)
mRTB.AppendText(e.UserState.ToString());
}
private void completed(object sender, RunWorkerCompletedEventArgs e)
{
MessageBox.Show("Ok....");
}
Umm.... When I run this program, it looks that my richtextbox is not prints the string line by line from the file, but it prints it just once at the end... :-3, .. Nah that's my real problem here. I have read this article http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx, but still have no idea.... :3
If you call the ReportProgress method too rapidly, it's possible that the UI thread will not have a chance to process the "progress" and update appropriately before the BackgroundWorker is hitting it again.
private void doWork(object sender, DoWorkEventArgs e)
{
var wrk = sender as BackgroundWorker;
// obviously you wouldn't really do this :)
while(true)
wrk.ReportProgress(0);
}
To see the effect you're looking expecting, you could set an artificial "pause" in your DoWork event, in order to give the UI time to update appropriately:
private void doWork(object sender, DoWorkEventArgs e)
{
var wrk = sender as BackgroundWorker;
var p = 0;
while(true)
{
wrk.ReportProgress(p++);
Thread.Sleep(100);
}
}
As for your situation, if the code is executing that quickly, you may not actually need to be executing it in a separate thread.
Alternatively, you could update your UI to say "Please wait. Loading...", then do everything you need to do in the BackgroundWorker, and just return the final result back to the UI at the end.
I want to refresh my datagridview if there are changes in a particular xml file. I got a FileSystemWatcher to look for any changes in the file and call the datagirdview function to reload the xml data.
When i tried, i'm getting Invalid data Exception error Somebody please tell what is the mistake am i doing here??
public Form1()
{
InitializeComponent();
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = #"C:\test";
watcher.Changed += fileSystemWatcher1_Changed;
watcher.EnableRaisingEvents = true;
//watches only Person.xml
watcher.Filter = "Person.xml";
//watches all files with a .xml extension
watcher.Filter = "*.xml";
}
private const string filePath = #"C:\test\Person.xml";
private void LoadDatagrid()
{
try
{
using (XmlReader xmlFile = XmlReader.Create(filePath, new XmlReaderSettings()))
{
DataSet ds = new DataSet();
ds.ReadXml(xmlFile);
dataGridView1.DataSource = ds.Tables[0]; //Here is the problem
}
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
private void Form1_Load(object sender, EventArgs e)
{
LoadDatagrid();
}
private void fileSystemWatcher1_Changed(object sender, FileSystemEventArgs e)
{
LoadDatagrid();
}
This is because the FileSystemWatcher runs on a distinct thread, not the UI thread. On winforms apps only the UI thread - the main thread of the program - can interact with visual constrols. If you need to interact with visual controls from another thread - like this case - you must call Invoke on the target control.
// this event will be fired from the thread where FileSystemWatcher is running.
private void fileSystemWatcher1_Changed(object sender, FileSystemEventArgs e)
{
// Call Invoke on the current form, so the LoadDataGrid method
// will be executed on the main UI thread.
this.Invoke(new Action(()=> LoadDatagrid()));
}
The FileSystemWatcher is running in a seperate thread and not in the UI thread. To maintain thread safety, .NET prevents you from updating the UI from the non-UI thread (i.e. the one that created the Form components).
To resolve the issue easily, call the MethodInvoker method of the target Form from your fileSystemWatcher1_Changed event. See MethodInvoker Delegate for more details on how to do this. There are other options on how to do this, incl. setting up a synchronized (i.e. thread-safe) object for holding the results/flag of any event, but this requires no changes to the Form code (i.e. in case of games, one could just poll the synchronized object in the main game loop etc).
private void fileSystemWatcher1_Changed(object sender, FileSystemEventArgs e)
{
// Invoke an anonymous method on the thread of the form.
this.Invoke((MethodInvoker) delegate
{
this.LoadDataGrid();
});
}
Edit: Corrected previous answer which had a problem within the delegate, the LoadDataGrid was missing this. and it would not resolve as such.
I have a WPF application that executes external programs to process media files, and so that the GUI doesn't freeze when the media files are being processed, I execute the process on a separate thread through backgroundworker.
private void BackgroundWorkerExecProcess(Process process)
{
BackgroundWorker worker = new BackgroundWorker();
worker.WorkerReportsProgress = false;
worker.DoWork += DoWork;
worker.RunWorkerCompleted += WorkerCompleted;
worker.RunWorkerAsync(process);
}
void DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker worker = sender as BackgroundWorker;
Process process = e.Argument as Process;
process.Start();
string stderr = process.StandardError.ReadToEnd();
//I want to display stderr on main thread
process.WaitForExit();
}
void WorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
//some code to update gui telling user that process has finished
}
so, if there is something printed to stderr, I can see it in the debugger, but if I try to do anything with the string stderr, such as if I have a textbox called "_tbLog" and did
_tbLog.Text+=stderr;
I get an error from the compiler about them being on separate threads. is there a way to pass the object from the worker thread to the main thread?
In DoWork, set e.Result to your object. In the WorkerCompleted you can get that object back out... it will once again be e.Result of type object. Just cast it to the object it was. The WorkerCompleted should be on the correct thread.
Here is one of mine:
private void workerUpdateBuildHistory_DoWork(object sender, System.ComponentModel.DoWorkEventArgs e)
{
UpdateStatusModel model = (UpdateStatusModel)e.Argument;
BuildService buildService = new BuildService(model.TFSUrl);
e.Result = buildService.UpdateBuildHistoryList(model);
}
private void workerUpdateBuildHistory_RunWorkerCompleted(object sender, System.ComponentModel.RunWorkerCompletedEventArgs e)
{
BuildHistoryListModel model = (BuildHistoryListModel)e.Result;
if (model != null)
{
listViewPastBuilds.Items.Clear();
foreach (var item in model.Builds)
{
listViewPastBuilds.Items.Add(item);
}
}
}
Use your WorkerCompleted event handler to make changes the UI, it runs on the right thread. All you have to do is pass the string to the event handler. Which is what DoWorkEventArgs.Result was designed to do. You'll retrieve it in the event handler from e.Result. Thus:
void DoWork(object sender, DoWorkEventArgs e)
{
//...
e.Result = stderr;
}
void WorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
if (e.Error != null) DisplayError(e.Error);
else _tbLog.Text += (string)e.Result;
}
First you need to place whatever result object (in this example, a list of strings) in the DoWorkEventArgs.Result property, then retrieve this object via the RunWorkerCompletedArgs.Result property
Then, hook up an event handler RunWorkedCompleted event of the Background worker and have it pass back whatever object you want in the RunWorkerCompletedEventArgs.Result property.
Example:
void DoWork(object sender, DoWorkEventArgs arg)
{
List<string> results = new List<string>();
results.Add("one");
results.Add("two");
results.Add("three");
arg.Results = results;
}
void WorkComplete(object sender, runWorkerCompelteEventArgs arg)
{
//Get our result back as a list of strings
List<string> results = (List<string>)arg.Result;
PrintResults(results);
}
Note: I have not tested this code, but I believe it should compile.
http://msdn.microsoft.com/en-us/library/system.componentmodel.runworkercompletedeventargs.result.aspx
http://msdn.microsoft.com/en-us/library/system.componentmodel.doworkeventargs.aspx
you can also use the dispatcher as #Zembi mentiones:
this.Dispatcher.Invoke( new Action( () => {
_tbLog.Text+=stderr;
} ) );
you can also use TPL to make sure things get run on the right thread
-edit-
Here is a good article on diffrent ways to do ui updates, including using TPL
I have a console application that utilizes the BackgroundWorker object and wanted to test my output with a Console.WriteLine(fooBar). However, that application exits as the application executes the Console.WriteLine command.
Here's a watered down version that illustrates what I wanted to do:
protected static void bWorker_DoWork(object sender, DoWorkEventArgs e)
{
for (int i = startId; i <= endId; i++)
{
Console.WriteLine(i.ToString());
Thread.Sleep(1000);
}
}
Any ideas why the application would appear to exit like that?
The application exits because it in fact completes execution and has no more work to do ;-) Once you schedule the background worker and kick off the thread to do it's thing, you have to tell the main thread to stop and wait for one thing or another. A very common way of doing this (Generally used in test/sample code) is to simply issue a Console.ReadKey(); as the very last line of code in your main method. This will cause your application to wait until you press a key before exiting the process.
For your backgroundworker set WorkerReportsProgress to true. Subscribe to ProgressChanged event like this:
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
for (var i = 0; i < 1000; i++)
{
backgroundWorker1.ReportProgress(i);
}
}
private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
Console.WriteLine(e.ProgressPercentage);
}
If you need to transfer more than just int from your background thread to UI thread, then you could do something like this:
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
for (var i = 0; i < 1000; i++)
{
var myObjectInstance = new MyObject{ ...};
backgroundWorker1.ReportProgress(null, myObjectInstance);
}
}
private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
var myObjectInstance = (MyObject)e.UserState;
Console.WriteLine(myObjectInstance);
}
I might not be understanding your setup correctly. I'm guessing that you're running the background thread, but the main process is exiting which causes the thread to be stopped before it gets to do anything. Maybe try putting something in your main process that prevents the main thread from exiting like Console.ReadKey();
I have a Winforms application which is working fine.. using a BackgroundWorkerThread to manage GUI usability during processing of serial data to a device.
It's working fine.
Now, I am adding a new method, and copying what I did in other forms. But I am getting a cross thread exception.
I declare my BWT like this:
BackgroundWorker bw = new BackgroundWorker();
bw.WorkerSupportsCancellation = true;
bw.DoWork += DownloadGpsDataFromDevice;
bw.WorkerReportsProgress = true;
bw.RunWorkerAsync();
I then have a method delared like this, which does the background work:
private void DownloadGpsDataFromDevice(object sender, DoWorkEventArgs e)
{
_performScreenUpdate = true;
tsStatus.Text = "Downloading GPS Data...";
Invalidate();
Refresh();
Common.WriteLog("Extracting raw GPS data. Sending LL.");
ReplyString raw = DeviceServices.ExecuteCommand("$LL");
The DeviceServices.ExecuteCommand("$LL"); is the bit that does the work, but I am getting the exception on the previous line, where I log to a text file. Now, that makes you worry - writing to a file. However, I have done this thousands of times in another BWT.
I made the writing thread safe. Here this my Common.WriteLog method:
public static void WriteLog(string input)
{
lock (_lockObject)
{
WriteLogThreadSafe(input);
}
}
private static void WriteLogThreadSafe(string input)
{
Directory.CreateDirectory(LogFilePath);
StreamWriter w = File.AppendText(LogFilePath + #"\" + LogFileName);
try
{
w.WriteLine(string.Format("{0}\t{1}", DateTime.Now, input));
}
catch (Exception e)
{
System.Console.WriteLine("Error writing to log file!");
System.Console.WriteLine("Tried to write: [" + input + "]");
System.Console.WriteLine("Failed with error: [" + e.Message + "]");
}
finally
{
w.Close();
}
}
This have been working for ages. I don't believe the error is there. I think I am just missing something on the call maybe?
You cannot change UI elements from BackgroundWorker thread. You'll have to marshall back to UI thread by calling Invoke().
Try this
private void DownloadGpsDataFromDevice(object sender, DoWorkEventArgs e)
{
_performScreenUpdate = true;
Invoke((MethodInvoker)(() => {
tsStatus.Text = "Downloading GPS Data...";
Invalidate();
Refresh();
});
...
The issue is that you are updating UI elements from non-UI thread:
Those lines should not be inside of DownloadGpsDataFromDevice
tsStatus.Text = "Downloading GPS Data...";
Invalidate();
Refresh();
To take advantage of BackgroundWorker run method bw.ReportProgress(0);. Update UI in ProgressChanged handler, which was specifically designed for this purpose.
void bw_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
if (e.ProgressPercentage = 0)
{
tsStatus.Text = "Downloading GPS Data...";
Invalidate();
Refresh();
}
}
Some instances can't or should not be accessed by multiple threads. You have two options to protect your data from cross thread exceptions.
You can lock your object when you access it from multiple threads with a lock:
object locker = new object();
SomeObject MyObject = new SomeObject();
private void FromMultipleThread()
{
lock(locker)
{
MyObject = OtherObject;
}
}
Your second option is to lock your thread with a ManualResetEvent. It is very simple, you only have to call WaitOne() from you ManualResetEvent to lock your thread while an other thread access your "cross threaded" Object.
In your case, you would want to change your UI from the reportProgress of your backgroundWorker. The reportProgress will come back to the initial thread, then you can modify your UI.
Are you sure that is the right line? I don't think you should be able to update the ui in your worker. Try commenting out the gui update and clean and build your solution to see if the logging is really the problem. To update the ui, set WorkerReportsProgress and create an event handler for that to update the ui and report progress in the worker.