I have an exe file, say XYZ.exe which takes in a csv file, and do some other manipulation, like querying DB based on the things in csv file.
Now, there are 4 csv files, file_1.csv to file_4.csv, same format, but different contents. What i wanna do is to initial 4 process, all running XYZ.exe, and each with one of the csv files. They all run in background.
I have tried to use Process.Start(#"XYZ.exe", input arguments). However, it looks like the second process would not start till the first process finishes. I wonder how should i change the code to accomplish the work.
Use this overload:
Process.Start Method (ProcessStartInfo)
#JimMischel you should be rewarded for points!
i think what you are trying to do here is called : Multithreading, my idea is that you can create an application that initiate 4 threads and you start all of them together, it is something like that :
using System.Threading;
{
class Program
{
static void Main(string[] args)
{
//Here is your main program :
//Initiate the thread (tell it what to do)
ThreadStart testThreadStart = new ThreadStart(new Program().testThread);
//Create 4 Threads and give the Path of the 4 CSV files to process
Thread Thread1 = new Thread(() => testThreadStart(PathOfThe1stCSVFile));
Thread Thread2 = new Thread(() => testThreadStart(PathOfThe2ndCSVFile));
Thread Thread3 = new Thread(() => testThreadStart(PathOfThe3rdCSVFile));
Thread Thread4 = new Thread(() => testThreadStart(PathOfThe4thCSVFile));
//Start All The Threads Together
testThread1.Start();
testThread2.Start();
testThread3.Start();
testThread4.Start();
//All The Threads are finished
Console.ReadLine();
}
public void testThread(String CSVfilePathToProcess)
{
//executing in thread
//put the : Process.Start(#"XYZ.exe", input arguments). here !!!
//and put the CSVfilePathToProcess as arguments
}
}
}
Edit : If Multithreading is little complicated for you, you can also use backgroundworker in C# , but the idea is the same.
Related
This question already has answers here:
Kill child process when parent process is killed
(16 answers)
Closed 4 months ago.
I am doing a simple console application that loads files from a database into a hashset. These files are then processed in a parallel foreach loop. This console application does launch a new Process object for each file it needs to process. So it opens new console windows with the application running. I am doing it this way because of logging issues I have if I run parsing from within the application where logs from different threads write into each other.
The issue is, when I do close the application, the parallel foreach loop still tries to process more files before exiting. I want all tasks in the code to stop immediately when I kill the application. Here is code excerpts:
My cancel is borrowed from: Capture console exit C#
Essentially the program performs some cleanup duties when it receives a cancel command such as CTRL+C or closing window with X button
The code I am trying to cancel is here:
class Program
{
private static bool _isFileLoadingDone;
static ConcurrentDictionary<int, Tuple<Tdx2KlarfParserProcInfo, string>> _currentProcessesConcurrentDict = new ConcurrentDictionary<int, Tuple<Tdx2KlarfParserProcInfo, string>>();
static void Main(string[] args)
{
try
{
if (args.Length == 0)
{
// Some boilerplate to react to close window event, CTRL-C, kill, etc
LaunchFolderMode();
}
}
}
}
Which calls:
private static void LaunchFolderMode()
{
//Some function launched from Task
ParseFilesUntilEmpty();
}
And this calls:
private static void ParseFilesUntilEmpty()
{
while (!_isFileLoadingDone)
{
ParseFiles();
}
ParseFiles();
}
Which calls:
private static void ParseFiles()
{
filesToProcess = new HashSet<string>(){#"file1", "file2", "file3", "file4"} //I actuall get files from a db. this just for example
//_fileStack = new ConcurrentStack<string>(filesToProcess);
int parallelCount = 2
Parallel.ForEach(filesToProcess, new ParallelOptions { MaxDegreeOfParallelism = parallelCount },
tdxFile =>{
ConfigureAndStartProcess(tdxFile);
});
}
Which finally calls:
public static void ConfigureAndStartProcess(object fileName)
{
string fileFullPath = fileName.ToString();
Process proc = new Process();
string fileFullPathArg1 = fileFullPath;
string appName = #".\TDXXMLParser.exe";
if (fileFullPathArg1.Contains(".gz"))
{
StartExe(appName, proc, fileFullPathArg1); //I set up the arguments and launch the exes. And add the processes to _currentProcessesConcurrentDict
proc.WaitForExit();
_currentProcessesConcurrentDict.TryRemove(proc.Id, out Tuple<Tdx2KlarfParserProcInfo, string> procFileTypePair);
proc.Dispose();
}
}
The concurrent dictionary to monitor processes uses the following class in the tuple:
public class Tdx2KlarfParserProcInfo
{
public int ProcId { get; set; }
public List<long> MemoryAtIntervalList { get; set; } = new List<long>();
}
For the sake of how long these code excerpts are, I omitted the 'StartExe()' function. All it does is set up arguments and starts the Process object process.
Why is the parallel.Foreach insisting on running even after I close the program? Is there a better parallel processing method I can use which will allow me to kill whatever files I am currently processing immedietly without trying to start a new process. Which the parallel.Foreach does?
I have tried killing it with Parallel State Stop method but it still tries to process more files before finally exiting.
Unless I'm mistaking, your code seems to do no work on its own, it just launches executables and waits for them to end. And yet you're starving your thread pool on code that's just sitting there waiting for the external processes to end. Now, again if I understand correctly, this part works. It's very wasteful and utterly non-scalable, but it works.
The only thing you seem to be missing is closing the processes early when your own process ends. This is rather trivial: CancellationToken. You simply create a CancellationTokenSource in your main function and pass it down to every worker object, and when your program is meant to end you set it. That only leaves you to respond to it, and that's as easy as replacing your proc.WaitForExit(); with something like
// this is how we coded in .Net 1.0, released in Feb. 2002.
while(!proc.HasExited && !ct.IsCancellationRequested)
Thread.Sleep(1000);
if(ct.IsCancellationRequested)
proc.Kill();
Now, if you also want to fix your first problem, start writing async code. Process.WaitForExitAsync(CancellationToken) returns an awaitable task that you can await with a cancellation token, so the work is done for you. Stop using Parallel.ForEach, this isn't the 90s, you have Task.WhenAll to do the collection. And at the end of all this, you'll see that your code will boil down to perhaps 10 good lines of code, instead of the mess you made for yourself.
FURTHER EDIT
the following is not production code - I'm just playing around with a couple of classes trying to figure out how I run processes within threads - or even if that is viable. I've read various definitions on MSDN but am a newbie to threads and processes so any further definitive references to articles would be appreciated
this is fine...
class Program {
static void Main(string[] args) {
Notepad np = new Notepad();
Thread th = new Thread(new ThreadStart(np.startNPprocess));
th.Start();
Console.WriteLine("press [enter] to exit");
Console.ReadLine();
}
}
public class Notepad {
public void startNPprocess() {
Process pr = new Process();
ProcessStartInfo prs = new ProcessStartInfo();
prs.FileName = #"notepad.exe";
pr.StartInfo = prs;
pr.Start();
}
}
this isn't...
class Program {
static void Main(string[] args) {
Process pr = new Process();
ProcessStartInfo prs = new ProcessStartInfo();
prs.FileName = #"notepad.exe";
pr.StartInfo = prs;
ThreadStart ths = new ThreadStart(pr.Start);
Thread th = new Thread(ths);
th.Start();
Console.WriteLine("press [enter] to exit");
Console.ReadLine();
}
}
Why does the second not do the same as the first? In the second script I'm trying to pass Process.Start using the Threadstart delegate ...I thought this would be ok as its a void method?
Is the first script the only option or can I change the second slightly so that it effectively does the same job as the first i.e start an instance of Notepad in a specified thread?
EDIT
Some background as to why I'm playing around with this code: ultimately I need to build an application which will be running several Excel processes simultaneously. These processes can be troublesome when VBA errors as it results in a dialogbox. So I thought if each process was running in a thread then if a particular thread has been running for too long then I could kill the thread. I'm a newbie to Threads/Processes so basically playing around with possibilities at the moment.
A ThreadStart expects a delegate that returns void. Process.Start returns bool, so is not a compatible signature. You can swallow the return value in by using a lambda that gives you a delegate of the correct return type (i.e. void) as follows:
Process pr = new Process();
ProcessStartInfo prs = new ProcessStartInfo();
prs.FileName = #"notepad.exe";
pr.StartInfo = prs;
ThreadStart ths = new ThreadStart(() => pr.Start());
Thread th = new Thread(ths);
th.Start();
...but it's probably advisable to check the return value:
ThreadStart ths = new ThreadStart(() => {
bool ret = pr.Start();
//is ret what you expect it to be....
});
Of course, a process starts in a new process (a completely separate bunch of threads), so starting it on a thread is completely pointless.
you can make changes like
ThreadStart ths = new ThreadStart(delegate() { pr.Start(); });
Just start the process normally using this code:
Process.Start("notepad.exe");
There is no point and no benefits in creating a thread to run a new process. It's like running a batch file that executes "cmd.exe" when you can directly execute "cmd.exe"... you are just doing more than what's necessary for nothing. Don't reinvent the wheel and play easy :P
Not answering directly the OP, but as this thread helped me track the right direction, I do want to answer this:
"starting it on a thread is completely pointless"
I have a .Net server that uses NodeJS as its TCP socket manager. I wanted the NodeJS to write to the same output as the .Net server and they both run in parallel. So opening in a new thread allowed me to use
processNodeJS.BeginErrorReadLine()
processNodeJS.BeginOutputReadLine()
processNodeJS.WaitForExit()
while not blocking the main thread of the .Net server.
I hope it makes sense to someone and if you have a better way to implement what I've just described, I'll be more than happy to hear.
You can start the process in another thread by using the start keyword like below this cod:
Process.Start("start notepad.exe");
in this way, your GUI program doesn't freeze when you run the notepad.
I have a .NET 4.0 ASP.NET project which requires some threading work I've never really messed with before and I've been looking at this for days and I'm still clueless =/
Basically I want something like when you take a ticket at the deli and wait your turn before they get back to you. I'll try and relate this and see if it makes any sense...
function starts ---> gets to section where it needs to "take a ticket" (I assume queue some type of item in a blockingcollection) and waits until other "tickets" (a.k.a other instances of the same function) are completed before it gives the function the OK to resume (blocking collection gets to the item in the queue) ---> finish function.
I don't need/want to do any work in the queue, I just want the function to statically wait it's turn among other instances of the function. Does that make sense? Is that possible?
Please provide code if possible as I've seen tons of examples but none of them make sense/don't do what I want.
If you want to have the timer solution, I'd enqueue all operations into a BlockingCollection and have a dedicated thread dequeue them. This thread would wait 5s and then push the dequeued item onto the thread pool. This dedicated thread should do this in an infinite loop. Dequeue, wait, push.
What I actually recommend however, is that you use the SemaphoreSlim class to throttle the number of concurrent requests to this fragile web service. Probably you should pick a number between 1 and 5 or so as the allowed amount of concurrency.
Alright so after researching document after document and playing with numerous rewrites of code I finally figured out I wasn't using the AutoResetEvent right and how to use a blocking collection on a dedicated thread. So here was the final solution using an AutoResetEvent with a BlockingCollection. This solution below might not show the same results 100% of the time (just because I believe it has to do with thread timing of when something was entered into the blocking collection) but the end result is that it does exactly what I want.
class Program
{
static void Main(string[] args)
{
TaskProcessor tp = new TaskProcessor();
Thread t1 = new Thread(new ParameterizedThreadStart(tp.SubmitRequest));
t1.Start(1);
Thread t2 = new Thread(new ParameterizedThreadStart(tp.SubmitRequest));
t2.Start(2);
Thread t3 = new Thread(new ParameterizedThreadStart(tp.SubmitRequest));
t3.Start(3);
}
}
class TaskProcessor
{
private AutoResetEvent _Ticket;
public TaskProcessor()
{
_Continue = new AutoResetEvent(false);
}
public void SubmitRequest(object i)
{
TicketingQueue dt = new TicketingQueue();
Console.WriteLine("Grab ticket for customer {0}", (int)i);
dt.GrabTicket(_Ticket);
_Continue.WaitOne();
Console.WriteLine("Customer {0}'s turn", (int)i);
}
}
public class TicketingQueue
{
private static BlockingCollection<AutoResetEvent> tickets = new BlockingCollection<AutoResetEvent>();
static TicketingQueue()
{
var thread = new Thread(
() =>
{
while (true)
{
AutoResetEvent e = tickets.Take();
e.Set();
Thread.Sleep(1000);
}
});
thread.Start();
}
public void GrabTicket(AutoResetEvent e)
{
tickets.Add(e);
}
}
I have been playing around with Threads and Tasks (.net 4) and noticed some odd behavior when you launch multiple threads without waiting a few miliseconds between each thread started call.
The example below when run does not output what I expected:
1
2
1
2
But instead only outputs:
2
2
2
2
Below is the code that I am running.
public static void Main()
{
var items = new[] {"1", "2"};
foreach (var item in items)
{
var thread = new Thread(() => Print(item));
thread.Start();
//var task = Task.Factory.StartNew(() => Print(item));
}
}
static void Print(string something)
{
while (true)
{
Console.WriteLine(something);
Thread.Sleep(1000);
}
}
Now when I call Thread.Sleep(50) after the thread.Start() then only does the output look as expected
1
2
1
2
My question is:
Why when you do not wait between launching both threads does the first
thread loose the method's parameter value you initially started it with?
i.e. first thread is launched with parameter of "1", second thread is launched with parameter of "2", however first thread's parameter becomes "2" as well? This makes no sense, especially since Print() method paramter is a value type of string.
Google "access to modified closure". What's happening is your local variable "item" is getting it's value changed before the Print function is invoked. A solution would be to create a new variable inside the scope of the loop and assign item to it.
The item is evaluated at the time that the thread you create starts due to c# closures. Another way to force the item to evaluate is to introduce a variable so that the closure will include it like so:
foreach (var item in items)
{
var closedItem = item;
var thread = new Thread(() => Print(closedItem));
thread.Start();
}
Your problem is not with threads. Your problem is with the closure and the foreach. You can read here why:
http://blogs.msdn.com/b/ericlippert/archive/2009/11/12/closing-over-the-loop-variable-considered-harmful.aspx
When you play with the timing of the threads you reorder the timings of the main thread as well so sometimes the loop will be executed before the print method of the new thread runs and sometimes after.
Show us the thread starting code and you'll find that you do not pass a constant string but a reference variable and in between calling those Start methods you are probably changing the variable.
super simple question, but I just wanted some clarification. I want to be able to restart a thread using AutoResetEvent, so I call the following sequence of methods to my AutoResetEvent.
setupEvent.Reset();
setupEvent.Set();
I know it's really obvious, but MSDN doesn't state in their documentation that the Reset method restarts the thread, just that it sets the state of the event to non-signaled.
UPDATE:
Yes the other thread is waiting at WaitOne(), I'm assuming when it gets called it will resume at the exact point it left off, which is what I don't want, I want it to restart from the beginning. The following example from this valuable resource illustrates this:
static void Main()
{
new Thread (Work).Start();
_ready.WaitOne(); // First wait until worker is ready
lock (_locker) _message = "ooo";
_go.Set(); // Tell worker to go
_ready.WaitOne();
lock (_locker) _message = "ahhh"; // Give the worker another message
_go.Set();
_ready.WaitOne();
lock (_locker) _message = null; // Signal the worker to exit
_go.Set();
}
static void Work()
{
while (true)
{
_ready.Set(); // Indicate that we're ready
_go.WaitOne(); // Wait to be kicked off...
lock (_locker)
{
if (_message == null) return; // Gracefully exit
Console.WriteLine (_message);
}
}
}
If I understand this example correctly, notice how the Main thread will resume where it left off when the Work thread signals it, but in my case, I would want the Main thread to restart from the beginning.
UPDATE 2:
#Jaroslav Jandek - It's quite involved, but basically I have a CopyDetection thread that runs a FileSystemWatcher to monitor a folder for any new files that are moved or copied into it. My second thread is responsible for replicating the structure of that particular folder into another folder. So my CopyDetection thread has to block that thread from working while a copy/move operation is in progress. When the operation completes, the CopyDetection thread restarts the second thread so it can re-duplicate the folder structure with the newly added files.
UPDATE 3:
#SwDevMan81 - I actually didn't think about that and that would work save for one caveat. In my program, the source folder that is being duplicated is emptied once the duplication process is complete. That's why I have to block and restart the second thread when new items are added to the source folder, so it can have a chance to re-parse the folder's new structure properly.
To address this, I'm thinking of maybe adding a flag that signals that it is safe to delete the source folder's contents. Guess I could put the delete operation on it's own Cleanup thread.
#Jaroslav Jandek - My apologies, I thought it would be a simple matter to restart a thread on a whim. To answer your questions, I'm not deleting the source folder, only it's content, it's a requirement by my employer that unfortunately I cannot change. Files in the source folder are getting moved, but not all of them, only files that are properly validated by another process, the rest must be purged, i.e. the source folder is emptied. Also, the reason for replicating the source folder structure is that some of the files are contained within a very strict sub-folder hierarchy that must be preserved in the destination directory. Again sorry for making it complicated. All of these mechanisms are in place, have been tested and are working, which is why I didn't feel the need to elaborate on them. I only need to detect when new files are added so I may properly halt the other processes while the copy/move operation is in progress, then I can safely replicate the source folder structure and resume processing.
So thread 1 monitors and thread 2 replicates while other processes modify the monitored files.
Concurrent file access aside, you can't continue replicating after a change. So a successful replication only occurs when there is long enough delay between modifications. Replication cannot be stopped immediately since you replicate in chunks.
So the result of monitoring should be a command (file copy, file delete, file move, etc.).
The result of a successful replication should be an execution of a command.
Considering multiple operations can occur, you need a queue (or queued dictionary - to only perform 1 command on a file) of commands.
// T1:
somethingChanged(string path, CT commandType)
{
commandQueue.AddCommand(path, commandType);
}
// T2:
while (whatever)
{
var command = commandQueue.Peek();
if (command.Execute()) commandQueue.Remove();
else // operation failed, do what you like.
}
Now you may ask how to create a thread-safe query, but that probably belongs to another question (there are many implementations on the web).
EDIT (queue-less version with whole dir replication - can be used with query):
If you do not need multiple operations (eg. always replication the whole directory) and expect the replication to always finish or fail and cancel, you can do:
private volatile bool shouldStop = true;
// T1:
directoryChanged()
{
// StopReplicating
shouldStop = true;
workerReady.WaitOne(); // Wait for the worker to stop replicating.
// StartReplicating
shouldStop = false;
replicationStarter.Set();
}
// T2:
while (whatever)
{
replicationStarter.WaitOne();
... // prepare, throw some shouldStops so worker does not have to work too much.
if (!shouldStop)
{
foreach (var file in files)
{
if (shouldStop) break;
// Copy the file or whatever.
}
}
workerReady.Set();
}
I think this example clarifies (to me anyway) how reset events work:
var resetEvent = new ManualResetEvent(false);
var myclass = new MyAsyncClass();
myclass.MethodFinished += delegate
{
resetEvent.Set();
};
myclass.StartAsyncMethod();
resetEvent.WaitOne(); //We want to wait until the event fires to go on
Assume that MyAsyncClass runs the method on a another thread and fires the event when complete.
This basically turns the asynchronous "StartAsyncMethod" into a synchronous one. Many times I find a real-life example more useful.
The main difference between AutoResetEvent and ManualResetEvent, is that using AutoResetEvent doesn't require you to call Reset(), but automatically sets the state to "false". The next call to WaitOne() blocks when the state is "false" or Reset() has been called.
You just need to make it loop like the other Thread does. Is this what you are looking for?
class Program
{
static AutoResetEvent _ready = new AutoResetEvent(false);
static AutoResetEvent _go = new AutoResetEvent(false);
static Object _locker = new Object();
static string _message = "Start";
static AutoResetEvent _exitClient = new AutoResetEvent(false);
static AutoResetEvent _exitWork = new AutoResetEvent(false);
static void Main()
{
new Thread(Work).Start();
new Thread(Client).Start();
Thread.Sleep(3000); // Run for 3 seconds then finish up
_exitClient.Set();
_exitWork.Set();
_ready.Set(); // Make sure were not blocking still
_go.Set();
}
static void Client()
{
List<string> messages = new List<string>() { "ooo", "ahhh", null };
int i = 0;
while (!_exitClient.WaitOne(0)) // Gracefully exit if triggered
{
_ready.WaitOne(); // First wait until worker is ready
lock (_locker) _message = messages[i++];
_go.Set(); // Tell worker to go
if (i == 3) { i = 0; }
}
}
static void Work()
{
while (!_exitWork.WaitOne(0)) // Gracefully exit if triggered
{
_ready.Set(); // Indicate that we're ready
_go.WaitOne(); // Wait to be kicked off...
lock (_locker)
{
if (_message != null)
{
Console.WriteLine(_message);
}
}
}
}
}