So I have a list with 900+ entries in C#. For every entry in the list a method has to be executed, though these must go all at the same time. First I thought of doing this:
public void InitializeThread()
{
Thread myThread = new Thread(run());
myThread.Start();
}
public void run()
{
foreach(Object o in ObjectList)
{
othermethod();
}
}
Now the problem here is that this will execute 1 method at a time for each entry in the list. But I want every single one of them to be running at the same time.
Then I tried making a seperate thread for each entry like this:
public void InitializeThread()
{
foreach(Object o in ObjectList)
{
Thread myThread = new Thread(run());
myThread.Start();
}
}
public void run()
{
while(//thread is allowed to run)
{
// do stuff
}
}
But this seems to give me system.outofmemory exceptions (not a suprise since the list has almost a 1000 entries.
Is there a way to succesfully run all those methods at the same time? Either using multiple threads or only one?
What I'm ultimately trying to achieve is this: I have a GMap, and want to have a few markers on it. These markers represent trains. The marker pops up on the GMap at a certain point in time, and dissappears when it reaches it's destination. All the trains move about at the same time on the map.
If I need to post more of the code I tried please let me know.
Thanks in advance!
What you're looking for is Parallel.ForEach:
Executes a foreach operation on an IEnumerable in which iterations may
run in parallel.
And you use it like this:
Parallel.ForEach(ObjectList, (obj) =>
{
// Do parallel work here on each object
});
Related
I've attempted to make a simple step mode for an algorithm I'm running, and here is how it looks like:
public async Task<bool> AStarAlgorithmAsync(PFSquare curr = null)
{
// some algorithm code here
foreach(var square in Sorroundings)
{
if (SteppedMode)
{
await Task.Run(Pause);
}
if (await AStarAlgorithmAsync(square))
{
return true;
}
}
}
In my application, I have a Boolean called SteppedMode that decides if the algorithm should run one iteration per click event.
Pause() looks like this:
private void Pause()
{
while (!ContinueStep) { }
ContinueStep = false;
return;
}
And in another part of my (GUI) application I have an event which sets the boolean ContinueStep to true which in theory should end the while loop and continue the algorithm function. Currently this bit of code locks my GUI thread up and I'm almost certain there is a better way to do this.
I'm trying to get my algorithm function to run one iteration, wait for a click from the user and only then continue running the algorithm. Is there an easier and cleaner way to do this?
(This is a GUI application, not a console application.)
Your property is moonlighting as a method.
It makes no sense to set a property, to then have that property revert back to its original state immediately. As a consumer, I would be majorly confused by that behavior. Think about this code:
var myObj = new MyObject();
myObj.MyBoolean = true;
Console.WriteLine(myObj.MyBoolean); // FALSE!?
It just doesn't make sense.
The only effect you want to trigger by setting this property is to execute some code. That's exactly what methods are supposed to be used for:
public void ContinueStep()
{
Console.WriteLine("I did some work");
}
So instead of this:
myObj.ContinueStep = true;
you should be doing this:
myObject.ContinueStep();
This doesn't lock up your UI thread, while also being a lot more sensical to your consumer. The method suggests that some action will be taken (which may or may not lead to state changes in the object - that's a contextual expectation).
Infinite recursion
As an aside; based on your code, AStarAlgorithmAsync is a recursive function, and seemingly infinitely so. There doesn't seem to be an ending condition.
Every recursive level will interate over the first surrounding and then trigger the next level, which again will interate over the first surrounding and then trigger the next level, which again ...
That can't be right, but it's unclear to me how to fix it as the bigger picture is not explained in your question
A simple implementation
What I'm trying to do is get my algorithm function to run one iteration, wait for a click from the user and only then continue running the algorithm, is there an easier and cleaner way to do this?
A simple example of such a thing:
private int _index = 0;
private List<object> _myList = ...; // assume this list contains some elements
public void ProcessNextObject()
{
if(_index < _myList.Length)
{
Process(_myList[_index]);
_index++;
}
}
private void Process(object o)
{
Console.WriteLine("Processing this object!");
}
You can then hook up your click event to call ProcessNextObject().
Note that in this example, the list is processed once and cannot be processed again. By manipulating the index value, you can change that behavior as you like.
First about my goal:
I am importing a table with about 1000-5000 rows to a DataTable. This one is bound to a DataGridView. Now for every row there has to run a process that takes about 5-10 seconds. After a single process finished I want to write back the result to the DataTabel (result-column).
Because this process is independent I want to use multithreading to speed it up.
This is an example structure of my current code:
// Will be created for each row
public class FooObject
{
public int RowIndex;
public string Name;
//...
}
// Limiting running tasks to 50
private Semaphore semaphore = new Semaphore(50, 50);
// The DataTable is set up at start-up of the App (columns etc)
private DataTable DtData { get; set; } = new DataTable();
// The button that starts the process
private void btnStartLongRun(object sender, EventArgs e)
{
// some init-stuff
StartRun();
}
private async void StartRun()
{
for (int rowIndex = 0; rowIndex < DtData.Rows.Count)
{
// Creating a task to not block the UI
// Using semaphore here to not create objects
// for all lines before they get in use.
// Having this inside the real task it consumed
// a lot of ram (> 1GB)
await Task.Factory.StartNew(() =>
{
semaphore.WaitOne();
});
// The row to process
var currentRow = DtData.Rows[rowIndex];
// Creating an object from the row-data
FooObject foo = new FooObject()
{
RowIndex = rowIndex;
Name = currentRow["Name"].ToString();
}
// Not awaiting because I want multiple threads
// to run at the same time. The semaphore is
// handling this
TaskScheduler scheduler = TaskScheduler.Current;
Task.Factory.StartNew(() =>
{
// Per-row process
return ProcessFoo(foo);
}).ContinueWith((result) =>
{
FinishProcessFoo(result.Result);
}, CancellationToken.None, TaskContinuationOptions.OnlyOnRanToCompletion, scheduler);
}
}
private FooObject ProcessFoo(FooObject foo)
{
// the actual big process per line
}
private void FinishProcessFoo(FooObject foo)
{
// Locking here because I got broken index errors without
lock(DtGrid.Rows.SyncRoot)
{
// Getting the row that got processed
var procRow = DtData.Rows[foo.RowIndex];
// Writing the result to that row
procRow["Result"] = foo.Result;
// Raising the progressbar
pbData.Value++;
}
// Letting the next task start.
semaphore.Release();
}
The big problem:
In the beginning everything is working fine. All threads are running smooth and doing their job. But as longer the app runs, as more it is getting unresponsive. It looks like the app is slowly starting to block more and more.
I started a test-run with 5000 rows. It got in stuck at around row 2000. Sometimes even an error raises that the app isn't responding.
I haven't got a lot experience in multi threading. So maybe this code is totally bad. I appreciate every help in here. I would also be happy about pointing me into another direction to get this running better.
Thank you very much.
Edit
If there is anything I can debug to help in here just tell me.
Edit 2
I already enabled all Common Language Runtime Exceptions to check if there is anything that's not raising an error. Nothing.
If you want to process up to 50 rows in parallel, you could consider using a Parallel.For with a MaxDegreeOfParallelism of 50:
Parallel.For(0, DtData.Rows.Count, new ParallelOptions() { MaxDegreeOfParallelism = 50 }, rowIndex =>
{
//...
});
Starting a new task just to call WaitOne on a Semaphore is a waste of time.
You are using the UI thread to coordinate thousands of async tasks. This is bad. Wrap your call to StartRun in a new task to avoid this.
The better way of doing this is to divide the number of rows by the number of processors, then start one task per processor for just those rows. No need for Semaphore then.
I have List newJobs. Some threads add items to that list and other thread removes items from it, if it's not empty. I have ManualResetEvent newJobEvent which is set when items are added to the list, and reset when items are removed from it:
Adding items to the list is performed in the following way:
lock(syncLock){
newJobs.Add(job);
}
newJobEvent.Set();
Jobs removal is performed in the following way:
if (newJobs.Count==0)
newJobEvent.WaitOne();
lock(syncLock){
job = newJobs.First();
newJobs.Remove(job);
/*do some processing*/
}
newJobEvent.Reset();
When the line
job=newJobs.First()
is executed I sometimes get an exception that the list is empty. I guess that the check:
if (newJobs.Count==0)
newJobEvent.WaitOne();
should also be in the lock statement but I'm afraid of deadlocks on the line newJobEvent.WaitOne();
How can I solve it?
Many thanks and sorry for the long post!
You are right. Calling WaitOne inside a lock could lead to a deadlock. And the check to see if the list is empty needs to be done inside the lock otherwise there could be a race with another thread trying to remove an item. Now, your code looks suspiciously like the producer-consumer pattern which is usually implemented with a blocking queue. If you are using .NET 4.0 then you can take advantage of the BlockingCollection class.
However, let me go over a couple of ways you can do it youself. The first uses a List and a ManualResetEvent to demonstrate how this could be done using the data structures in your question. Notice the use of a while loop in the Take method.
public class BlockingJobsCollection
{
private List<Job> m_List = new List<Job>();
private ManualResetEvent m_Signal = new ManualResetEvent(false);
public void Add(Job item)
{
lock (m_List)
{
m_List.Add(item);
m_Signal.Set();
}
}
public Job Take()
{
while (true)
{
lock (m_List)
{
if (m_List.Count > 0)
{
Job item = m_List.First();
m_List.Remove(item);
if (m_List.Count == 0)
{
m_Signal.Reset();
}
return item;
}
}
m_Signal.WaitOne();
}
}
}
But this not how I would do it. I would go with the simplier solution below with uses Monitor.Wait and Monitor.Pulse. Monitor.Wait is useful because it can be called inside a lock. In fact, it is suppose to be done that way.
public class BlockingJobsCollection
{
private Queue<Job> m_Queue = new Queue<Job>();
public void Add(Job item)
{
lock (m_Queue)
{
m_Queue.Enqueue(item);
Monitor.Pulse(m_Queue);
}
}
public Job Take()
{
lock (m_Queue)
{
while (m_Queue.Count == 0)
{
Monitor.Wait(m_Queue);
}
return m_Queue.Dequeue();
}
}
}
Not answering your question, but if you are using .NET framework 4, you can use the new ConcurrentQueue which does all the locking for you.
Regarding your question:
One scenario that I can think of causing such a problem is the following:
The insertion thread enters the lock, calls newJob.Add, leaves the lock.
Context switch to the removal thread. It checks for emptyness, sees an item, enters the locked area, removes the item, resets the event - which hasn't even been set yet.
Context switch back to the insertion thread, the event is set.
Context switch back to the removal thread. It checks for emptyness, sees no items, waits for the event - which is already set, trys to get the first item... Bang!
Set and reset the event inside the lock and you should be fine.
I don't see why object removal in case of zero objects should wait for one to be added and then remove it. It looks to be being against logic.
I've a little problem with this code:
This is the "main" method of the app:
private Thread main_process;
private Clases.GestorTR processor;
public void begin()
{
processor = new Clases.GestorTR();
main_process = new Thread(new ThreadStart(processor.ExecuteP));
main_process.Start();
}
I've created a Thread to process other "Transacction Threads" to avoid blocking the GUI.
This is the method ExecuteP, on processor object:
public void ExecuteP()
{
// Readed an DataTable with BD transacction, filled with numbers
foreach (DataRow dr in dtResults.Rows)
{
int Local_number = Convert.toInt32(dr["autonum"].ToString());
ThreadStart starter;
starter = delegate { new QueryBD.QueryCounter(Local_number); };
new Thread(starter).Start();
}
}
This is QueryCounter method of QueryBD class:
....
private void QueryCounter(int _counter)
{
logs.log("ON QUERY_PROCESS: " + _counter);
}
...
Now, the problem. When calling the delegate, some threads are crossing parameters. For example, in the foreach method the log shows correct (1,2,3,4,5,6,7,8) but, in the QueryCounter method (called each time with the new thread, the log shows (1,1,1,4,5,6,6,8) for example. I've also tried to use locks, but the problem is the same. Also testing with the ThreadPool way with the same result.
I think I'm missing something in the foreach loop, because if I debug the first run, the thread is Started, but without action in the log.
Thanks!,
You should try to change some parts of your code like that:
public void ExecuteP()
{
QueryBD facade = new QueryBD.
foreach (DataRow dr in dtResults.Rows)
{
int Local_number = Convert.toInt32(dr["autonum"].ToString());
new Thread(new ParameterizedThreadStart(facade.QueryCounter)).Start(Local_number);
}
}
public void QueryCounter(object _counter)
{
...
}
Hope it works.
Btw. I've created one object called facade and I'm passing that object to various threads. It can also result in some side effects if there will be thread sensitive part of code in the facade object, so you can also consider locking there:
public void QueryCounter(object _counter)
{
lock(this)
{
//
}
}
or providing new QueryBD to each thread, but it can affect performance.
EDIT: Hey, 4 things:
While using ParametrizedThread, the variable passed to Start method of the thread (thread.Start(variable)) is copied at the time of call. Such copied variable is then used in the child thread. Anonymous delegate works different. It keeps the reference to the variable, so when the variable is used by the child thread, it can be changed by the time in your parent thread. That is why you had unpredicted behaviour.
Better explanation you can find here: Differing behavior when starting a thread: ParameterizedThreadStart vs. Anonymous Delegate. Why does it matter?.
The performance depends. If creation of your object is heavy (ex. it creates new connection to DB each time it is created) performance can be seriously affected by creation of many such objects - it is where lock is better. If creation of the object is light, you can create as many objects as you want. It depends.
If you want your code to be run in defined order, you shouldn't use threads at all. If you want to preserve execution order, sequential invoking is the right way - see Hans Passant explanation.
I have a thread that gathers a list of URLs from a website and updates the UI as it's doing so. That works fine. But, I need the Main thread to wait until the links have been gathered. I tried to do a join, but this locks up the UI. Here is my example. As you can see at the moment the foreach loop gets called at the same time as the thread is running. I want the foreach to run after the thread has ran.
Any ideas?
/** This thread will add links to list<string> linkList **/
Thread linkListThread = new Thread(new ThreadStart(getLinkList));
linkListThread.Start();
foreach (String link in linkList)
{
txtOutput.Text += link;
}
You can use a background worker. Or have the thread method call a method in main context when it is done, passing the list of items you gathered.
EDIT
I thought I should elaborate my second approach a little.
You could prepare a list instance before creating the thread:
List<string> links = new List<string>();
Then you pass this list to the thread, which fills it:
Thread t = new Thread(new ParameterizedThreadStart(FillList));
t.Start(links);
The thread method takes the list, fills it and calls a method that shows the details in the UI:
private static void FillList(object state)
{
List<string> links = (List<string>)state;
// Fill data
this.Invoke((MethodInvoker)delegate() { HandleNewLinks(links); }));
}
The HandleNewLinks method works as one would expect:
private void HandleNewLinks(List<string> links)
{
foreach (string link in links)
// Do something...
}
Move the code that needs to run after the thread has completed into an event handler for BackgroundWorker.RunWorkerCompleted
Update: The handler is invoked on the right (calling ) thread - so you can safely update the UI.
See the code snippet on the above msdn page.
It is not clear what you want: either the application waits (and is unresponsive), or the application does not wait and remains responsive. In the latter case, you might want to disable some controls / possible actions until the list has finished loading.
A dirty workaround is to do some sort of spin waiting (Join with a timeout of a few ms, returns the result whether it is done or not) with some Application.DoEvents() calls.
Something simple would be to have your worker threads call back to the main application on completion, you then keep a count of completed threads and in your main UI do something like:
while(runningThreads != 0)
{
Application.DoEvents();
}
and have the threads call:
void OnThreadCompleted()
{
runningThreads--;
}
Better to use BackgroundWorker for this instead of creating your own threads as this has all the callback mechanisms ready to go.
We have used the Background worker for something similar and it worked well, with two observations:
Don't append text to a textbox with += because it will slow you down considerably after a few hundred lines. Use AppendText instead.
If you add a lot of info to the interface and have sleep times (during processing), the thread might 'fall asleep'. We fixed it by deleting the text in the textbox every 200 lines (the results were written to a file, so we didn't lose anything).
One alternative is simply use Invoke on the main thread:
void YourMethod()
{
Thread linkListThread = new Thread(new ThreadStart(getLinkList));
linkListThread.Start();
}
void getLinkList()
{
List<string> linkList = new List<string>();
// Your tasks
// Done
LinkListComplete(linkList);
}
void LinkListComplete(List<string> linkList)
{
if (InvokeRequired)
{
Invoke(new Action<List<string>>(LinkListComplete),linkList);
return;
}
foreach (String link in linkList)
{
txtOutput.Text += link;
}
}