I have a function that connects to a Multiple SQL server instances to fetch a set of data from each server for comparison of the data among multiple environments
I have a collection of the connection strings
I am calling this method in a foreach loop for each connection string in the collection
Since the data is fetched from different servers individually one at a time it takes up a lot of time
I would like to know if i use threads to call this method every time what would be the best way to do it?
There are a couple of ways to do this
1.) create a set of tasks and then do "await Task.WaitAll(listOfTasks)"
2.) Use Parallel.ForEach
3.) Manage threads
Managing threads
I do this in 2 steps:
1.) Create a list of threads
List<Thread> threads = new List<Thread>();
foreach(var connectionString in ConnectionStrings)
{
var thread = new Thread(DoWork);
thread.Start(connectionString);
threads.Add(thread);
}
2.) Join the threads to the current thread, has the effect of blocking until all are complete.
foreach(var thread in threads)
{
thread.Join()
}
You could Join the threads and make the program wait all of them until they are finished. It is a good pratice before moving to the next step. Look the code bellow with the comment, for sample:
// list of threads
List<Thread> threads = new List<Thread>();
// list of the results for each thread
List<SomeTypeDto> results = new List<SomeTypeDto>();
foreach(var connectionString in connectionStringList) {
// create the thread
Thread thread = new Thread(() => {
// access the database
SomeTypeDto result = /* process some result data*/;
lock (results) {
results.Add(result);
}
});
threads.Add(thread);
}
// start all threads
foreach (Thread thread in threads) {
thread.Start();
}
// Wait for all the threads to finish
foreach (Thread thread in threads) {
thread.Join();
}
Related
I wanted to know how to reuse a thread. I have a websocket connection that constantly sends messages which need some computation to be done. I want to add this computation to a thread, but don't want to create a new thread everytime. How can I make it so that the thread is reused?
client.MsgRecieved.Subscribe(info =>
{
Thread t = new Thread(() => Do_work(info));
};
Is there a way I can create a thread, name it and then just add Do_work() on that thread?
Edit:
I get multiple messages from the websocket per second. I rather have them wait in a single queue, rather than all run on a new thread.
The simplest pattern is simply
client.MsgRecieved.Subscribe(async info =>{
await Task.Run(Do_work(info));
});
Which queues the method to run on the built-in threadpool.
If you want to queue the messages to run on a single background thread, you can use a BlockingCollection, something like:
var workQueue = new System.Collections.Concurrent.BlockingCollection<Object>();
var workThread = new System.Threading.Thread(() =>
{
foreach (var work in workQueue.GetConsumingEnumerable())
{
Do_Work(work);
}
});
workThread.Start();
then
client.MsgRecieved.Subscribe(info => {
workQueue.Add(info);
};
I am working on windows service.
I am need to thread 380 methods in a program.
Here is my code.
threadChunkList is a list of 20 threads and there is an array of thread in the list.
like this.
List<Thread[]> threadChunkList = new List<Thread[]>();
When it comes to following loop to execute method, the CPU usage in the task manager goes to over 90%.
foreach (Thread[] mainthread in threadChunkList)
{
Thread OneFinalThread = null;
OneFinalThread = new Thread(() =>
{
foreach (Thread chunkthread in mainthread)
{
chunkthread.Start();
}
foreach (var thread in mainthread)
{
thread.Join();
}
});
OneFinalThread.Priority = ThreadPriority.Lowest;
OneFinalThread.Start();
for (; ; )
{
if (OneFinalThread.IsAlive)
{
}
else
{
break;
}
}
}
Can anyone please let me know what is going wrong with above code. Also I dont know its proper way to manage thread or now. How can I use sleep method for thread to reduce CPU usage and how to remove previous thread from memory to clear memory?
Thank you
Can anybody tell me how to invoke all threads at once from a list of threads.
actually I have saved all threads in a list after creating them but if I use th.start() and th.join() methods, every thread starts one by one. I want them to start all at once.
I've used time function to see the starting and ending time for each thread but non of them starts simultaneously.
I am starting thread like this:
foreach (Thread st in threadss)
{
st.Start();
st.Join();
}
but I want to start them like this:
Parallel.Invoke(() => st1.Start(), () => st2.Start(), .....);
please help
Why not .Start() them first, then .Join() them? Should work fine.
foreach (var st in threadss) {
st.Start();
}
foreach (var st in threadss) {
st.Join();
}
I have a .NET 4.0 ASP.NET project which requires some threading work I've never really messed with before and I've been looking at this for days and I'm still clueless =/
Basically I want something like when you take a ticket at the deli and wait your turn before they get back to you. I'll try and relate this and see if it makes any sense...
function starts ---> gets to section where it needs to "take a ticket" (I assume queue some type of item in a blockingcollection) and waits until other "tickets" (a.k.a other instances of the same function) are completed before it gives the function the OK to resume (blocking collection gets to the item in the queue) ---> finish function.
I don't need/want to do any work in the queue, I just want the function to statically wait it's turn among other instances of the function. Does that make sense? Is that possible?
Please provide code if possible as I've seen tons of examples but none of them make sense/don't do what I want.
If you want to have the timer solution, I'd enqueue all operations into a BlockingCollection and have a dedicated thread dequeue them. This thread would wait 5s and then push the dequeued item onto the thread pool. This dedicated thread should do this in an infinite loop. Dequeue, wait, push.
What I actually recommend however, is that you use the SemaphoreSlim class to throttle the number of concurrent requests to this fragile web service. Probably you should pick a number between 1 and 5 or so as the allowed amount of concurrency.
Alright so after researching document after document and playing with numerous rewrites of code I finally figured out I wasn't using the AutoResetEvent right and how to use a blocking collection on a dedicated thread. So here was the final solution using an AutoResetEvent with a BlockingCollection. This solution below might not show the same results 100% of the time (just because I believe it has to do with thread timing of when something was entered into the blocking collection) but the end result is that it does exactly what I want.
class Program
{
static void Main(string[] args)
{
TaskProcessor tp = new TaskProcessor();
Thread t1 = new Thread(new ParameterizedThreadStart(tp.SubmitRequest));
t1.Start(1);
Thread t2 = new Thread(new ParameterizedThreadStart(tp.SubmitRequest));
t2.Start(2);
Thread t3 = new Thread(new ParameterizedThreadStart(tp.SubmitRequest));
t3.Start(3);
}
}
class TaskProcessor
{
private AutoResetEvent _Ticket;
public TaskProcessor()
{
_Continue = new AutoResetEvent(false);
}
public void SubmitRequest(object i)
{
TicketingQueue dt = new TicketingQueue();
Console.WriteLine("Grab ticket for customer {0}", (int)i);
dt.GrabTicket(_Ticket);
_Continue.WaitOne();
Console.WriteLine("Customer {0}'s turn", (int)i);
}
}
public class TicketingQueue
{
private static BlockingCollection<AutoResetEvent> tickets = new BlockingCollection<AutoResetEvent>();
static TicketingQueue()
{
var thread = new Thread(
() =>
{
while (true)
{
AutoResetEvent e = tickets.Take();
e.Set();
Thread.Sleep(1000);
}
});
thread.Start();
}
public void GrabTicket(AutoResetEvent e)
{
tickets.Add(e);
}
}
I work with new Parallel.For that creates multiple threads to perform the same operation.
In case one of the threads fail, it means that I'm working "too fast" and I need to put all the threads to rest for a few seconds.
Is there a way to perform something like Thread.Sleep - only to do the same on all threads at once?
This is a direct answer to the question, except for the Parallel.For bit.
It really is a horrible pattern; you should probably be using a proper synchronization mechanism, and get the worker threads to, without preemption, occasionally check if they need to 'back off.'
In addition, this uses Thread.Suspend and Thread.Resume which are both deprecated, and with good reason (from Thread.Suspend):
"Do not use the Suspend and Resume methods to synchronize the activities of threads. You have no way of knowing what code a thread is executing when you suspend it. If you suspend a thread while it holds locks during a security permission evaluation, other threads in the AppDomain might be blocked. If you suspend a thread while it is executing a class constructor, other threads in the AppDomain that attempt to use that class are blocked. Deadlocks can occur very easily."
(Untested)
public class Worker
{
private readonly Thread[] _threads;
private readonly object _locker = new object();
private readonly TimeSpan _tooFastSuspensionSpan;
private DateTime _lastSuspensionTime;
public Worker(int numThreads, TimeSpan tooFastSuspensionSpan)
{
_tooFastSuspensionSpan = tooFastSuspensionSpan;
_threads = Enumerable.Repeat(new ThreadStart(DoWork), numThreads)
.Select(ts => new Thread(ts))
.ToArray();
}
public void Run()
{
foreach (var thread in _threads)
{
thread.Start();
}
}
private void DoWork()
{
while (!IsWorkComplete())
{
try
{
// Do work here
}
catch (TooFastException)
{
SuspendAll();
}
}
}
private void SuspendAll()
{
lock (_locker)
{
// We don't want N near-simultaneous failures causing a sleep-duration of N * _tooFastSuspensionSpan
// 1 second is arbitrary. We can't be deterministic about it since we are forcefully suspending threads
var now = DateTime.Now;
if (now.Subtract(_lastSuspensionTime) < _tooFastSuspensionSpan + TimeSpan.FromSeconds(1))
return;
_lastSuspensionTime = now;
var otherThreads = _threads.Where(t => t.ManagedThreadId != Thread.CurrentThread.ManagedThreadId).ToArray();
foreach (var otherThread in otherThreads)
otherThread.Suspend();
Thread.Sleep(_tooFastSuspensionSpan);
foreach (var otherThread in otherThreads)
otherThread.Resume();
}
}
}
You need to create an inventory of your worker threads and then perhaps you can use Thread.Suspend and Resume methods. Mind you that using Suspend can be dangerous (for example, thread may have acquired lock before suspending). And suspend/resume have been marked obsolate due to such issues.