I have an app that takes on unknown amount of task. The task are blocking (they wait on network) i'll need multiple threads to keep busy.
Is there an easy way for me to have a giant list of task and worker threads which will pull the task when they are idle? ATM i just start a new thread for each task, which is fine but i'd like some control so if there are 100task i dont have 100threads.
Assuming that the network I/O classes that you are dealing with expose Begin/End style async methods, then what you want to do is use the TPL TaskFactory.FromAsync method. As laid out in TPL TaskFactory.FromAsync vs Tasks with blocking methods, the FromAsync method will use async I/O under the covers, rather than keeping a thread busy just waiting for the I/O to complete (which is actually not what you want).
The way that Async I/O works is that you have a pool of threads that can handle the result of I/O when the result is ready, so that if you have 100 outstanding I/Os you don't have 100 threads blocked waiting for those I/Os. When the whole pool is busy handling I/O results, subsequent results get queued up automatically until a thread frees up to handle them. Keeping a huge pool of threads waiting like that is a scalability disaster- threads are hugely expensive objects to keep around idling.
here a msdn sample to manage through a threadpool many threads:
using System;
using System.Threading;
public class Fibonacci
{
public Fibonacci(int n, ManualResetEvent doneEvent)
{
_n = n;
_doneEvent = doneEvent;
}
// Wrapper method for use with thread pool.
public void ThreadPoolCallback(Object threadContext)
{
int threadIndex = (int)threadContext;
Console.WriteLine("thread {0} started...", threadIndex);
_fibOfN = Calculate(_n);
Console.WriteLine("thread {0} result calculated...", threadIndex);
_doneEvent.Set();
}
// Recursive method that calculates the Nth Fibonacci number.
public int Calculate(int n)
{
if (n <= 1)
{
return n;
}
return Calculate(n - 1) + Calculate(n - 2);
}
public int N { get { return _n; } }
private int _n;
public int FibOfN { get { return _fibOfN; } }
private int _fibOfN;
private ManualResetEvent _doneEvent;
}
public class ThreadPoolExample
{
static void Main()
{
const int FibonacciCalculations = 10;
// One event is used for each Fibonacci object
ManualResetEvent[] doneEvents = new ManualResetEvent[FibonacciCalculations];
Fibonacci[] fibArray = new Fibonacci[FibonacciCalculations];
Random r = new Random();
// Configure and launch threads using ThreadPool:
Console.WriteLine("launching {0} tasks...", FibonacciCalculations);
for (int i = 0; i < FibonacciCalculations; i++)
{
doneEvents[i] = new ManualResetEvent(false);
Fibonacci f = new Fibonacci(r.Next(20,40), doneEvents[i]);
fibArray[i] = f;
ThreadPool.QueueUserWorkItem(f.ThreadPoolCallback, i);
}
// Wait for all threads in pool to calculation...
WaitHandle.WaitAll(doneEvents);
Console.WriteLine("All calculations are complete.");
// Display the results...
for (int i= 0; i<FibonacciCalculations; i++)
{
Fibonacci f = fibArray[i];
Console.WriteLine("Fibonacci({0}) = {1}", f.N, f.FibOfN);
}
}
}
Related
I have the following code:
public const int ThreadLimitMax = 128;
private static object setThreadLimitLock = new object();
private static SemaphoreSlim totalThreadLimiter = new SemaphoreSlim(ThreadLimit, ThreadLimitMax);
public static int ThreadLimit { get; private set; } = 128;
public static async Task SetThreadLimit(int max)
{
if (max > ThreadLimitMax)
throw new ArgumentOutOfRangeException(nameof(max), $"Cannot have more than {ThreadLimitMax} threads.");
if (max < 1)
throw new ArgumentOutOfRangeException(nameof(max), $"Cannot have less than 1 threads.");
lock (setThreadLimitLock)
{
int difference = Math.Abs(ThreadLimit - max);
if (max < ThreadLimit)
{
for (int i = 0; i < difference; i++)
{
await totalThreadLimiter.WaitAsync().ConfigureAwait(false);
}
}
else if (max > ThreadLimit)
{
totalThreadLimiter.Release(difference);
}
ThreadLimit = max;
}
}
I am trying to make a method that will modify the amount of available threads in totalThreadLimiter. I keep the maximum number of threads in the ThreadMaxLimit integer.
To change the amount of threads, I need to ensure the ThreadLimit is not accessed until the operation of changing max threads is complete. I also need to ensure that the method is blocked until the totalThreadLimiter has completed with all WaitAsync() calls.
How can I do that?
lock is a helper API around Monitor, which is a thread-bound synchronization primitive, which means it isn't suitable for use with await, because there is no guarantee what thread you'll be on when you come back from an incomplete asynchronous operation.
Ultimately, you need an async-aware synchronization primitive; the most readily available would be SemaphoreSlim, which has the WaitAsync() API that you would use to acquire the lock, with a try/finally that calls Release().
In the code in the question, depending on the code branch you either acquire (only) the semaphore, or release the semaphore; that is almost certainly wrong. Correct usage would be something more like:
await totalThreadLimiter.WaitAsync();
try
{
// some code with "await" here
}
finally
{
totalThreadLimiter.Release();
}
I've made a clean project of .net Core 3.0 Web application and I am trying to understand, how ThreadPool works in C#.
namespace TestASPSelf.Controllers
{
public class HomeController : Controller
{
private readonly ILogger<HomeController> _logger;
public static int countThread = 0;
public HomeController(ILogger<HomeController> logger)
{
_logger = logger;
}
public IActionResult Index()
{
int workerThreads;
int portThreads;
ThreadPool.GetMaxThreads(out workerThreads, out portThreads);
Console.WriteLine("\nMaximum worker threads: \t{0}" +
"\nMaximum completion port threads: {1}",
workerThreads, portThreads);
ThreadPool.GetAvailableThreads(out workerThreads,
out portThreads);
Console.WriteLine("\nAvailable worker threads: \t{0}" +
"\nAvailable completion port threads: {1}\n",
workerThreads, portThreads);
Console.WriteLine("countThread = " + countThread);
return View();
}
class Z
{
public static void WaitTest(object o)
{
countThread++;
while (true)
{
Thread.Sleep(1000);
}
}
}
public IActionResult Privacy()
{
for (int i = 0; i < 100; i++)
{
Console.WriteLine("starting thread "+i);
ThreadPool.QueueUserWorkItem(new WaitCallback(Z.WaitTest));
}
return View();
}
}
}
When http://localhost:5000/Home/Privacy opened, it hangs for some time (for about 40-80 seconds), but I see, that logic of for cycles in it completes almost instantly.
When http://localhost:5000/ opened after that, it hangs for 40-80 seconds too and result is in console countThread = 100.
CPU usage of app is about 5-10%, when threads were started.
I am trying to understand:
1) The first one is why ASP controller hangs for 40-80 seconds per page, when 100 Threads are running by CPU usage 5-10 percents. CPU has a lot of resources, RAM is free too, but why ASP Controller methods of pages are hangs?
2) How to create ThreadPool in C# with limited count of running threads? If I understand method public static bool SetMinThreads (int workerThreads, int completionPortThreads); correctly, it affects globally all threads of the app. How to create object of ThreadPool with limited count of active threads, like ExecutorService in Java? For example, Java code of thread pool could look like
ExecutorService executor = Executors.newFixedThreadPool(5);
for (int i = 0; i < 10; i++) {
Runnable worker = new WorkerThread("" + i);
executor.execute(worker);
}
executor.shutdown();
while (!executor.isTerminated()) {
}
3) How to prevent hangs of all methods of ASP Controller and to make "truly real" threads, like in Java?
ThreadPool.QueueUserWorkItem(new WaitCallback(Z.WaitTest));
With this you do something very wrong. You cause threads in the threadpool to block and thus the pool is unable to finish processing your request and process new requests.
At some point a thread from the pool manages to return and process your next request, but than it hangs again due to the overloaded pool.
As for your other questions, please explain what you want to achieve. Your question seems to try to solve a problem that is not understood well.
Update: After Arthur's comment.
If you will be downloading files, you should use Tasks and async-await. IO operations do not consume threads (more here).
Create N tasks, each downloading a file, and then await Task.WhenAll. Pseudo code:
List<Task> tasks = new List<Task>();
for (int i = 0; i < filesToDownloadCount; i++)
{
var t = new Task ( () => { /* ... code to download your file here ... */});
tasks.Add (t);
}
await t.WhenAll (tasks);
This approach will give you the best throughput and your bottleneck will be your bandwidth, not the CPU.
The ThreadPool class has several static methods including the QueueUserWorkItem that is responsible for calling a thread pool worker thread when it is available. If no worker thread is available in the thread pool, it waits until the thread becomes available.
using System;
using System.Threading;
class ThreadPoolSample
{
// Background task
static void BackgroundTask(Object stateInfo)
{
Console.WriteLine("Hello! I'm a worker from ThreadPool");
Thread.Sleep(1000);
}
static void BackgroundTaskWithObject(Object stateInfo)
{
Person data = (Person)stateInfo;
Console.WriteLine($"Hi {data.Name} from ThreadPool.");
Thread.Sleep(1000);
}
static void Main(string[] args)
{
// Use ThreadPool for a worker thread
ThreadPool.QueueUserWorkItem(BackgroundTask);
Console.WriteLine("Main thread does some work, then sleeps.");
Thread.Sleep(500);
// Create an object and pass it to ThreadPool worker thread
Person p = new Person("Mahesh Chand", 40, "Male");
ThreadPool.QueueUserWorkItem(BackgroundTaskWithObject, p);
int workers, ports;
// Get maximum number of threads
ThreadPool.GetMaxThreads(out workers, out ports);
Console.WriteLine($"Maximum worker threads: {workers} ");
Console.WriteLine($"Maximum completion port threads: {ports}");
// Get available threads
ThreadPool.GetAvailableThreads(out workers, out ports);
Console.WriteLine($"Availalbe worker threads: {workers} ");
Console.WriteLine($"Available completion port threads: {ports}");
// Set minimum threads
int minWorker, minIOC;
ThreadPool.GetMinThreads(out minWorker, out minIOC);
ThreadPool.SetMinThreads(4, minIOC);
// Get total number of processes availalbe on the machine
int processCount = Environment.ProcessorCount;
Console.WriteLine($"No. of processes available on the system: {processCount}");
// Get minimum number of threads
ThreadPool.GetMinThreads(out workers, out ports);
Console.WriteLine($"Minimum worker threads: {workers} ");
Console.WriteLine($"Minimum completion port threads: {ports}");
Console.ReadKey();
}
// Create a Person class
public class Person
{
public string Name { get; set; }
public int Age { get; set; }
public string Sex { get; set; }
public Person(string name, int age, string sex)
{
this.Name = name;
this.Age = age;
this.Sex = sex;
}
}
}
I need to call a worker method multiple times to load data into a database. I want to do some parallelism with this and be able to specify the number of threads to use. I thought of using the mod operator to split the workload, but getting stuck on how to implement with async await.
So the async method must create n number of threads and then call the worker method so there are n streams of work happening in parallel. The worker method is synchronous.
I had a go at it, but quite sure how to implement what I want. Is there a pattern for this?
Some code I was playing around with:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace TestingAsync
{
class Program
{
static void Main(string[] args)
{
int threads = 3;
int numItems = 10;
Task task = ThreadIt(threads, numItems);
}
static async Task ThreadIt(int threads, int numItems)
{
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
for (int i = 0; i < numItems; i++)
{
Console.Write($"Adding item: {i} mod 1: {i % threads}. ");
int task = await DoSomeWork(i%threads, 500);
}
}
static async Task<int> DoSomeWork(int Item, int time)
{
Console.Write($" Working.on item {Item}..");
Thread.Sleep(time );
Console.WriteLine($"done.");
return Item;
}
}
}
EDIT:
I'm going to rephrase because maybe I wasn't clear in my requirements.
What I want is to create n number of threads. There will be x number of items to process and I want them to be queued up using mod (or something else) and then processed in order in parallel across the n threads. When one item has finished, I want the next item to be processed immediately and not wait for all three threads to finish. Some items will take longer to process than others, maybe even up to 10 times longer, so other threads should not be waiting for one of the threads to complete.
For example if we have 3 threads and 9 items, this would happen:
thread1: items 0,3,6
thread2: items 1,4,7
thread3: items 2,5,8
each thread processes it's workload in order and does not wait in between each item.
You can try creating a List<Task<T>> and start them and then await it with WhenAll if you want all tasks to be completed or WhenAny if any of them completes:
static async Task ThreadIt(int threads, int numItems)
{
List<Task<int>> tasks = new List<Task<int>>();
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
for (int i = 0; i < numItems; i++)
{
Console.Write($"Adding item: {i} mod 1: {i % threads}. ");
tasks.Add(DoSomeWork(i%threads, 500));
}
var result = await Task.WhenAll(tasks);
}
and when using Task, async and await we should be using Task.Delay instead of Thread.Sleep:
static async Task<int> DoSomeWork(int Item, int time)
{
Console.Write($" Working.on item {Item}..");
await Task.Delay(time); // note this
Console.WriteLine($"done.");
return Item;
}
EDIT:
You can create a ConcurrentQueue and then dequeue each time when 3 Tasks complete and generate next 3 like:
static async Task ThreadIt(int threads, int numItems)
{
ConcurrentQueue<int> queue = new ConcurrentQueue<int>();
Enumerable.Range(0, 10).ForEach(x => queue.Enqueue(x));
List<Task<int>> tasks = new List<Task<int>>();
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
while (!queue.IsEmpty)
{
for (int i = 0; i < threads; i++)
{
if(queue.TryDequeue(out int val))
{
Console.Write($"Adding item: {val} mod 1: {val % threads}. ");
tasks.Add(DoSomeWork(val%threads, 500));
}
}
var result = await Task.WhenAll(tasks);
}
}
I need to call a worker method multiple times to load data into a database. I want to do some parallelism with this and be able to specify the number of threads to use... The worker method is synchronous... Is there a pattern for this?
Yes, the Task Parallel Library.
Given:
static int DoSomeWork(int Item, int time)
{
Console.Write($" Working.on item {Item}..");
Thread.Sleep(time);
Console.WriteLine($"done.");
return Item;
}
You can parallelize it as such:
static List<int> ThreadIt(int threads, int numItems)
{
Console.WriteLine($"Item limit: {numItems}, threads: {threads}");
var items = Enumerable.Range(0, numItems);
return items.AsParallel().WithDegreeOfParallelism(threads)
.Select(i => DoSomeWork(i, 500))
.ToList();
}
Inspired by The Little Book of Semaphores, I decided to implement the Producer-Consumer problem using Semaphores.
I specifically want to be able to stop all Worker threads at will.
I've tested my methodolodgy extensively and can't find anything faulty.
Following code is a prototype for testing and can be ran as a Console application:
using System;
using System.Collections.Concurrent;
using System.Threading;
using NUnit.Framework;
public class ProducerConsumer
{
private static readonly int _numThreads = 5;
private static readonly int _numItemsEnqueued = 10;
private static readonly Semaphore _workItems = new Semaphore(0, int.MaxValue);
private static readonly ManualResetEvent _stop = new ManualResetEvent(false);
private static ConcurrentQueue<int> _queue;
public static void Main()
{
_queue = new ConcurrentQueue<int>();
// Create and start threads.
for (int i = 1; i <= _numThreads; i++)
{
Thread t = new Thread(new ParameterizedThreadStart(Worker));
// Start the thread, passing the number.
t.Start(i);
}
// Wait for half a second, to allow all the
// threads to start and to block on the semaphore.
Thread.Sleep(500);
Console.WriteLine(string.Format("Main thread adds {0} items to the queue and calls Release() {0} times.", _numItemsEnqueued));
for (int i = 1; i <= _numItemsEnqueued; i++)
{
Console.WriteLine("Waking up a worker thread.");
_queue.Enqueue(i);
_workItems.Release(); //wake up 1 worker
Thread.Sleep(2000); //sleep 2 sec so it's clear the threads get unblocked 1 by 1
}
// sleep for 5 seconds to allow threads to exit
Thread.Sleep(5000);
Assert.True(_queue.Count == 0);
Console.WriteLine("Main thread stops all threads.");
_stop.Set();
// wait a while to exit
Thread.Sleep(5000);
Console.WriteLine("Main thread exits.");
Console.WriteLine(string.Format("Last value of Semaphore was {0}.", _workItems.Release()));
Assert.True(_queue.Count == 0);
Console.WriteLine("Press Enter to exit.");
Console.ReadLine();
}
private static void Worker(object num)
{
// Each worker thread begins by requesting the semaphore.
Console.WriteLine("Thread {0} begins and waits for the semaphore.", num);
WaitHandle[] wait = { _workItems, _stop };
int signal;
while (0 == (signal = WaitHandle.WaitAny(wait)))
{
Console.WriteLine("Thread {0} becomes unblocked by Release() and has work to do.", num);
int res;
if (_queue.TryDequeue(out res))
{
Console.WriteLine("Thread {0} dequeues {1}.", num, res);
}
else
{
throw new Exception("this should not happen.");
}
}
if (signal == 1)
Console.WriteLine("Thread {0} was stopped.", num);
Console.WriteLine("Thread {0} exits.", num);
}
}
Now for my question, I'm using WaitHandle.WaitAny(semaphore) under the assumption that when I call Release() on the semaphore, only 1 Worker will be woken up. However, I can't find reassurance in the documentation that this is actually true. Can anyone confirm this is true?
It is indeed interesting that it seems that the documentation doesn't state explicitly that in the case of WaitOne only 1 thread will receive the signal. When you get familiar with multithreading theory this becomes somewhat self-evident.
Yes, WaitOne that is called on Semaphore (and WaitAny that is called on a list of WaitHandles that include Semaphore) is received by a single thread. If you want reference from MSDN so here it is, Semaphore is child class of WaitHandle, which is:
Encapsulates operating system–specific objects that wait for exclusive access to shared resources.
So yes, unless explicitly stated methods provide exclusive access.
For example method WaitOne of ManualResetEvent will unblock for all waiting threads, but documentation is explicit about it:
Notifies one or more waiting threads that an event has occurred.
Even if I am using TPL from long time but since it sounds new to me. I want to understand the TPL with thread pool and I created a POC in .NET framework 4.0 for that which is as below.
public class CustomData
{
public long CreationTime;
public int Name;
public int ThreadNum;
}
public class TPLSample
{
public int MaxThread = 0;
public void Start()
{
Task[] taskArray = new Task[10000];
for (int i = 0; i < taskArray.Length; i++)
{
taskArray[i] = Task.Factory.StartNew((Object obj) =>
{
var data = new CustomData() { Name = i, CreationTime = DateTime.Now.Ticks };
Thread.SpinWait(10000);
data.ThreadNum = Thread.CurrentThread.ManagedThreadId;
if (Thread.CurrentThread.ManagedThreadId > MaxThread)
{
MaxThread = Thread.CurrentThread.ManagedThreadId;
}
Console.WriteLine("Task #{0} created at {1} on thread #{2}.",
data.Name, data.CreationTime, data.ThreadNum);
},
i);
}
Task.WaitAll(taskArray);
Console.WriteLine("Max no of threads {0}", MaxThread);
}
}
I found that only 14 threads are created to do this task!!
But why the 14? what is the criteria ? can I increase or decrease this number?
How can I change this number. Is it really possible or totally abstracted from a developer.
From MSDN:
The number of operations that can be queued to the thread pool is limited only by available memory; however, the thread pool limits the number of threads that can be active in the process simultaneously. Beginning with the .NET Framework 4, the default size of the thread pool for a process depends on several factors, such as the size of the virtual address space. A process can call the GetMaxThreads method to determine the number of threads.
Another MSDN:
The TPL may employ various optimizations, especially with large numbers of delegates.
Another SO question about this. Hopefully this will quench your thirst.