I have a method called asyncStartList, which sends a list of emails provided it, and I'm trying to figure out how to use multiple threads to speed up the process in cases where there are a lot of emails:
public async Task asyncStartList()
{
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
for (int i = 0; i < listLength; i++)
{
currentMailAddress = emailingList[i];
await Task.Run(() => MailingFunction());
currentMailAddress = "";
Console.WriteLine("Your mail to {0} was successfully sent!", emailingList[i]);
}
stopWatch.Stop();
TimeSpan ts = stopWatch.Elapsed;
string elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}",
ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);
Console.WriteLine("Time for completion " + elapsedTime);
Console.ReadLine();
}
The MailingFunction() is a simple SmtpClient and mail message.
Your solution actually not run parallel, because of you wait for every each send operation. You can use paralel foreach/for keyword. Otherwise, you have to wait after all send operation executed.
public async Task asyncStartList()
{
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
// option 1
Task[] tasks = emailingList.Select(s => Task.Run(() => { SendEmail(s); }).ToArray();
Task.WaitAll(tasks);
// option 1 end
// option 2
Parallel.ForEach(emailingList, email =>
{
SendEmail(email);
});
// option 2 end
// option 3
Parallel.For(0, emailingList.Length, i =>
{
SendEmail(emailingList[i]);
});
// option 3 end
stopWatch.Stop();
TimeSpan ts = stopWatch.Elapsed;
string elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10);
Console.WriteLine("Time for completion " + elapsedTime);
Console.ReadLine();
}
private void SendEmail(string emailAddress)
{
// Do send operation
}
Use Parallel.ForEach from the System.Threading.Tasks namespace. So instead for for int i = 0;... use Parallel.ForEach(emailingList, address => {...})
See https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/how-to-write-a-simple-parallel-foreach-loop for an example
If your solution's performance is CPU-bound, that is when you want to use parallel threads. If your solution is bound by something else-- e.g. the ability of the email server to handle requests-- what you actually should use is async, which is much simpler and much safer.
There are many ways to use async in this scenario, but here is a short and simple pattern that would work:
await Task.WhenAll
(
emailingList.Select( async address => MailingFunctionAsync(address) )
);
Yes, that is all there is to it. This assumes that your email client not only has a MailingFunction() method but also a MailingFunctionAsync() method (e.g. using Outlook's SendAsync() method or something similar).
Here is a sample MailingFunctionAsync() stolen from this question:
public async Task MailingFunctionAsync(string toEmailAddress)
{
var message = new MailMessage();
message.To.Add(toEmailAddress);
message.Subject = SOME_SUBJECT;
message.Body = SOME_BODY;
using (var smtpClient = new SmtpClient())
{
await smtpClient.SendMailAsync(message);
}
}
The common answer here is to use Parallel.ForEach (well apart from John Wu's answer that you should really consider). While on-the-outset Parallel.ForEach seems like an easy and good idea, its actually not the most optimal approach.
Here is the problem:
Parallel.ForEach uses the thread pool. Moreover, IO bound operations will block those threads waiting for a device to respond and tie up resources.
If you have CPU bound code, Parallelism is appropriate;
Though if you have IO bound code, Asynchrony is appropriate.
In this case, sending mail is clearly I/O, so the ideal consuming code would be asynchronous.
Furthermore, to use asynchronous and parallel features of the .NET properly, you should also understand the concept of I/O threads.
Not everything in a program consumes CPU time. When a thread tries to read data from a file on disk or sends a TCP/IP packet through network, the only thing it does is delegate the actual work to a device; disk or network adapter; and wait for results.
It’s very expensive to spend a threads time on waiting. Even through threads sleep and don’t consume CPU time while waiting for the results, it doesn’t really pay off because it’s a waste of system resources.
To be simplistic, every thread holds memory for stack variables, local storage and so on. Also, the more threads you have, the more time it takes to switch among them.
Though, the nice thing about Parallel.ForEach is its easy to implement, you can also set up options like Max Degree of Parallelism.
So what can you do...
You are best to use async/await pattern and/or some type of limit on concurrent tasks, another neat solution is to ActionBlock<TInput> Class in the TPL dataflow library.
Dataflow example
var block = new ActionBlock<MySomething>(
mySomething => MyMethodAsync(mySomething),
new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 50 });
foreach (var something in ListOfSomethings)
{
block.Post(something );
}
block.Complete();
await block.Completion;
This approach gives you Asynchrony, it also gives you MaxDegreeOfParallelism, it doesn't waste resources, and lets IO be IO without chewing up unnecessary resources
Disclaimer, DataFlow may not be where you want to be, however I just thought I'd give you some more information on the different
approaches on offer.
Related
This is the code that I wrote to better understand asynchronous methods. I knew that an asynchronous method is not the same as multithreading, but it does not seem so in this particular scenario:
class Program
{
static void Main(string[] args)
{
Thread.CurrentThread.CurrentCulture = new System.Globalization.CultureInfo("en-US");
//the line above just makes sure that the console output uses . to represent doubles instead of ,
ExecuteAsync();
Console.ReadLine();
}
private static async Task ParallelAsyncMethod() //this is the method where async parallel execution is taking place
{
List<Task<string>> tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(Task.Run(() => DownloadWebsite()));
}
var strings = await Task.WhenAll(tasks);
foreach (var str in strings)
{
Console.WriteLine(str);
}
}
private static string DownloadWebsite() //Imitating a website download
{
Thread.Sleep(1500); //making the thread sleep for 1500 miliseconds before returning
return "Download finished";
}
private static async void ExecuteAsync()
{
var watch = Stopwatch.StartNew();
await ParallelAsyncMethod();
watch.Stop();
Console.WriteLine($"It took the machine {watch.ElapsedMilliseconds} milliseconds" +
$" or {Convert.ToDouble(watch.ElapsedMilliseconds) / 1000} seconds to complete this task");
Console.ReadLine();
}
}
//OUTPUT:
/*
Download finished
Download finished
Download finished
Download finished
Download finished
It took the machine 1537 milliseconds or 1.537 seconds to complete this task
*/
As you can see, the DownloadWebsite method waits for 1.5 seconds and then returns "a". The method called ParallelAsyncMethod adds five of these methods into the "tasks" list and then starts the parallel asynchronous execution. As you can see, I also tracked the amount of time that it takes for the ExecuteAsync method to be executed. The result is always somewhere around 1540 milliseconds. Here is my question: if the DownloadWebsite method required a thread to sleep 5 times for 1500 milliseconds, does it mean that the parallel execution of these methods required 5 different threads? If not, then how come it only took the program 1540 milliseconds to be executed and not ~7500 ms?
I knew that an asynchronous method is not the same as multi-threading
That is correct, an asynchronous method releases the current thread whilst I/O occurs, and schedules a continuation after it's completion.
Async and threads are completely unrelated concepts.
but it does not seem so in this particular scenario
That is because you explicitly run DownloadWebsite on the ThreadPool using Task.Run, which imitates asynchronous code by returning a Task after instructing the provided delegate to run.
Because you are not waiting for each Task to complete before starting the next, multiple threads can be used simultaneously.
Currently each thread is being blocked, as you have used Thread.Sleep in the implementation of DownloadWebsite, meaning you are actually running 5 synchronous methods on the ThreadPool.
In production code your DownloadWebsite method should be written asynchronously, maybe using HttpClient.GetAsync:
private static async Task<string> DownloadWebsiteAsync()
{
//...
await httpClinet.GetAsync(//...
//...
}
In that case, GetAsync returns a Task, and releases the current thread whilst waiting for the HTTP response.
You can still run multiple async methods concurrently, but as the thread is released each time, this may well use less than 5 separate threads and may even use a single thread.
Ensure that you dont use Task.Run with an asynchronous method; this simply adds unnecessary overhead:
var tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(DownloadWebsiteAsync()); // No need for Task.Run
}
var strings = await Task.WhenAll(tasks);
As an aside, if you want to imitate an async operation, use Task.Delay instead of Thread.Sleep as the former is non-blocking:
private static async Task<string> DownloadWebsite() //Imitating a website download
{
await Task.Delay(1500); // Release the thread for ~1500ms before continuing
return "Download finished";
}
I've got an ASP.NET site that is running a modest amount of requests (about 500rpm split across 3 servers), and usually the requests take about 15ms. However, I've found that there are frequently requests that take much longer (1s or more). I've narrowed the latency down to a call to Task.WhenAll. Here's an example of the offending code:
var taskA = dbA.GetA(id);
var taskB = dbB.GetB(id);
var taskC = dbC.GetC(id);
var taskD = dbD.GetD(id);
await Task.WhenAll(taskA, taskB, taskC, taskD);
Each individual task is measured and takes less than 10ms to complete. I've pinpointed the delay down to the Task.WhenAll call, and it seems to have something to do with how the task is scheduled. As far as I can tell, there's not a lot of pressure on the TPL task pool, so I'm at a loss for why the performance is so sporadic.
Async operation involve context switches, which are time consuming. Unfortunately, not always in a deterministic way. To speed things up in your case, try to prefix your Task.WhenAll call with ConfigureAwait(false), as follows:
await Task.WhenAll(taskA, taskB, taskC, taskD).ConfigureAwait(false);
This will eliminate an additional context switch, which is actually recommended approach for server-side applications.
Creating threads takes overhead. Depending on what you're doing, you can also try a Parallel.ForEach.
public static void yourMethod(int id){
var tasks = new List<IMyCustomType> { new dbA.GetA(id), new dbB.GetB(id), new dbC.GetC(id), new dbD.GetD(id)};
// Your simple stopwatch for timing
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
// For each 'tasks' list item, call 'executeTasks' (Max 10 occurrences)
// - Processing for all tasks will be complete before
// continuing processing on the main thread
Parallel.ForEach(tasks, new ParallelOptions { MaxDegreeOfParallelism = 10 }, executeTasks);
stopWatch.Stop();
Console.WriteLine("Completed execution in: " + stopWatch.Elapsed.TotalSeconds);
}
private static void executeTasks(string obj)
{
// Your task's work here.
}
I want to scrape a website with plenty of pages with interesting data but as the source is very large I want to multithread and limit the overload.
I use a Parallel.ForEach to start each chunk of 10 tasks and I wait in the main for loop until the numbers of active threads started drop below a threshold. For that I use a counter of active threads I increment when starting a new thread with a WebClient and decrement when the DownloadStringCompleted event of the WebClient is triggered.
Originally the questions was how to use DownloadStringTaskAsync instead of DownloadString and wait that each of the threads started in the Parallel.ForEach has completed. This has been solved with a workaround:
a counter (activeThreads) and a Thread.Sleep in the main foor loop.
Is using await DownloadStringTaskAsync instead of DownloadString supposed to improve at all the speed by freeing a thread while waiting for the DownloadString data to arrive ?
And to get back to the original question, is there a way to do this more elegantly using TPL without the workaround of involving a counter ?
private static volatile int activeThreads = 0;
public static void RecordData()
{
var nbThreads = 10;
var source = db.ListOfUrls; // Thousands urls
var iterations = source.Length / groupSize;
for (int i = 0; i < iterations; i++)
{
var subList = source.Skip(groupSize* i).Take(groupSize);
Parallel.ForEach(subList, (item) => RecordUri(item));
//I want to wait here until process further data to avoid overload
while (activeThreads > 30) Thread.Sleep(100);
}
}
private static async Task RecordUri(Uri uri)
{
using (WebClient wc = new WebClient())
{
Interlocked.Increment(ref activeThreads);
wc.DownloadStringCompleted += (sender, e) => Interlocked.Decrement(ref iterationsCount);
var jsonData = "";
RootObject root;
jsonData = await wc.DownloadStringTaskAsync(uri);
var root = JsonConvert.DeserializeObject<RootObject>(jsonData);
RecordData(root)
}
}
If you want an elegant solution you should use Microsoft's Reactive Framework. It's dead simple:
var source = db.ListOfUrls; // Thousands urls
var query =
from uri in source.ToObservable()
from jsonData in Observable.Using(
() => new WebClient(),
wc => Observable.FromAsync(() => wc.DownloadStringTaskAsync(uri)))
select new { uri, json = JsonConvert.DeserializeObject<RootObject>(jsonData) };
IDisposable subscription =
query.Subscribe(x =>
{
/* Do something with x.uri && x.json */
});
That's the entire code. It's nicely multi-threaded and it's kept under control.
Just NuGet "System.Reactive" to get the bits.
Parallel.ForEach
Will create ProcessorCount tasks to execute the function for each item in the source Enumerable. It will take care that there are not to many tasks and will wait for all items and tasks to be executed.
Task.WhenAll
Only awaits the given tasks it does not execute them. Its on your hand to execute them in a proper way and not to many at once.
But there is some fault in your code. The function RecordUri will return a task that has to be awaited otherwise the ForEach will just create more and more as the function will never know when the current task is completed. Also problematic is that you create a task in a task and the first task does nothing else then wait for the first one.
You might also want to take a look at this overload of Parallel.ForEach
https://msdn.microsoft.com/en-us/library/dd782934(v=vs.110).aspx
Edit
Is using await DownloadStringTaskAsync instead of DownloadString supposed to improve at all the speed by freeing a thread while waiting for the DownloadString data to arrive ?
No. As when a task is awaiting a external resource it enters a Suspended state (Windows api that is not using some old/dirty iteration waiting). So there is no much difference.
What differs is the overhead the compiler will generate when compiling your async code. The DownloadStringTaskAsync will create a task that contains the long operation. If you use await it, you will attach yourself to that task (by ContinueWith). So you just create a Task for awaiting another. This is the overhead i was talking about in the upper text.
My approach would be: Use the synchronous method inside your Parallel.ForEach. The Threadding will be done by PLinq and you are free to go on.
Remember "KISS"
I'm currently trying to loop through a text file that is about 1.5gb's in size and then use the URL's that are grabbed from it to pull down the html from the site.
For speed I'm trying to process all the HTTP request on a new thread but since C# is not my strongest language but a requirement for what I'm doing I'm a bit confused on good thread practice.
This is how I'm processing the list
private static void Main()
{
const Int32 BufferSize = 128;
using (var fileStream = File.OpenRead("dump.txt"))
using (var streamReader = new StreamReader(fileStream, Encoding.UTF8, true, BufferSize))
{
String line;
var progress = 0;
while ((line = streamReader.ReadLine()) != null)
{
var stuff = line.Split('|');
getHTML(stuff[3]);
progress += 1;
Console.WriteLine(progress);
}
}
}
And I'm pulling down the HTML as so
private static void getHTML(String url)
{
new Thread(() =>
{
var client = new DecompressGzipResponse();
var html = client.DownloadString(url);
}).Start();
}
Though the speeds are fast doing this initially, after about 20 thousand they slow down and eventually after 32 thousand the application will hang and crash. I was under the impression C# threads terminated when the function completed?
Can anyone give any examples/ suggestions on how to do this better?
One very reliable way to do this is by using the producer-consumer pattern. You create a thread-safe queue of URLs (for example, BlockingCollection<Uri>). Your main thread is the producer, which adds items to the queue. You then have multiple consumer threads, each of which reads Urls from the queue and does the HTTP requests. See BlockingCollection.
Setting it up isn't terribly difficult:
BlockingCollection<Uri> UrlQueue = new BlockingCollection<Uri>();
// Main thread starts the consumer threads
Task t1 = Task.Factory.StartNew(() => ProcessUrls, TaskCreationOptions.LongRunning);
Task t2 = Task.Factory.StartNew(() => ProcessUrls, TaskCreationOptions.LongRunning);
// create more tasks if you think necessary.
// Now read your file
foreach (var line in File.ReadLines(inputFileName))
{
var theUri = ExtractUriFromLine(line);
UrlQueue.Add(theUri);
}
// when done adding lines to the queue, mark the queue as complete
UrlQueue.CompleteAdding();
// now wait for the tasks to complete.
t1.Wait();
t2.Wait();
// You could also use Task.WaitAll if you have an array of tasks
The individual threads process the urls with this method:
void ProcessUrls()
{
foreach (var uri in UrlQueue.GetConsumingEnumerable())
{
// code here to do a web request on that url
}
}
That's a simple and reliable way to do things, but it's not especially quick. You can do much better by using a second queue of WebCient objects that make asynchronous requests For example, say you want to have 15 asynchronous requests. You start the same way with a BlockingCollection, but you only have one persistent consumer thread.
const int MaxRequests = 15;
BlockingCollection<WebClient> Clients = new BlockingCollection<WebClient>();
// start a single consumer thread
var ProcessingThread = Task.Factory.StartNew(() => ProcessUrls, TaskCreationOptions.LongRunning);
// Create the WebClient objects and add them to the queue
for (var i = 0; i < MaxRequests; ++i)
{
var client = new WebClient();
// Add an event handler for the DownloadDataCompleted event
client.DownloadDataCompleted += DownloadDataCompletedHandler;
// And add this client to the queue
Clients.Add(client);
}
// add the code from above that reads the file and populates the queue
Your processing function is somewhat different:
void ProcessUrls()
{
foreach (var uri in UrlQueue.GetConsumingEnumerable())
{
// Wait for an available client
var client = Clients.Take();
// and make an asynchronous request
client.DownloadDataAsync(uri, client);
}
// When the queue is empty, you need to wait for all of the
// clients to complete their requests.
// You know they're all done when you dequeue all of them.
for (int i = 0; i < MaxRequests; ++i)
{
var client = Clients.Take();
client.Dispose();
}
}
Your DownloadDataCompleted event handler does something with the data that was downloaded, and then adds the WebClient instance back to the queue of clients.
void DownloadDataCompleteHandler(Object sender, DownloadDataCompletedEventArgs e)
{
// The data downloaded is in e.Result
// be sure to check the e.Error and e.Cancelled values to determine if an error occurred
// do something with the data
// And then add the client back to the queue
WebClient client = (WebClient)e.UserState;
Clients.Add(client);
}
This should keep you going with 15 concurrent requests, which is about all you can do without getting a bit more complicated. Your system can likely handle many more concurrent requests, but the way that WebClient starts asynchronous requests requires some synchronous work up front, and that overhead makes 15 about the maximum number you can handle.
You might be able to have multiple threads initiating the asynchronous requests. In that case, you could potentially have as many threads as you have processor cores. So on a quad core machine, you could have the main thread and three consumer threads. With three consumer threads this technique could give you 45 concurrent requests. I'm not certain that it scales that well, but it might be worth a try.
There are ways to have hundreds of concurrent requests, but they're quite a bit more complicated to implement.
You need thread management.
My advice is to use Tasks instead of creating your own Threads.
By using the Task Parallel Library, you let the runtime deal with the thread management. By default, it will allocate your tasks on threads from the ThreadPool, and will allow a level of concurrency which is contingent on the number of CPU cores you have. It will also reuse existing Threads when they become available instead of wasting time creating new ones.
If you want to get more advanced, you can create your own task scheduler to manage the scheduling aspect yourself.
See also What is difference between Task and Thread?
I am experimenting / learning the new Task library and I have written a very simple html downloader using WebClient and Task.Run. However I can never reach anything more than 5% on my network usage. I would like to understand why and how I can improve my code to reach 100% network usage / throughput (probably not possible but it has to be a lot more than 5%).
I would also like to be able to limit the number of thread however it seems it's not as easy as I thought (i.e. custom task scheduler). Is there a way to just do something like this to set the max thread count: something.SetMaxThread(2)?
internal static class Program
{
private static void Main()
{
for (var i = 0; i < 1000000; i++)
{
Go(i, Thread.CurrentThread.ManagedThreadId);
}
Console.Read();
}
private static readonly Action<int, int> Go = (counter, threadId) => Task.Run(() =>
{
var stopwatch = new Stopwatch();
stopwatch.Start();
var webClient = new WebClient();
webClient.DownloadString(new Uri("http://stackoverflow.com"));
stopwatch.Stop();
Console.Write("{0} == {1} | ", threadId.ToString("D3"), Thread.CurrentThread.ManagedThreadId.ToString("D3"));
Console.WriteLine("{0}: {1}ms ", counter.ToString("D3"), stopwatch.ElapsedMilliseconds.ToString("D4"));
});
}
This is the async version according to #spender. However my understanding is that await will "remember" the point in time and hand off the download to OS level and skip (the 2 console.write) and return to main immediately and continue scheduling the remaining Go method in the for loop. Am I understanding it correctly? So there's no blocking on the UI.
private static async void Go(int counter, int threadId)
{
using (var webClient = new WebClient())
{
var stopWatch = new Stopwatch();
stopWatch.Start();
await webClient.DownloadStringTaskAsync(new Uri("http://ftp.iinet.net.au/test500MB.dat"));
stopWatch.Stop();
Console.Write("{0} == {1} | ", threadId.ToString("D3"), Thread.CurrentThread.ManagedThreadId.ToString("D3"));
Console.WriteLine("{0}: {1}ms ", counter.ToString("D3"), stopWatch.ElapsedMilliseconds.ToString("D4"));
}
}
What I noticed was that when I am downloading large files there's no that much difference in terms of download speed / network usage. They (threading version and the async version) both peaked at about 12.5% network usage and about 12MByte download /sec. I also tried to run multiple instances (multiple .exe running) and again there's no huge difference between the two. And when I am trying to download large files from 2 URLs concurrently (20 instances) I get similar network usage (12.5%) and download speed (10-12MByte /sec). I guess I am reaching the peak?
As it stands, your code is suboptimal because, although you are using Task.Run to create asynchronous code that runs in the ThreadPool, the code that is being run in the ThreadPool is still blocking on the line:
webClient.DownloadString(...
This amounts to an abuse of the ThreadPool because it is not designed to run blocking tasks, and is slow to spin up additional threads to deal with peaks in workload. This in turn will have a seriously degrading effect on the smooth running of any API that uses the ThreadPool (timers, async callbacks, they're everywhere), because they'll schedule work that goes to the back of the (saturated) queue for the ThreadPool (which is tied up reluctantly and slowly spinning up hundreds of threads that are going to spend 99.9% of their time doing nothing).
Stop blocking the ThreadPool and switch to proper async methods that do not block.
So now you can literally break your router and seriously upset the SO site admins with the following simple mod:
private static void Main()
{
for (var i = 0; i < 1000000; i++)
{
Go(i, Thread.CurrentThread.ManagedThreadId);
}
Console.Read();
}
private static async Task Go(int counter, int threadId)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
using (var webClient = new WebClient())
{
await webClient.DownloadStringTaskAsync(
new Uri("http://stackoverflow.com"));
}
//...
}
HttpWebRequest (and therefore WebClient) are also constrained by a number of limits.