Related
I've been doing some investigation to see how we can create a multithreaded application that runs through a tree.
To find how this can be implemented in the best way I've created a test application that runs through my C:\ disk and opens all directories.
class Program
{
static void Main(string[] args)
{
//var startDirectory = #"C:\The folder\RecursiveFolder";
var startDirectory = #"C:\";
var w = Stopwatch.StartNew();
ThisIsARecursiveFunction(startDirectory);
Console.WriteLine("Elapsed seconds: " + w.Elapsed.TotalSeconds);
Console.ReadKey();
}
public static void ThisIsARecursiveFunction(String currentDirectory)
{
var lastBit = Path.GetFileName(currentDirectory);
var depth = currentDirectory.Count(t => t == '\\');
//Console.WriteLine(depth + ": " + currentDirectory);
try
{
var children = Directory.GetDirectories(currentDirectory);
//Edit this mode to switch what way of parallelization it should use
int mode = 3;
switch (mode)
{
case 1:
foreach (var child in children)
{
ThisIsARecursiveFunction(child);
}
break;
case 2:
children.AsParallel().ForAll(t =>
{
ThisIsARecursiveFunction(t);
});
break;
case 3:
Parallel.ForEach(children, t =>
{
ThisIsARecursiveFunction(t);
});
break;
default:
break;
}
}
catch (Exception eee)
{
//Exception might occur for directories that can't be accessed.
}
}
}
What I have encountered however is that when running this in mode 3 (Parallel.ForEach) the code completes in around 2.5 seconds (yes I have an SSD ;) ). Running the code without parallelization it completes in around 8 seconds. And running the code in mode 2 (AsParalle.ForAll()) it takes a near infinite amount of time.
When checking in process explorer I also encounter a few strange facts:
Mode1 (No Parallelization):
Cpu: ~25%
Threads: 3
Time to complete: ~8 seconds
Mode2 (AsParallel().ForAll()):
Cpu: ~0%
Threads: Increasing by one per second (I find this strange since it seems to be waiting on the other threads to complete or a second timeout.)
Time to complete: 1 second per node so about 3 days???
Mode3 (Parallel.ForEach()):
Cpu: 100%
Threads: At most 29-30
Time to complete: ~2.5 seconds
What I find especially strange is that Parallel.ForEach seems to ignore any parent threads/tasks that are still running while AsParallel().ForAll() seems to wait for the previous Task to either complete (which won't soon since all parent Tasks are still waiting on their child tasks to complete).
Also what I read on MSDN was: "Prefer ForAll to ForEach When It Is Possible"
Source: http://msdn.microsoft.com/en-us/library/dd997403(v=vs.110).aspx
Does anyone have a clue why this could be?
Edit 1:
As requested by Matthew Watson I've first loaded the tree in memory before looping through it. Now the loading of the tree is done sequentially.
The results however are the same. Unparallelized and Parallel.ForEach now complete the whole tree in about 0.05 seconds while AsParallel().ForAll still only goes around 1 step per second.
Code:
class Program
{
private static DirWithSubDirs RootDir;
static void Main(string[] args)
{
//var startDirectory = #"C:\The folder\RecursiveFolder";
var startDirectory = #"C:\";
Console.WriteLine("Loading file system into memory...");
RootDir = new DirWithSubDirs(startDirectory);
Console.WriteLine("Done");
var w = Stopwatch.StartNew();
ThisIsARecursiveFunctionInMemory(RootDir);
Console.WriteLine("Elapsed seconds: " + w.Elapsed.TotalSeconds);
Console.ReadKey();
}
public static void ThisIsARecursiveFunctionInMemory(DirWithSubDirs currentDirectory)
{
var depth = currentDirectory.Path.Count(t => t == '\\');
Console.WriteLine(depth + ": " + currentDirectory.Path);
var children = currentDirectory.SubDirs;
//Edit this mode to switch what way of parallelization it should use
int mode = 2;
switch (mode)
{
case 1:
foreach (var child in children)
{
ThisIsARecursiveFunctionInMemory(child);
}
break;
case 2:
children.AsParallel().ForAll(t =>
{
ThisIsARecursiveFunctionInMemory(t);
});
break;
case 3:
Parallel.ForEach(children, t =>
{
ThisIsARecursiveFunctionInMemory(t);
});
break;
default:
break;
}
}
}
class DirWithSubDirs
{
public List<DirWithSubDirs> SubDirs = new List<DirWithSubDirs>();
public String Path { get; private set; }
public DirWithSubDirs(String path)
{
this.Path = path;
try
{
SubDirs = Directory.GetDirectories(path).Select(t => new DirWithSubDirs(t)).ToList();
}
catch (Exception eee)
{
//Ignore directories that can't be accessed
}
}
}
Edit 2:
After reading the update on Matthew's comment I've tried to add the following code to the program:
ThreadPool.SetMinThreads(4000, 16);
ThreadPool.SetMaxThreads(4000, 16);
This however does not change how the AsParallel peforms. Still the first 8 steps are being executed in an instant before slowing down to 1 step / second.
(Extra note, I'm currently ignoring the exceptions that occur when I can't access a Directory by the Try Catch block around the Directory.GetDirectories())
Edit 3:
Also what I'm mainly interested in is the difference between Parallel.ForEach and AsParallel.ForAll because to me it's just strange that for some reason the second one creates one Thread for every recursion it does while the first once handles everything in around 30 threads max. (And also why MSDN suggests to use the AsParallel even though it creates so much threads with a ~1 second timeout)
Edit 4:
Another strange thing I found out:
When I try to set the MinThreads on the Thread pool above 1023 it seems to ignore the value and scale back to around 8 or 16:
ThreadPool.SetMinThreads(1023, 16);
Still when I use 1023 it does the first 1023 elements very fast followed by going back to the slow pace I've been experiencing all the time.
Note: Also literally more then 1000 threads are now created (compared to 30 for the whole Parallel.ForEach one).
Does this mean Parallel.ForEach is just way smarter in handling tasks?
Some more info, this code prints twice 8 - 8 when you set the value above 1023: (When you set the values to 1023 or lower it prints the correct value)
int threadsMin;
int completionMin;
ThreadPool.GetMinThreads(out threadsMin, out completionMin);
Console.WriteLine("Cur min threads: " + threadsMin + " and the other thing: " + completionMin);
ThreadPool.SetMinThreads(1023, 16);
ThreadPool.SetMaxThreads(1023, 16);
ThreadPool.GetMinThreads(out threadsMin, out completionMin);
Console.WriteLine("Now min threads: " + threadsMin + " and the other thing: " + completionMin);
Edit 5:
As of Dean's request I've created another case to manually create tasks:
case 4:
var taskList = new List<Task>();
foreach (var todo in children)
{
var itemTodo = todo;
taskList.Add(Task.Run(() => ThisIsARecursiveFunctionInMemory(itemTodo)));
}
Task.WaitAll(taskList.ToArray());
break;
This is also as fast as the Parallel.ForEach() loop. So we still don't have the answer to why AsParallel().ForAll() is so much slower.
This problem is pretty debuggable, an uncommon luxury when you have problems with threads. Your basic tool here is the Debug > Windows > Threads debugger window. Shows you the active threads and gives you a peek at their stack trace. You'll easily see that, once it gets slow, that you'll have dozens of threads active that are all stuck. Their stack trace all look the same:
mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout, bool exitContext) + 0x16 bytes
mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout) + 0x7 bytes
mscorlib.dll!System.Threading.ManualResetEventSlim.Wait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken) + 0x182 bytes
mscorlib.dll!System.Threading.Tasks.Task.SpinThenBlockingWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken) + 0x93 bytes
mscorlib.dll!System.Threading.Tasks.Task.InternalRunSynchronously(System.Threading.Tasks.TaskScheduler scheduler, bool waitForCompletion) + 0xba bytes
mscorlib.dll!System.Threading.Tasks.Task.RunSynchronously(System.Threading.Tasks.TaskScheduler scheduler) + 0x13 bytes
System.Core.dll!System.Linq.Parallel.SpoolingTask.SpoolForAll<ConsoleApplication1.DirWithSubDirs,int>(System.Linq.Parallel.QueryTaskGroupState groupState, System.Linq.Parallel.PartitionedStream<ConsoleApplication1.DirWithSubDirs,int> partitions, System.Threading.Tasks.TaskScheduler taskScheduler) Line 172 C#
// etc..
Whenever you see something like this, you should immediately think fire-hose problem. Probably the third-most common bug with threads, after races and deadlocks.
Which you can reason out, now that you know the cause, the problem with the code is that every thread that completes adds N more threads. Where N is the average number of sub-directories in a directory. In effect, the number of threads grows exponentially, that's always bad. It will only stay in control if N = 1, that of course never happens on an typical disk.
Do beware that, like almost any threading problem, that this misbehavior tends to repeat poorly. The SSD in your machine tends to hide it. So does the RAM in your machine, the program might well complete quickly and trouble-free the second time you run it. Since you'll now read from the file system cache instead of the disk, very fast. Tinkering with ThreadPool.SetMinThreads() hides it as well, but it cannot fix it. It never fixes any problem, it only hides them. Because no matter what happens, the exponential number will always overwhelm the set minimum number of threads. You can only hope that it completes finishing iterating the drive before that happens. Idle hope for a user with a big drive.
The difference between ParallelEnumerable.ForAll() and Parallel.ForEach() is now perhaps also easily explained. You can tell from the stack trace that ForAll() does something naughty, the RunSynchronously() method blocks until all the threads are completed. Blocking is something threadpool threads should not do, it gums up the thread pool and won't allow it to schedule the processor for another job. And has the effect you observed, the thread pool is quickly overwhelmed with threads that are waiting on the N other threads to complete. Which isn't happening, they are waiting in the pool and are not getting scheduled because there are already so many of them active.
This is a deadlock scenario, a pretty common one, but the threadpool manager has a workaround for it. It watches the active threadpool threads and steps in when they don't complete in a timely manner. It then allows an extra thread to start, one more than the minimum set by SetMinThreads(). But not more then the maximum set by SetMaxThreads(), having too many active tp threads is risky and likely to trigger OOM. This does solve the deadlock, it gets one of the ForAll() calls to complete. But this happens at a very slow rate, the threadpool only does this twice a second. You'll run out of patience before it catches up.
Parallel.ForEach() doesn't have this problem, it doesn't block so doesn't gum up the pool.
Seems to be the solution, but do keep in mind that your program is still fire-hosing the memory of your machine, adding ever more waiting tp threads to the pool. This can crash your program as well, it just isn't as likely because you have a lot of memory and the threadpool doesn't use a lot of it to keep track of a request. Some programmers however accomplish that as well.
The solution is a very simple one, just don't use threading. It is harmful, there is no concurrency when you have only one disk. And it does not like being commandeered by multiple threads. Especially bad on a spindle drive, head seeks are very, very slow. SSDs do it a lot better, it however still takes an easy 50 microseconds, overhead that you just don't want or need. The ideal number of threads to access a disk that you can't otherwise expect to be cached well is always one.
The first thing to note is that you are trying to parallelise an IO-bound operation, which will distort the timings significantly.
The second thing to note is the nature of the parallelised tasks: You are recursively descending a directory tree. If you create multiple threads to do this, each thread is likely to be accessing a different part of the disk simultaneously - which will cause the disk read head to be jumping all over the place and slowing things down considerably.
Try changing your test to create an in-memory tree, and access that with multiple threads instead. Then you will be able to compare the timings properly without the results being distorted beyond all usefulness.
Additionally, you may be creating a great number of threads, and they will (by default) be threadpool threads. Having a great number of threads will actually slow things down when they exceed the number of processor cores.
Also note that when you exceed the thread pool minimum threads (defined by ThreadPool.GetMinThreads()), a delay is introduced by the thread pool manager between each new threadpool thread creation. (I think this is around 0.5s per new thread).
Also, if the number of threads exceeds the value returned by ThreadPool.GetMaxThreads(), the creating thread will block until one of the other threads has exited. I think this is likely to be happening.
You can test this hypothesis by calling ThreadPool.SetMaxThreads() and ThreadPool.SetMinThreads() to increase these values, and see if it makes any difference.
(Finally, note that if you are really trying to recursively descend from C:\, you will almost certainly get an IO exception when it reaches a protected OS folder.)
NOTE: Set the max/min threadpool threads like this:
ThreadPool.SetMinThreads(4000, 16);
ThreadPool.SetMaxThreads(4000, 16);
Follow Up
I have tried your test code with the threadpool thread counts set as described above, with the following results (not run on the whole of my C:\ drive, but on a smaller subset):
Mode 1 took 06.5 seconds.
Mode 2 took 15.7 seconds.
Mode 3 took 16.4 seconds.
This is in line with my expectations; adding a load of threading to do this actually makes it slower than single-threaded, and the two parallel approaches take roughly the same time.
In case anyone else wants to investigate this, here's some determinative test code (the OP's code is not reproducible because we don't know his directory structure).
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
namespace Demo
{
internal class Program
{
private static DirWithSubDirs RootDir;
private static void Main()
{
Console.WriteLine("Loading file system into memory...");
RootDir = new DirWithSubDirs("Root", 4, 4);
Console.WriteLine("Done");
//ThreadPool.SetMinThreads(4000, 16);
//ThreadPool.SetMaxThreads(4000, 16);
var w = Stopwatch.StartNew();
ThisIsARecursiveFunctionInMemory(RootDir);
Console.WriteLine("Elapsed seconds: " + w.Elapsed.TotalSeconds);
Console.ReadKey();
}
public static void ThisIsARecursiveFunctionInMemory(DirWithSubDirs currentDirectory)
{
var depth = currentDirectory.Path.Count(t => t == '\\');
Console.WriteLine(depth + ": " + currentDirectory.Path);
var children = currentDirectory.SubDirs;
//Edit this mode to switch what way of parallelization it should use
int mode = 3;
switch (mode)
{
case 1:
foreach (var child in children)
{
ThisIsARecursiveFunctionInMemory(child);
}
break;
case 2:
children.AsParallel().ForAll(t =>
{
ThisIsARecursiveFunctionInMemory(t);
});
break;
case 3:
Parallel.ForEach(children, t =>
{
ThisIsARecursiveFunctionInMemory(t);
});
break;
default:
break;
}
}
}
internal class DirWithSubDirs
{
public List<DirWithSubDirs> SubDirs = new List<DirWithSubDirs>();
public String Path { get; private set; }
public DirWithSubDirs(String path, int width, int depth)
{
this.Path = path;
if (depth > 0)
for (int i = 0; i < width; ++i)
SubDirs.Add(new DirWithSubDirs(path + "\\" + i, width, depth - 1));
}
}
}
The Parallel.For and .ForEach methods are implemented internally as equivalent to running iterations in Tasks, e.g. that a loop like:
Parallel.For(0, N, i =>
{
DoWork(i);
});
is equivalent to:
var tasks = new List<Task>(N);
for(int i=0; i<N; i++)
{
tasks.Add(Task.Factory.StartNew(state => DoWork((int)state), i));
}
Task.WaitAll(tasks.ToArray());
And from the perspective of every iteration potentially running in parallel with every other iteration, this is an ok mental model, but does not happen in reality. Parallel, in fact, does not necessarily use one Task per iteration, as that is significantly more overhead than is necessary. Parallel.ForEach tries to use the minimum number of tasks necessary to complete the loop as fast as possible. It spins up tasks as threads become available to process those tasks, and each of those tasks participates in a management scheme (I think its called chunking): A task asks for multiple iterations to be done, gets them, and then processes that work, and then goes back for more. The chunk sizes vary based the number of tasks participating, the load on the machine, etc.
PLINQ’s .AsParallel() has a different implementation, but it ‘can’ still similarly fetch multiple iterations into a temporary store, do the calculations in a thread (but not as a task), and put the query results into a small buffer. (You get something based on ParallelQuery, and then further .Whatever() functions bind to an alternative set of extension methods that provide parallel implementations).
So now that we have a small idea of how these two mechanisms work, I will try to provide an answer to your original question:
So why is .AsParallel() slower than Parallel.ForEach? The reason stems from the following. Tasks (or their equivalent implementation here) do NOT block on I/O-like calls. They ‘await’ and free up the CPU to do something else. But (quoting C# nutshell book): “PLINQ cannot perform I/O-bound work without blocking threads”. The calls are synchronous. They were written with the intention that you increase the degree of parallelism if (and ONLY if) you are doing such things as downloading web pages per task that do not hog CPU time.
And the reason why your function calls are exactly analogous to I/O bound calls is this: One of your threads (call it T) blocks and does nothing until all of its child threads have finished, which can be a slow process here. T itself is not CPU-intensive while it waits for the children to unblock, it is doing nothing but waiting. Hence it is identical to a typical I/O bound function call.
Based on the accepted answer to How exactly does AsParallel work?
.AsParallel.ForAll() casts back to IEnumerable before calling .ForAll()
so it creates 1 new thread + N recursive calls (each of which generates a new thread).
I'm working on one program that takes information from files and then stores them in MySQL database. This MySQL database is located in another dedicated server which is much more powerful than this server here. Data is being sent over LAN using 1gbps connection.
It is using 8 threads because my server has 8 cores, but somehow it runs so slowly.
CPU is: Intel Xeon E3-1270 v 3 # 3.50Ghz
RAM: 16 GB ECC
HDD: SATA 3 1TB
My program's CPU usage is only 0-5%
CPU affinity is all 8 cores
So, do you have any ideas what's wrong or how can I increase the speed of my program?
UPDATE:
I updated my code and it appears to be faster:
Parallel.For(0, this.data_files.Count, new ParallelOptions { MaxDegreeOfParallelism = this.MaxThreads }, i =>
{
this.ThreadCount++;
this.ParseFile(this.GetSource());
});
Here's a code snippet that deploys threads:
while (true)
{
if (this.ThreadCount < this.MaxThreads)
{
Task.Factory.StartNew(() =>
this.ParseFile(this.GetFile())
);
this.ThreadCount++;
}
else
{
Thread.Sleep(1);
}
this.UpdateConsole();
}
GetFile function:
private string GetFile()
{
string file = "";
string source = "";
while (true)
{
if (this.data_files.Count() != 0)
{
file = this.data_files[0];
this.data_files.RemoveAt(0);
if (File.Exists(file) == true)
{
source = File.ReadAllText(file);
File.Delete(file);
break;
}
}
}
return source;
}
I'm working on one program that takes information from files and then stores them in MySQL database.
Clearly your program is not CPU bound, it's IO bound. The bottlenecks are going to be based on your hard disk(s) and your network connection. Odds are even a single thread is going to be able to ensure proper utilization of these resources (in a well designed application). Adding extra threads generally won't help, it'll just create a bunch of threads that will spend their time waiting on various IO operations.
To use all the hardware resources is not the right goal for a program to have.
Instead, a better goal is to be as fast as possible. This is significantly different. While using more hardware resources can help, it is not always sufficient.
Sometimes, adding more resources to a problem doesn't help. In those cases, don't. Adding threads makes your program more complex, but not necessarily faster as you've seen.
C# already has good Asynchronous programming features with the TPL (which you are already using), so why not take advantage of that?
This will mean that the .NET framework will automatically manage the threads for you in an efficient way.
Here's what I propose:
foreach (var file in GetFilesToRead()) {
var task = PerformOperation(file);
// Keep a list of tasks, if you wish.
}
...
Task PerformOperation (string filename) {
var file = await ReadFile(file);
await ParseFile(file);
DoSomething();
}
Note that even in CPU-bound programs, threads (and tasks) may not help you if you're using locks.
Although locks help keep programs well-behaved, they come at a significant performance cost.
Within a lock, only one thread may be executing at a time.
This means that the first thread is locking your _lock instance, and then the other threads are waiting for that lock to be released.
In your program, only one thread is active at a time.
To solve this, don't use locks. Instead, write programs that do not need locks at all. Copy variables instead of sharing them. Use immutable collections instead of mutable collections and so on.
My program above uses exactly zero locks and, as such, will better utilize your threads.
I'm building a console application that have to process a bunch of data.
Basically, the application grabs references from a DB. For each reference, parse the content of the file and make some changes. The files are HTML files, and the process is doing a heavy work with RegEx replacements (find references and transform them into links). The results in then stored on the file system and sent to an external system.
If I resume the process, in a sequential way :
var refs = GetReferencesFromDB(); // ~5000 Datarow returned
foreach(var ref in refs)
{
var filePath = GetFilePath(ref); // This method looks up in a previously loaded file list
var html = File.ReadAllText(filePath); // Read html locally, or from a network drive
var convertedHtml = ParseHtml(html);
File.WriteAllText(destinationFilePath); // Copy the result locally, or a network drive
SendToWs(ref, convertedHtml);
}
My program is working correctly but is quite slow. That's why I want to parallelise the process.
By now, I made a simple Parallelization adding AsParallel :
var refs = GetReferencesFromDB().AsParallel();
refs.ForAll(ref=>
{
var filePath = GetFilePath(ref);
var html = File.ReadAllText(filePath);
var convertedHtml = ParseHtml(html);
File.WriteAllText(destinationFilePath);
SendToWs(ref, convertedHtml);
});
This simple change decrease the duration of the process (25% less time). However, what I understand with parallelization is that there won't be much benefits (or worse, less benefits) if parallelyzing over resources relying on I/O, because the i/o won't magically doubles.
That's why I think I should change my approach not to parallelize the whole process, but to create dependent chained queued tasks.
I.E., I should create a flow like :
Queue read file. When finished, Queue ParseHtml. When finished, Queue both send to WS and write locally. When finished, log the result.
However, I don't know how to realize such think.
I feel it will ends in a set of consumer/producer queues, but I didn't find a correct sample.
And moreover, I'm not sure if there will be benefits.
thanks for advices
[Edit] In fact, I'm the perfect candidate for using c# 4.5... if only it was rtm :)
[Edit 2] Another thing making me thinking it's not correctly parallelized, is that in the resource monitor, I see graphs of CPU, network I/O and disk I/O not stable. when one is high, others are low to medium
You're not leveraging any async I/O APIs in any of your code. Everything you're doing is CPU bound and all your I/O operations are going to waste CPU resources blocking. AsParallel is for compute bound tasks, if you want to take advantage of async I/O you need to leverage the Asynchronous Programming Model (APM) based APIs today in <= v4.0. This is done by looking for BeginXXX/EndXXX methods on the I/O based classes you're using and leveraging those whenever available.
Read this post for starters: TPL TaskFactory.FromAsync vs Tasks with blocking methods
Next, you don't want to use AsParallel in this case anyway. AsParallel enables streaming which will result in an immediately scheduling a new Task per item, but you don't need/want that here. You'd be much better served by partitioning the work using Parallel::ForEach.
Let's see how you can use this knowledge to achieve max concurrency in your specific case:
var refs = GetReferencesFromDB();
// Using Parallel::ForEach here will partition and process your data on separate worker threads
Parallel.ForEach(
refs,
ref =>
{
string filePath = GetFilePath(ref);
byte[] fileDataBuffer = new byte[1048576];
// Need to use FileStream API directly so we can enable async I/O
FileStream sourceFileStream = new FileStream(
filePath,
FileMode.Open,
FileAccess.Read,
FileShare.Read,
8192,
true);
// Use FromAsync to read the data from the file
Task<int> readSourceFileStreamTask = Task.Factory.FromAsync(
sourceFileStream.BeginRead
sourceFileStream.EndRead
fileDataBuffer,
fileDataBuffer.Length,
null);
// Add a continuation that will fire when the async read is completed
readSourceFileStreamTask.ContinueWith(readSourceFileStreamAntecedent =>
{
int soureFileStreamBytesRead;
try
{
// Determine exactly how many bytes were read
// NOTE: this will propagate any potential exception that may have occurred in EndRead
sourceFileStreamBytesRead = readSourceFileStreamAntecedent.Result;
}
finally
{
// Always clean up the source stream
sourceFileStream.Close();
sourceFileStream = null;
}
// This is here to make sure you don't end up trying to read files larger than this sample code can handle
if(sourceFileStreamBytesRead == fileDataBuffer.Length)
{
throw new NotSupportedException("You need to implement reading files larger than 1MB. :P");
}
// Convert the file data to a string
string html = Encoding.UTF8.GetString(fileDataBuffer, 0, sourceFileStreamBytesRead);
// Parse the HTML
string convertedHtml = ParseHtml(html);
// This is here to make sure you don't end up trying to write files larger than this sample code can handle
if(Encoding.UTF8.GetByteCount > fileDataBuffer.Length)
{
throw new NotSupportedException("You need to implement writing files larger than 1MB. :P");
}
// Convert the file data back to bytes for writing
Encoding.UTF8.GetBytes(convertedHtml, 0, convertedHtml.Length, fileDataBuffer, 0);
// Need to use FileStream API directly so we can enable async I/O
FileStream destinationFileStream = new FileStream(
destinationFilePath,
FileMode.OpenOrCreate,
FileAccess.Write,
FileShare.None,
8192,
true);
// Use FromAsync to read the data from the file
Task destinationFileStreamWriteTask = Task.Factory.FromAsync(
destinationFileStream.BeginWrite,
destinationFileStream.EndWrite,
fileDataBuffer,
0,
fileDataBuffer.Length,
null);
// Add a continuation that will fire when the async write is completed
destinationFileStreamWriteTask.ContinueWith(destinationFileStreamWriteAntecedent =>
{
try
{
// NOTE: we call wait here to observe any potential exceptions that might have occurred in EndWrite
destinationFileStreamWriteAntecedent.Wait();
}
finally
{
// Always close the destination file stream
destinationFileStream.Close();
destinationFileStream = null;
}
},
TaskContinuationOptions.AttachedToParent);
// Send to external system **concurrent** to writing to destination file system above
SendToWs(ref, convertedHtml);
},
TaskContinuationOptions.AttachedToParent);
});
Now, here's few notes:
This is sample code so I'm using a 1MB buffer to read/write files. This is excessive for HTML files and wasteful of system resources. You can either lower it to suit your max needs or implement chained reads/writes into a StringBuilder which is an excercise I leave up to you since I'd be writing ~500 more lines of code to do async chained reads/writes. :P
You'll note that on the continuations for the read/write tasks I have TaskContinuationOptions.AttachedToParent. This is very important as it will prevent the worker thread that the Parallel::ForEach starts the work with from completing until all the underlying async calls have completed. If this was not here you would kick off work for all 5000 items concurrently which would pollute the TPL subsystem with thousands of scheduled Tasks and not scale properly at all.
I call SendToWs concurrent to writing the file to the file share here. I don't know what is underlying the implementation of SendToWs, but it too sounds like a good candidate for making async. Right now it's assumed it's pure compute work and, as such, is going to burn a CPU thread while executing. I leave it as an excercise to you to figure out how best to leverage what I've shown you to improve throughput there.
This is all typed free form and my brain was the only compiler here and SO's syntax higlighting is all I used to make sure syntax was good. So, please forgive any syntax errors and let me know if I screwed up anything too badly that you can't make heads or tails of it and I'll follow up.
The good news is your logic could be easily separated into steps that go into a producer-consumer pipeline.
Step 1: Read file
Step 2: Parse file
Step 3: Write file
Step 4: SendToWs
If you are using .NET 4.0 you can use the BlockingCollection data structure as the backbone for the each step's producer-consumer queue. The main thread will enqueue each work item into step 1's queue where it will be picked up and processed and then forwarded on to step 2's queue and so on and so forth.
If you are willing to move on to the Async CTP then you can take advantage of the new TPL Dataflow structures for this as well. There is the BufferBlock<T> data structure, among others, that behaves in a similar manner to BlockingCollection and integrates well with the new async and await keywords.
Because your algorithm is IO bound the producer-consumer strategies may not get you the performance boost you are looking for, but at least you will have a very elegant solution that would scale well if you could increase the IO throughput. I am afraid steps 1 and 3 will be the bottlenecks and the pipeline will not balance well, but it is worth experimenting with.
Just a suggestion, but have you looked into the Consumer / Producer pattern ? A certain number of threads would read your files on disk and feed the content to a queue. Then another set of threads, known as the consumers, would "consume" the queue as its filled. http://zone.ni.com/devzone/cda/tut/p/id/3023
Your best bet in these kind of scenario is definitely the producer-consumer model. One thread to pull the data and a bunch of workers to process it. There's no easy way around the I/O so you might as well just focus on optimizing the computation itself.
I will now try to sketch a model:
// producer thread
var refs = GetReferencesFromDB(); // ~5000 Datarow returned
foreach(var ref in refs)
{
lock(queue)
{
queue.Enqueue(ref);
event.Set();
}
// if the queue is limited, test if the queue is full and wait.
}
// consumer threads
while(true)
{
value = null;
lock(queue)
{
if(queue.Count > 0)
{
value = queue.Dequeue();
}
}
if(value != null)
// process value
else
event.WaitOne(); // event to signal that an item was placed in the queue.
}
You can find more details about producer/consumer in part 4 of Threading in C#: http://www.albahari.com/threading/part4.aspx
I think your approach to split up the list of files and process each file in one batch is ok.
My feeling is that you might get more performance gain if you play with degree of parallelism.
See: var refs = GetReferencesFromDB().AsParallel().WithDegreeOfParallelism(16); this would start processing 16 files at the same time. Currently you are processing probably 2 or 4 files depending on number of cores you have. This is only efficient when you have only computation without IO. For IO intensive tasks adjustment might bring incredible performance improvements reducing processor idle time.
If you are going to split up and join tasks back using producer-consumer look at this sample: Using Parallel Linq Extensions to union two sequences, how can one yield the fastest results first?
I was looking at this question, looking for a way to create a single-threaded, event-based nonblocking asynchronous web server in .NET.
This answer looked promising at first, by claiming that the body of the code runs in a single thread.
However, I tested this in C#:
using System;
using System.IO;
using System.Threading;
class Program
{
static void Main()
{
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
var sc = new SynchronizationContext();
SynchronizationContext.SetSynchronizationContext(sc);
{
var path = Environment.ExpandEnvironmentVariables(
#"%SystemRoot%\Notepad.exe");
var fs = new FileStream(path, FileMode.Open,
FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true);
var bytes = new byte[1024];
fs.BeginRead(bytes, 0, bytes.Length, ar =>
{
sc.Post(dummy =>
{
var res = fs.EndRead(ar);
// Are we in the same thread?
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
}, null);
}, null);
}
Thread.Sleep(100);
}
}
And the result was:
1
5
So it seems like, contrary to the answer, the thread initiating the read and the thread ending the read are not the same.
So now my question is, how do you to achieve a single-threaded, event-based nonblocking asynchronous web server in .NET?
The whole SetSynchronizationContext is a red herring, this is just a mechanism for marshalling, the work still happens in the IO Thread Pool.
What you are asking for is a way to queue and harvest Asynchronous Procedure Calls for all your IO work from the main thread. Many higher level frameworks wrap this kind functionality, the most famous one being libevent.
There is a great recap on the various options here: Whats the difference between epoll, poll, threadpool?.
.NET already takes care of scaling for you by have a special "IO Thread Pool" that handles IO access when you call the BeginXYZ methods. This IO Thread Pool must have at least 1 thread per processor on the box. see: ThreadPool.SetMaxThreads.
If single threaded app is a critical requirement (for some crazy reason) you could, of course, interop all of this stuff in using DllImport (see an example here)
However it would be a very complex and risky task:
Why don't we support APCs as a completion mechanism? APCs are really not a good general-purpose completion mechanism for user code. Managing the reentrancy introduced by APCs is nearly impossible; any time you block on a lock, for example, some arbitrary I/O completion might take over your thread. It might try to acquire locks of its own, which may introduce lock ordering problems and thus deadlock. Preventing this requires meticulous design, and the ability to make sure that someone else's code will never run during your alertable wait, and vice-versa. This greatly limits the usefulness of APCs.
So, to recap. If you want a single threaded managed process that does all its work using APC and completion ports, you are going to have to hand code it. Building it would be risky and tricky.
If you simply want high scale networking, you can keep using BeginXYZ and family and rest assured that it will perform well, since it uses APC. You pay a minor price marshalling stuff between threads and the .NET particular implementation.
From: http://msdn.microsoft.com/en-us/magazine/cc300760.aspx
The next step in scaling up the server is to use asynchronous I/O. Asynchronous I/O alleviates the need to create and manage threads. This leads to much simpler code and also is a more efficient I/O model. Asynchronous I/O utilizes callbacks to handle incoming data and connections, which means there are no lists to set up and scan and there is no need to create new worker threads to deal with the pending I/O.
An interesting, side fact, is that single threaded is not the fastest way to do async sockets on Windows using completion ports see: http://doc.sch130.nsc.ru/www.sysinternals.com/ntw2k/info/comport.shtml
The goal of a server is to incur as few context switches as possible by having its threads avoid unnecessary blocking, while at the same time maximizing parallelism by using multiple threads. The ideal is for there to be a thread actively servicing a client request on every processor and for those threads not to block if there are additional requests waiting when they complete a request. For this to work correctly however, there must be a way for the application to activate another thread when one processing a client request blocks on I/O (like when it reads from a file as part of the processing).
What you need is a "message loop" which takes the next task on a queue and executes it. Additionally, every task needs to be coded so that it completes as much work as possible without blocking, and then enqueues additional tasks to pick up a task that needs time later. There is nothing magical about this: never using a blocking call and never spawn additional threads.
For example, when processing an HTTP GET, the server can read as much data as is currently available on the socket. If this is not enough data to handle the request, then enqueue a new task to read from the socket again in the future. In the case of a FileStream, you want to set the ReadTimeout on the instance to a low value and be prepared to read fewer bytes than the entire file.
C# 5 actually makes this pattern much more trivial. Many people think that the async functionality implies multithreading, but that is not the case. Using async, you can essentially get the task queue I mentioned earlier without ever explicility managing it.
Yes, it's called Manos de mono
Seriously, the entire idea behind manos is a single threaded asynchronous event driven web server.
High performance and scalable. Modeled after tornadoweb, the technology that powers friend feed, Manos is capable of thousands of simultaneous connections, ideal for applications that create persistent connections with the server.
The project appears to be low on maintenance and probably wouldn't be production ready but it makes a good case study as a demonstration that this is possible.
Here's a great article series explaining what IO Completion Ports are and how they can be accessed via C# (i.e. you need to PInvoke into Win32 API calls from the Kernel32.dll).
Note: The libuv the cross platform IO framework behind node.js uses IOCP on Windows and libev on unix operating systems.
http://www.theukwebdesigncompany.com/articles/iocp-thread-pooling.php
i am wondering nobody mentioned kayak it's basicly C#s answer to Pythons twisted, JavaScripts node.js or Rubys eventmachine
I've been fiddling with my own simple implementation of such an architecture and I've put it up on github. I'm doing it more as a learning thing. But it's been a lot of fun and I think I'll flush it out more.
It's very alpha, so it's liable to change, but the code looks a little like this:
//Start the event loop.
EventLoop.Start(() => {
//Create a Hello World server on port 1337.
Server.Create((req, res) => {
res.Write("<h1>Hello World</h1>");
}).Listen("http://*:1337");
});
More information about it can be found here.
I developed a server based on HttpListener and an event loop, supporting MVC, WebApi and routing. For what i have seen the performances are far better than standard IIS+MVC, for the MVCMusicStore i moved from 100 requests per seconds and 100% CPU to 350 with 30% CPU.
If anybody would give it a try i am struggling for feedbacks!
Actually is present a template to create websites based on this structure.
Note that I DON'T USE ASYNC/AWAIT until absolutely necessary. The only tasks i use there are the ones for the I/O bound operations like writing on the socket or reading files.
PS any suggestion or correction is welcome!
Documentation
MvcMusicStore sample port on Node.Cs
Packages on Nuget
you can this framework SignalR
and this Blog about it
Some kind of the support from operating system is essential here. For example, Mono uses epoll on Linux with asynchronous I/O, so it should scale really well (still thread pool). If you are looking and performance and scalability, definitely try it.
On the other hand, the example of C# (with native libs) webserver which is based around idea you have mentioned can be Manos de Mono. Project has not been active lately; however, idea and code is generally available. Read this (especially the "A closer look at Manos" part).
Edit:
If you just want to have callback fired on your main thread, you can do a little abuse of existing synchronization contexts like the WPF dispatcher. Your code, translated to this approach:
using System;
using System.IO;
using System.Threading;
using System.Windows;
namespace Node
{
class Program
{
public static void Main()
{
var app = new Application();
app.Startup += ServerStart;
app.Run();
}
private static void ServerStart(object sender, StartupEventArgs e)
{
var dispatcher = ((Application) sender).Dispatcher;
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
var path = Environment.ExpandEnvironmentVariables(
#"%SystemRoot%\Notepad.exe");
var fs = new FileStream(path, FileMode.Open,
FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true);
var bytes = new byte[1024];
fs.BeginRead(bytes, 0, bytes.Length, ar =>
{
dispatcher.BeginInvoke(new Action(() =>
{
var res = fs.EndRead(ar);
// Are we in the same thread?
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
}));
}, null);
}
}
}
prints what you wish. Plus you can set priorities with dispatcher. But agree, this is ugly, hacky and I do not know why I would do it that way for another reason than answer your demo request ;)
First about SynchronizationContext. It's just like Sam wrote. Base class won't give You single-thread functionality. You probably got that idea from WindowsFormsSynchronizationContext which provides functionality to execute code on UI thread.
You can read more here
I've written a piece of code that works with ThreadPool parameters. (Again something Sam already pointed out).
This code registers 3 asynchronous actions to be executed on free thread. They run in parallel until one of them changes ThreadPool parameters. Then each action is executed on the same thread.
It only proves that you can force .net app to use one thread.
Real implementation of web server that would receive and process calls on only one thread is something entirely different :).
Here's the code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.IO;
namespace SingleThreadTest
{
class Program
{
class TestState
{
internal string ID { get; set; }
internal int Count { get; set; }
internal int ChangeCount { get; set; }
}
static ManualResetEvent s_event = new ManualResetEvent(false);
static void Main(string[] args)
{
Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
int nWorkerThreads;
int nCompletionPortThreads;
ThreadPool.GetMaxThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("Max Workers: {0} Ports: {1}",nWorkerThreads,nCompletionPortThreads));
ThreadPool.GetMinThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("Min Workers: {0} Ports: {1}",nWorkerThreads,nCompletionPortThreads));
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = "A ", Count = 10, ChangeCount = 0 });
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = " B ", Count = 10, ChangeCount = 5 });
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = " C", Count = 10, ChangeCount = 0 });
s_event.WaitOne();
Console.WriteLine("Press enter...");
Console.In.ReadLine();
}
static void LetsRunLikeCrazy(object o)
{
if (s_event.WaitOne(0))
{
return;
}
TestState oState = o as TestState;
if (oState != null)
{
// Are we in the same thread?
Console.WriteLine(String.Format("Hello. Start id: {0} in thread: {1}",oState.ID, Thread.CurrentThread.ManagedThreadId));
Thread.Sleep(1000);
oState.Count -= 1;
if (oState.ChangeCount == oState.Count)
{
int nWorkerThreads = 1;
int nCompletionPortThreads = 1;
ThreadPool.SetMinThreads(nWorkerThreads, nCompletionPortThreads);
ThreadPool.SetMaxThreads(nWorkerThreads, nCompletionPortThreads);
ThreadPool.GetMaxThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("New Max Workers: {0} Ports: {1}", nWorkerThreads, nCompletionPortThreads));
ThreadPool.GetMinThreads(out nWorkerThreads, out nCompletionPortThreads);
Console.WriteLine(String.Format("New Min Workers: {0} Ports: {1}", nWorkerThreads, nCompletionPortThreads));
}
if (oState.Count > 0)
{
Console.WriteLine(String.Format("Hello. End id: {0} in thread: {1}", oState.ID, Thread.CurrentThread.ManagedThreadId));
ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), oState);
}
else
{
Console.WriteLine(String.Format("Hello. End id: {0} in thread: {1}", oState.ID, Thread.CurrentThread.ManagedThreadId));
s_event.Set();
}
}
else
{
Console.WriteLine("Error !!!");
s_event.Set();
}
}
}
}
LibuvSharp is a wrapper for libuv, which is used in the node.js project for async IO. BUt it only contains only low level TCP/UDP/Pipe/Timer functionality. And it will stay like that, writing a webserver on top of it is an entire different story. It doesn't even support dns resolving, since this is just a protocol on top of udp.
I believe it's possible, here is an open-source example written in VB.NET and C#:
https://github.com/perrybutler/dotnetsockets/
It uses Event-based Asynchronous Pattern (EAP), IAsyncResult Pattern and thread pool (IOCP). It will serialize/marshal the messages (messages can be any native object such as a class instance) into binary packets, transfer the packets over TCP, and then deserialize/unmarshal the packets at the receiving end so you get your native object to work with. This part is somewhat like Protobuf or RPC.
It was originally developed as a "netcode" for real-time multiplayer gaming, but it can serve many purposes. Unfortunately I never got around to using it. Maybe someone else will.
The source code has a lot of comments so it should be easy to follow. Enjoy!
Here is one more implementation of the event-loop web server called SingleSand. It executes all custom logic inside single-threaded event loop but the web server is hosted in asp.net.
Answering the question, it is generally not possible to run a pure single threaded app because of .NET multi-threaded nature. There are some activities that run in separate threads and developer cannot change their behavior.
We have a C# application that connects to a FTP server, downloads some files, disconnects, and after a certain amount of time (selected by the user through the UI) reconnects and repeats the process. We implemented this using BackgroundWorker, but we noticed that after running for a longer time, the program stopped logging its actions, both in the UI and the log file.
At that point, there were no files for it to download, so we uploaded some and it resumed activity as if nothing had happened.
The problem was that the regular users had no way of knowing that the program was still working, so we decided to implement it using our own threading. We did a simpler program, to rule out any other problems, and this one only connects to the FTP and disconnects. It stopped displaying messages just like BackgroundWorker (one time after 2 hours, one time after 22 hours, without any pattern that we could find, and on a computer that did nothing else).
DoFTPWork += new DoFTPWorkDelegate(WriteFTPMessage);
FTPWorkThread = new Thread(new ParameterizedThreadStart(Process));
//seData is the FTP login info
FTPWorkThread.Start(seData);
and the FTP method is:
private void Process(object seData1)
{
seData = (SEData)seData1;
while (!stopped)
{
try
{
ftp = null;
ftp = new FTP_Client();
if (ftp.IsConnected)
{
logMessages += DateTime.Now + "\t" + "info" + "\t" + "Ftp disconnected from " + seData.host + "\r\n";
ftp.Disconnect();
}
ftp.Connect(seData.host, 21);
ftp.Authenticate(seData.userName, seData.password);
logMessages += DateTime.Now + "\t" + "info" + "\t" + "Ftp connected to " + seData.host + "\r\n";
error = false;
logMessages += DateTime.Now + "\t" + "info" + "\t" + "Trying to reconnect in 5 seconds\r\n";
System.Threading.Thread.Sleep(5000);
SlaveEventArgs ev = new SlaveEventArgs();
ev.Message = logMessages;
txtLog.Invoke(DoFTPWork, ev);
System.Threading.Thread.Sleep(200);
logMessages = "";
}
catch (Exception ex)
{
logMessages = "";
if (ftp.IsConnected)
{
ftp.Disconnect();
}
ftp.Dispose();
logMessages += DateTime.Now + "\t" + "ERR" + "\t" + ex.Message + "\r\n";
logMessages += DateTime.Now + "\t" + "info" + "\t" + "Trying to reconnect in 5 seconds\r\n";
SlaveEventArgs ev = new SlaveEventArgs();
ev.Message = logMessages;
txtLog.Invoke(DoFTPWork, ev);
System.Threading.Thread.Sleep(5 * 1000);
error = true;
}
}
}
WriteFTPMessage displays the message in a TextBox and in the original program wrote to a .txt file.
If I'm understanding you correctly this while(!stopped) loop is the loop that is running for several hours? If that is the case, where are you terminating your ftp connection if anywhere? The only time you close it in the code you've posted is if an exception is thrown, otherwise you simply dereference the object and create a new one which is a pretty serious resource leak and at least contributing to the problem if not causing it.
Also it seems that ftp is globally accessible. Are you accessing it anywhere using a different thread? Is the object thread safe??
EDIT:
The biggest issue I see here is design. Not that I'm trying to bag on you or anything but you've got all sorts of operations intermixed. Threading, logging and ftp access code all in the same function.
What I would recommend is restructuring your program. Create a method much like the following:
// Called by thread
void MyThreadOperation()
{
while(!stopped)
{
// This is poor design in terms of performance.
// Consider using a ResetEvent instead.
Thread.Sleep(5000);
try
{
doFTPDownload();
}
catch(Exception ex)
{
logMessage(ex.ToString());
}
}
}
doFTPDownload() should be self contained. The FTP object should be created and opened inside the function when it is called and prior to it finishing it should be closed. This same concept should be applied to logMessage() as well. I would also recommend using a database to store log messages instead of a file so that locking issues don't complicate matters.
I know this isn't an answer in that you may still experience issues since I can't say for certain what could be the cause. However I'm confident with a little design restructuring you will be much better able to track down the source of the problem.
I would suggest putting anything that can go wrong in the catch block (in particular the bit which disconnects from the FTP server) in its own try/catch block. In addition, log something as soon as you've caught the exception, before doing anything else - that way you're more likely to be able to tell if the logging dies half way through for some reason.
Also, add a log message to the end of the while loop so that you can tell if it's finished "normally".
I'd suggest using adplus when the issue reproduces and getting yourself a hang dump. Analyze in Windbg and SoS.
Is that in a Winforms application? Maybe the ISynchronizeInvoke implementation is hanging. Is this running as an interactive user?
Rupert: I have added ftp.Disconnect() after the catch block and started it again. I've checked the original application and we disconnected before reconnecting, so while it can influence the problem, I don't think it's causing it.
There are no other threads accessing it, so there are no problems there.
Jon: I will, thanks for the suggestion.
JD: It is a Windows application, and after selecting the delay and FTP connect data, the user doesn't give any input. I'll look into ISynchronizeInvoke
I think you'll have to work on making it more thread safe. You have a lot of shared fields: ftp, logMessages, error.
For instance this part:
ev.Message = logMessages;
txtLog.Invoke(DoFTPWork, ev);
System.Threading.Thread.Sleep(200);
logMessages = "";
Sounds to me like you're trying to solve a multithreading problem by sleeping and crossing your fingers you slept enough...
you could solve this via:
ev.Message = logMessages.Clone();
txtLog.Invoke(DoFTPWork, ev);
or use a different way of communicating.
Instead of the stopped boolean you could use a ManualResetEvent, which is a thread safe communication method. And for error you could use the same, or a Semaphore.
The nice thing about the ManualResetEvent is that you can use it to sleep your thread without locking it up completely. If I'm not mistaken the only way to stop your thread while it's sleeping is to call a thread.Abort. If you use the ManualResetEvent you could do the following:
if (!shouldStop.WaitOne(5000))
{
// do thread stuff
}
else
{
// do cleanup stuff and exit thread.
}
The nice thing is, you'll say I want to know if the event was signalled or not, but I will wait 5 seconds for it to signal or else I'll continue as not signalled.
So if your application decides to quit after 3 seconds in the sleep, it can just do a shouldStop.Set() and the thread will stop. It's still possible that the thread is in the process of communicating with the ftp server, so after the set you should do a thread.Join() to wait for it to exit.
I'm not saying your problem is related to my suggestions, if not I'm only trying to help reducing the possible causes.