We Keep Coding

No synchronized non-blocking read method in basic Stream/StreamReader class

Recently I'm trying some .Net.Sockets secured networking by using BouncyCastle library.
The TlsStream class in BouncyCastle inherits the original Stream (not NetworkStream), and StreamReader/StreamWriter seem to be a convenient way for read/write.
Since I tend to use 1 thread for 1 end(server or client) to handle both read and write :
void CommunicationLoop() // Loops in Thread A
{
while (true)
{
ReadFromStream(); // If data available. It always hangs/blocks here(if there's no data to be read.)
WriteToStream(); // If user input something.
}
}
void ReadFromStream()
{
String line;
while ( StreamReader.Peek() > -1 )
// Or ((line = StreamReader.ReadLine()) != null) / (Stream.Read(buff, 0, buff.Length) > 0)
// or any synchronized Readxxx() methods.
// It always hangs/blocks here(if there's no data to be read.)
{
line = StreamReader.ReadLine();
Console.WriteLine($"Received: {line}");
}
}
void WriteToStream()
{
//...
}
I did a lot of research, everyone suggests to use async method to solve the problem.
I would like to know that, is there really no official method/function to check if there is data to be read in StreamReader/Stream, if no data then skip(instead of hanging there waiting for the input, like the NetworkStream.DataAvailable)?
Also, if the the communication for 1 connection is not heavy, isn't using 1 thread dealing with both read/write in server side (there might be multiple connections from MANY CLIENTS to ONE SERVER) more efficient(saves resource)?
Thanks.

I would like to know that, is there really no official method/function to check if there is data to be read in StreamReader/Stream
Check the documentation for StreamReader. As far as I can see there is no way to check for waiting data without using the async methods.
Also, if the the communication for 1 connection is not heavy, isn't using 1 thread dealing with both read/write in server side (there might be multiple connections from MANY CLIENTS to ONE SERVER) more efficient(saves resource)
This should not be more efficient than using the async methods. Consider the case where all clients are idle. Your method would use 1 thread per client. Using async methods would not use any threads. Assuming the async methods use non-blocking IO in the backend. It is possible the sync methods have slightly lower overhead since they can do the synchronization in the kernel rather than in .Net, but I think this would need benchmarking to verify.
Is there some specific reason you do not want to use the async methods?

Program doesn't use all hardware resources

I'm working on one program that takes information from files and then stores them in MySQL database. This MySQL database is located in another dedicated server which is much more powerful than this server here. Data is being sent over LAN using 1gbps connection.
It is using 8 threads because my server has 8 cores, but somehow it runs so slowly.
CPU is: Intel Xeon E3-1270 v 3 # 3.50Ghz
RAM: 16 GB ECC
HDD: SATA 3 1TB
My program's CPU usage is only 0-5%
CPU affinity is all 8 cores
So, do you have any ideas what's wrong or how can I increase the speed of my program?
UPDATE:
I updated my code and it appears to be faster:
Parallel.For(0, this.data_files.Count, new ParallelOptions { MaxDegreeOfParallelism = this.MaxThreads }, i =>
{
this.ThreadCount++;
this.ParseFile(this.GetSource());
});
Here's a code snippet that deploys threads:
while (true)
{
if (this.ThreadCount < this.MaxThreads)
{
Task.Factory.StartNew(() =>
this.ParseFile(this.GetFile())
);
this.ThreadCount++;
}
else
{
Thread.Sleep(1);
}
this.UpdateConsole();
}
GetFile function:
private string GetFile()
{
string file = "";
string source = "";
while (true)
{
if (this.data_files.Count() != 0)
{
file = this.data_files[0];
this.data_files.RemoveAt(0);
if (File.Exists(file) == true)
{
source = File.ReadAllText(file);
File.Delete(file);
break;
}
}
}
return source;
}

I'm working on one program that takes information from files and then stores them in MySQL database.
Clearly your program is not CPU bound, it's IO bound. The bottlenecks are going to be based on your hard disk(s) and your network connection. Odds are even a single thread is going to be able to ensure proper utilization of these resources (in a well designed application). Adding extra threads generally won't help, it'll just create a bunch of threads that will spend their time waiting on various IO operations.

To use all the hardware resources is not the right goal for a program to have.
Instead, a better goal is to be as fast as possible. This is significantly different. While using more hardware resources can help, it is not always sufficient.
Sometimes, adding more resources to a problem doesn't help. In those cases, don't. Adding threads makes your program more complex, but not necessarily faster as you've seen.
C# already has good Asynchronous programming features with the TPL (which you are already using), so why not take advantage of that?
This will mean that the .NET framework will automatically manage the threads for you in an efficient way.
Here's what I propose:
foreach (var file in GetFilesToRead()) {
var task = PerformOperation(file);
// Keep a list of tasks, if you wish.
}
...
Task PerformOperation (string filename) {
var file = await ReadFile(file);
await ParseFile(file);
DoSomething();
}
Note that even in CPU-bound programs, threads (and tasks) may not help you if you're using locks.
Although locks help keep programs well-behaved, they come at a significant performance cost.
Within a lock, only one thread may be executing at a time.
This means that the first thread is locking your _lock instance, and then the other threads are waiting for that lock to be released.
In your program, only one thread is active at a time.
To solve this, don't use locks. Instead, write programs that do not need locks at all. Copy variables instead of sharing them. Use immutable collections instead of mutable collections and so on.
My program above uses exactly zero locks and, as such, will better utilize your threads.

How to properly parallelise job heavily relying on I/O

I'm building a console application that have to process a bunch of data.
Basically, the application grabs references from a DB. For each reference, parse the content of the file and make some changes. The files are HTML files, and the process is doing a heavy work with RegEx replacements (find references and transform them into links). The results in then stored on the file system and sent to an external system.
If I resume the process, in a sequential way :
var refs = GetReferencesFromDB(); // ~5000 Datarow returned
foreach(var ref in refs)
{
var filePath = GetFilePath(ref); // This method looks up in a previously loaded file list
var html = File.ReadAllText(filePath); // Read html locally, or from a network drive
var convertedHtml = ParseHtml(html);
File.WriteAllText(destinationFilePath); // Copy the result locally, or a network drive
SendToWs(ref, convertedHtml);
}
My program is working correctly but is quite slow. That's why I want to parallelise the process.
By now, I made a simple Parallelization adding AsParallel :
var refs = GetReferencesFromDB().AsParallel();
refs.ForAll(ref=>
{
var filePath = GetFilePath(ref);
var html = File.ReadAllText(filePath);
var convertedHtml = ParseHtml(html);
File.WriteAllText(destinationFilePath);
SendToWs(ref, convertedHtml);
});
This simple change decrease the duration of the process (25% less time). However, what I understand with parallelization is that there won't be much benefits (or worse, less benefits) if parallelyzing over resources relying on I/O, because the i/o won't magically doubles.
That's why I think I should change my approach not to parallelize the whole process, but to create dependent chained queued tasks.
I.E., I should create a flow like :
Queue read file. When finished, Queue ParseHtml. When finished, Queue both send to WS and write locally. When finished, log the result.
However, I don't know how to realize such think.
I feel it will ends in a set of consumer/producer queues, but I didn't find a correct sample.
And moreover, I'm not sure if there will be benefits.
thanks for advices
[Edit] In fact, I'm the perfect candidate for using c# 4.5... if only it was rtm :)
[Edit 2] Another thing making me thinking it's not correctly parallelized, is that in the resource monitor, I see graphs of CPU, network I/O and disk I/O not stable. when one is high, others are low to medium

You're not leveraging any async I/O APIs in any of your code. Everything you're doing is CPU bound and all your I/O operations are going to waste CPU resources blocking. AsParallel is for compute bound tasks, if you want to take advantage of async I/O you need to leverage the Asynchronous Programming Model (APM) based APIs today in <= v4.0. This is done by looking for BeginXXX/EndXXX methods on the I/O based classes you're using and leveraging those whenever available.
Read this post for starters: TPL TaskFactory.FromAsync vs Tasks with blocking methods
Next, you don't want to use AsParallel in this case anyway. AsParallel enables streaming which will result in an immediately scheduling a new Task per item, but you don't need/want that here. You'd be much better served by partitioning the work using Parallel::ForEach.
Let's see how you can use this knowledge to achieve max concurrency in your specific case:
var refs = GetReferencesFromDB();
// Using Parallel::ForEach here will partition and process your data on separate worker threads
Parallel.ForEach(
refs,
ref =>
{
string filePath = GetFilePath(ref);
byte[] fileDataBuffer = new byte[1048576];
// Need to use FileStream API directly so we can enable async I/O
FileStream sourceFileStream = new FileStream(
filePath,
FileMode.Open,
FileAccess.Read,
FileShare.Read,
8192,
true);
// Use FromAsync to read the data from the file
Task<int> readSourceFileStreamTask = Task.Factory.FromAsync(
sourceFileStream.BeginRead
sourceFileStream.EndRead
fileDataBuffer,
fileDataBuffer.Length,
null);
// Add a continuation that will fire when the async read is completed
readSourceFileStreamTask.ContinueWith(readSourceFileStreamAntecedent =>
{
int soureFileStreamBytesRead;
try
{
// Determine exactly how many bytes were read
// NOTE: this will propagate any potential exception that may have occurred in EndRead
sourceFileStreamBytesRead = readSourceFileStreamAntecedent.Result;
}
finally
{
// Always clean up the source stream
sourceFileStream.Close();
sourceFileStream = null;
}
// This is here to make sure you don't end up trying to read files larger than this sample code can handle
if(sourceFileStreamBytesRead == fileDataBuffer.Length)
{
throw new NotSupportedException("You need to implement reading files larger than 1MB. :P");
}
// Convert the file data to a string
string html = Encoding.UTF8.GetString(fileDataBuffer, 0, sourceFileStreamBytesRead);
// Parse the HTML
string convertedHtml = ParseHtml(html);
// This is here to make sure you don't end up trying to write files larger than this sample code can handle
if(Encoding.UTF8.GetByteCount > fileDataBuffer.Length)
{
throw new NotSupportedException("You need to implement writing files larger than 1MB. :P");
}
// Convert the file data back to bytes for writing
Encoding.UTF8.GetBytes(convertedHtml, 0, convertedHtml.Length, fileDataBuffer, 0);
// Need to use FileStream API directly so we can enable async I/O
FileStream destinationFileStream = new FileStream(
destinationFilePath,
FileMode.OpenOrCreate,
FileAccess.Write,
FileShare.None,
8192,
true);
// Use FromAsync to read the data from the file
Task destinationFileStreamWriteTask = Task.Factory.FromAsync(
destinationFileStream.BeginWrite,
destinationFileStream.EndWrite,
fileDataBuffer,
0,
fileDataBuffer.Length,
null);
// Add a continuation that will fire when the async write is completed
destinationFileStreamWriteTask.ContinueWith(destinationFileStreamWriteAntecedent =>
{
try
{
// NOTE: we call wait here to observe any potential exceptions that might have occurred in EndWrite
destinationFileStreamWriteAntecedent.Wait();
}
finally
{
// Always close the destination file stream
destinationFileStream.Close();
destinationFileStream = null;
}
},
TaskContinuationOptions.AttachedToParent);
// Send to external system **concurrent** to writing to destination file system above
SendToWs(ref, convertedHtml);
},
TaskContinuationOptions.AttachedToParent);
});
Now, here's few notes:
This is sample code so I'm using a 1MB buffer to read/write files. This is excessive for HTML files and wasteful of system resources. You can either lower it to suit your max needs or implement chained reads/writes into a StringBuilder which is an excercise I leave up to you since I'd be writing ~500 more lines of code to do async chained reads/writes. :P
You'll note that on the continuations for the read/write tasks I have TaskContinuationOptions.AttachedToParent. This is very important as it will prevent the worker thread that the Parallel::ForEach starts the work with from completing until all the underlying async calls have completed. If this was not here you would kick off work for all 5000 items concurrently which would pollute the TPL subsystem with thousands of scheduled Tasks and not scale properly at all.
I call SendToWs concurrent to writing the file to the file share here. I don't know what is underlying the implementation of SendToWs, but it too sounds like a good candidate for making async. Right now it's assumed it's pure compute work and, as such, is going to burn a CPU thread while executing. I leave it as an excercise to you to figure out how best to leverage what I've shown you to improve throughput there.
This is all typed free form and my brain was the only compiler here and SO's syntax higlighting is all I used to make sure syntax was good. So, please forgive any syntax errors and let me know if I screwed up anything too badly that you can't make heads or tails of it and I'll follow up.

The good news is your logic could be easily separated into steps that go into a producer-consumer pipeline.
Step 1: Read file
Step 2: Parse file
Step 3: Write file
Step 4: SendToWs
If you are using .NET 4.0 you can use the BlockingCollection data structure as the backbone for the each step's producer-consumer queue. The main thread will enqueue each work item into step 1's queue where it will be picked up and processed and then forwarded on to step 2's queue and so on and so forth.
If you are willing to move on to the Async CTP then you can take advantage of the new TPL Dataflow structures for this as well. There is the BufferBlock<T> data structure, among others, that behaves in a similar manner to BlockingCollection and integrates well with the new async and await keywords.
Because your algorithm is IO bound the producer-consumer strategies may not get you the performance boost you are looking for, but at least you will have a very elegant solution that would scale well if you could increase the IO throughput. I am afraid steps 1 and 3 will be the bottlenecks and the pipeline will not balance well, but it is worth experimenting with.

Just a suggestion, but have you looked into the Consumer / Producer pattern ? A certain number of threads would read your files on disk and feed the content to a queue. Then another set of threads, known as the consumers, would "consume" the queue as its filled. http://zone.ni.com/devzone/cda/tut/p/id/3023

Your best bet in these kind of scenario is definitely the producer-consumer model. One thread to pull the data and a bunch of workers to process it. There's no easy way around the I/O so you might as well just focus on optimizing the computation itself.
I will now try to sketch a model:
// producer thread
var refs = GetReferencesFromDB(); // ~5000 Datarow returned
foreach(var ref in refs)
{
lock(queue)
{
queue.Enqueue(ref);
event.Set();
}
// if the queue is limited, test if the queue is full and wait.
}
// consumer threads
while(true)
{
value = null;
lock(queue)
{
if(queue.Count > 0)
{
value = queue.Dequeue();
}
}
if(value != null)
// process value
else
event.WaitOne(); // event to signal that an item was placed in the queue.
}
You can find more details about producer/consumer in part 4 of Threading in C#: http://www.albahari.com/threading/part4.aspx

I think your approach to split up the list of files and process each file in one batch is ok.
My feeling is that you might get more performance gain if you play with degree of parallelism.
See: var refs = GetReferencesFromDB().AsParallel().WithDegreeOfParallelism(16); this would start processing 16 files at the same time. Currently you are processing probably 2 or 4 files depending on number of cores you have. This is only efficient when you have only computation without IO. For IO intensive tasks adjustment might bring incredible performance improvements reducing processor idle time.
If you are going to split up and join tasks back using producer-consumer look at this sample: Using Parallel Linq Extensions to union two sequences, how can one yield the fastest results first?

Can we create 300,000 threads in a C# application and run it on a PC?

I am trying to imitate a scenario where 300,000 consumers are accessing a server. So I am trying to create the pseudo clients, by repeatedly querying the server from the concurrent threads.
But the first hurdle to be cleared is, whether it is possible to run 300,000 threads on a PC? Here is a code which I am using to see intially how many max threads I can get, and later then replace the test function with the actual function:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
namespace CheckThread
{
class Program
{
static int count;
public static void TestThread(int i)
{
while (true)
{
Console.Write("\rThread Executing : {0}", i);
Thread.Sleep(500);
}
}
static void Main(string[] args)
{
count = 0;
int limit = 0;
if (args.Length != 1)
{
Console.WriteLine("Usage CheckThread <number of threads>");
return;
}
else
{
limit = Convert.ToInt32(args[0]);
}
Console.WriteLine();
while (count < limit)
{
ThreadStart newThread = new ThreadStart(delegate { TestThread(count); });
Thread mythread = new Thread(newThread);
mythread.Start();
Console.WriteLine("Thread # {0}", count++);
}
while (true)
{
Thread.Sleep(30*1000);
}
} // end of main
} // end of CheckThread class
} // end of namespace
Now what I am trying might be unrealistic, but still, if there is a way out to do it and you know, then please help me.

Each thread will create its own stack and local storage, you are looking at roughly 512k of stack space per thread on a 32bit OS, I think the stack space doubles on a 64 bit OS. A quick back of the spreadsheet calc gives us 146.484375 gigs of stack space for your 300k clients.
So, no, don't create 300k threads, but rather use the threadpool to simulate 300k requests, although tbh I think you would be better off with several test clients spamming your server through a network interface.
There are a lot of web load-testing tools available. Good starting point : http://www.webperformance.com/library/reports/TestingAspDotNet/

You can alter the maximum nunmber of threads by calling the ThreadPool.SetMaxThreads method. 300,000 threads will probably make your PC explode*
*This is probably an exaggeration

Language-agnostic answer:
The better way to probably go about this is using the Reactor pattern, with a maximum of 1 or 2 concurrent threads per core.

As .net commits the entire stack (1MB) for each clr thread; as Ben says, your PC may actually explode. Or possibly OoM.

Well, what was the result of your test when you tried to create 300K threads? I'm not going to try it on mine!
You could not connect up 300K clients at once anyway because there are not enough sockets available on a single server, (hence farming).
I have done some server testing and, by tweaking the registry to make more sockets available, I have had 24K sockets connected to a server, all one one box. That was somewhat what I was expecting since the server<>client connection requires one socket object at each end and there are only 64K sockets available. I did not attempt to create 24K threads for my testing, I used a client thread class that opened/closed connections on multiple client socket objects in a list.
Rgds,
Martin

How to serialize access to an asynchronous resource?

Can anyone point me to a good working solution to the following problem?
The application I'm working on needs to communicate over TCP to software running on another system. Some of the requests I send to that system can take a long time to complete (up to 15sec).
In my application I have a number of threads, including the main UI thread, which can access the service which communicates with the remote system. There is only a single instance of the service which is accessed by all threads.
I need to only allow a single request to be processed at a time, i.e. it needs to be serialized, otherwise bad things happen with the TCP comms.
Attempted Solutions so far
Initially I tried using lock() with a static object to protect each 'command' method, as follows:
lock (_cmdLock)
{
SetPosition(position);
}
However I found that sometimes it wouldn't release the lock, even though there are timeouts on the remote system and on the TCP comms. Additionally, if two calls came in from the same thread (e.g. a user double clicked a button) then it would get past the lock - after reading up about locking again I know that the same thread won't wait for the lock.
I then tried to use AutoResetEvents to only allow a single call through at a time. But without the locking it wouldn't work with multiple threads. The following is the code I used to send a command (from the calling thread) and process a command request (running in the background on its own thread)
private static AutoResetEvent _cmdProcessorReadyEvent = new AutoResetEvent(false);
private static AutoResetEvent _resultAvailableEvent = new AutoResetEvent(false);
private static AutoResetEvent _sendCommandEvent = new AutoResetEvent(false);
// This method is called to send each command and can run on different threads
private bool SendCommand(Command cmd)
{
// Wait for processor thread to become ready for next cmd
if (_cmdProcessorReadyEvent.WaitOne(_timeoutSec + 500))
{
lock (_sendCmdLock)
{
_currentCommand = cmd;
}
// Tell the processor thread that there is a command present
_sendCommandEvent.Set();
// Wait for a result from the processor thread
if (!_resultAvailableEvent.WaitOne(_timeoutSec + 500))
_lastCommandResult.Timeout = true;
}
return _lastCommandResult.Success;
}
// This method runs in a background thread while the app is running
private void ProcessCommand()
{
try
{
do
{
// Indicate that we are ready to process another commnad
_cmdProcessorReadyEvent.Set();
_sendCommandEvent.WaitOne();
lock (_sendCmdLock)
{
_lastCommandResult = new BaseResponse(false, false, "No Command");
RunCOMCommand(_currentCommand);
}
_resultAvailableEvent.Set();
} while (_processCommands);
}
catch (Exception ex)
{
_lastCommandResult.Success = false;
_lastCommandResult.Timeout = false;
_lastCommandResult.LastError = ex.Message;
}
}
I haven't tried implementing a queue of command requests as the calling code expects everything to be synchronous - i.e. the previous command must have completed before I sent the next one.
Additional Background
The software running on the remote system is a 3rd party product and I don't have access to it, it is used to control a laser marking machine with an integrated XY table.
I'm actually using a legacy VB6 DLL to communicate with the laser as it has all the code for formatting commands and processing the responses. This VB6 DLL uses a WinSock control for the comms.

I'm not sure why a queueing solution wouldn't work.
Why not put each request, plus details for a callback with result, on a queue ? Your application would queue these requests, and the module interfacing to your 3rd party system can take each queue item in turn, process, and return the result.
I think it's a cleaner separation of concerns between modules rather than implementing locking around request dispatch etc. Your requestor is largely oblivious of the serialisation constraints, and the 3rd-party interfacing module can look after serialisation, managing timeouts and other errors etc.
Edit: In the Java world we have BlockingQueues which are synchronised for consumers/publishers and make this sort of thing quite easy. I'm not sure if you have the same in the C# world. A quick search suggests not, but there's source code floating around for this sort of thing (if anyone in the C# world can shed some light that would be appreciated)