We Keep Coding

How to optimize writing to multiple files in the background

My program needs to write very often messages to several files. As it is very time consuming, I need to optimise it. Below, you can find an extract from my program where I try to write async to file in the background. It seems to work, but I am not sure if it is the best practice as I do not dispose tasks (this part is commented). I do not do it because I do not want my program to wait for those tasks completion. Simply, I want my message to be written to few files in the background as quickly as possible. As those files could be accessed by several threads, I added lock.
I use static methods because these methods are used everywhere in my code and I do not want to instantiate this class, just to write one line of message to file, everywhere (maybe that's wrong).
================== Class ==============================================
namespace test
{
public static class MessageLOG
{
private static string debugFileName = Settings.DebugLOGfilename;
private static string logFileName = Settings.LOGfilename;
private static object DebuglockOn = new object();
private static object LoglockOn = new object();
private static StreamWriter DebugSW;
private static StreamWriter LogSW;
private static void DebugFile(string message)
{
uint linesCount = 0;
string _filename = debugFileName;
if(DebugSW == null && !string.IsNullOrEmpty(_filename))
DebugSW = new StreamWriter(_filename);
if(DebugSW != null)
{
lock(DebuglockOn)
{
DebugSW.WriteLine(message);
linesCount++;
if (linesCount > 10)
{
DebugSW.Flush();
linesCount = 0;
}
}
}
}
private static void LogFile(string message)
{
uint linesCount = 0;
string _filename = logFileName;
if(LogSW == null && !string.IsNullOrEmpty(_filename))
LogSW = new StreamWriter(_filename);
if(LogSW != null)
{
lock(LoglockOn)
{
LogSW.WriteLine(string.Format("{0} ({1}): {2}", DateTime.Now.ToShortDateString(), DateTime.Now.ToShortTimeString(), message));
linesCount++;
if (linesCount > 10)
{
LogSW.Flush();
linesCount = 0;
}
}
}
public static void LogUpdate(string message)
{
ThreadPool.QueueUserWorkItem(new WaitCallback( (x) => LogFile(message)));
ThreadPool.QueueUserWorkItem(new WaitCallback( (x) => DebugFile(message)));
ThreadPool.QueueUserWorkItem(new WaitCallback( (x) => Debug.WriteLine(message)));
}
//This method will be called when the main thread is being closed
public static void CloseAllStreams()
{
if (DebugSW != null)
{
DebugSW.Flush();
DebugSW.Close();
}
if (LogSW != null)
{
LogSW.Flush();
LogSW.Close();
}
}
=============== main window ===========
void MainWIndow()
{
... some code ....
MessageLog.LogUpdate("Message text");
... code cont ....
MessageLog.CloseAllStreams();
}

You should re-think your design. Your locks should not be local variables in your method. This is redundant because each method call creates a new object and locks to it. This will not force synchronization across multiple threads (https://msdn.microsoft.com/en-us/library/c5kehkcz(v=vs.80).aspx). Since your methods are static, the locks need to be static variables and you should have a different lock per file. You can use ThreadPool.QueueUserWorkItem (https://msdn.microsoft.com/en-us/library/kbf0f1ct(v=vs.110).aspx) instead of Tasks. ThreadPool is an internal .NET class that re-uses threads to run async operations. This is perfect for your use case because you don't need control over each thread. You just need some async operation to execute and finish on its own.
A better approach would be to create a logger class that runs on its own thread. You can have a queue and enqueue messages from multiple threads and then have the LoggerThread handle writing to the file. This will ensure that only one thread is ever writing to the file. This will also maintain logging order if you use a FIFO queue. You will no longer need to lock writing to the file, but you will need to lock your queue. You can use the .NET Monitor (https://msdn.microsoft.com/en-us/library/system.threading.monitor(v=vs.110).aspx) class to block the LoggerThread until a message is queued (look at methods Enter/Wait/Pulse). To optimize it even more, you can now keep a stream open to the file and push data to it as it gets queued. Since only one thread ever accesses the file, this will be OK. Just remember to close the stream to the file when you are done. You can also set up a timer that goes off once in a while to flush the content. Keeping the stream open is not always recommended, especially if you anticipate other applications attempting to lock the file. However, in this case, it might be OK. This will be a design decision you need to make that fits best with your application.

You´re opening a new stream and committing writings for each write action : very poor performance.
My recommendation is to use only one StreamWriter for each file, this instance must be a class field and you need to still using the lock to ensure is thread safe.
Also this would require that you don't use the using statement in each write method.
Also periodically , maybe every X number of writes, you could make a Stream.Flush to commit writings on disk. This Flush must be protected by the lock.

How to add to a List while using Multi-Threading?

I'm kinda new to Multi-Threading and have only played around with it in the past. But I'm curious if it is possible to have a List of byte arrays on a main thread and still be able to add to that List while creating the new byte array in a seperate Thread. Also, I'll be using a for-each loop that will go through a list of forms that will be used to parse into the byte array. So basically a pseudo code would be like this...
reports = new List();
foreach (form in forms)
{
newReport = new Thread(ParseForm(form));
reports.Add(newReport);
}
void ParseForm(form)
{
newArray = new byte[];
newArray = Convert.ToBytes(form);
return newArray;
}
Hopefully the pseudo-code above makes some sense. If anyone could tell me if this is possible and point me in the direction of a good example, I'm sure I can figure out the actual code.

If you need to access a collection from multiple threads, you should either use synchronization, or use a SynchronizedCollection if your .NET version is 3.0 or higher.
Here is one way to make the collection accessible to your thread:
SynchronizedCollection reports = new SynchronizedCollection();
foreach (form in forms) {
var reportThread = new Thread(() => ParseForm(form, reports));
reportThread.Start();
}
void ParseForm(Form form, SynchronizedCollection reports) {
newArray = new byte[];
newArray = Convert.ToBytes(form);
reports.Add(newArray);
}
If you are on .NET 4 or later, a much better alternative to managing your threads manually is presented by various classes of the System.Threading.Tasks namespace. Consider exploring this alternative before deciding on your threading implementation.

In before we realized it was .Net 3.5, keep for reference on .Net 4
If you don't need any order within the list, an easy "fix" is to use the ConcurrentBag<T> class instead of a list. If you need more order, there is also a ConcurrentQueue<T> collection too.
If you really need something more custom, you can implement your own blocking collection using BlockingCollection<T>. Here's a good article on the topic.
You can also use Parallel.Foreach to avoid the explicit thread creation too:
private void ParseForms()
{
var reports = new ConcurrentBag<byte[]>();
Parallel.ForEach(forms, (form) =>
{
reports.Add(ParseForm(form));
});
}
private byte[] ParseForm(form)
{
newArray = new byte[];
newArray = Convert.ToBytes(form);
return newArray;
}

Why is enumerate files returning the same file more than once?
Check that out. It shows I think exactly what you want to do.
It creates a list on the main thread then adds to it from a different thread.
your going to need
using System.Threading.Tasks
-
Files.Clear(); //List<string>
Task.Factory.StartNew( () =>
{
this.BeginInvoke( new Action(() =>
{
Files.Add("Hi");
}));
});

Below is a simple Blocking Collection (as a queue only) that I just whipped up now since you don't have access to C# 4.0. It's most likely less efficient than the 4.0 concurrent collections, but it should work well enough. I didn't re-implement all of the Queue methods, just enqueue, dequeue, and peek. If you need others and can't figure out how they would be implemented just mention it in the comments.
Once you have the working blocking collection you can simply add to it from the producer threads and remove from it using the consumer threads.
public class MyBlockingQueue<T>
{
private Queue<T> queue = new Queue<T>();
private AutoResetEvent signal = new AutoResetEvent(false);
private object padLock = new object();
public void Enqueue(T item)
{
lock (padLock)
{
queue.Enqueue(item);
signal.Set();
}
}
public T Peek()
{
lock (padLock)
{
while (queue.Count < 1)
{
signal.WaitOne();
}
return queue.Peek();
}
}
public T Dequeue()
{
lock (padLock)
{
while (queue.Count < 1)
{
signal.WaitOne();
}
return queue.Dequeue();
}
}
}

Thread-safe buffer for .NET

(Note: Though I would like ideas for the future for .Net 4.0, I'm limited to .Net 3.5 for this project.)
I have a thread, which is reading data asynchronously from an external device (simulated in the code example by the ever-so-creative strSomeData :-) and storing it in a StringBuilder 'buffer' (strBuilderBuffer :-)
In the 'main code' I want to 'nibble' at this 'buffer'. However, I am unsure as to how to do this in a thread safe manner, from a 'operational' perspective. I understand it is safe from a 'data' perspective, because according to msdn, "Any public static members of this (StringBuilder) type are thread safe. Any instance members are not guaranteed to be thread safe." However, my code below illustrates that it is possibly not thread-safe from an 'operational' perspective.
The key is that I'm worried about two lines of the code:
string strCurrentBuffer = ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.ToString();
// Thread 'randomly' slept due to 'inconvenient' comp resource scheduling...
ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.Length = 0;
if the computer OS sleeps my thread between the 'reading' of the buffer & the 'clearing' of the buffer, I can lose data (which is bad :-(
Is there any way to guarantee the 'atomocy?' of those two lines & force the computer to not interrupt them?
With respect to Vlad's suggestion below regarding the use of lock, I tried it but it didn't work (at all really):
public void BufferAnalyze()
{
String strCurrentBuffer;
lock (ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer)
{
strCurrentBuffer = ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.ToString();
Console.WriteLine("[BufferAnalyze()] ||<< Thread 'Randomly' Slept due to comp resource scheduling");
Thread.Sleep(1000); // Simulate poor timing of thread resourcing...
ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.Length = 0;
}
Console.WriteLine("[BufferAnalyze()]\r\nstrCurrentBuffer[{0}] == {1}", strCurrentBuffer.Length.ToString(), strCurrentBuffer);
}
Is there a better way of implementing a thread safe buffer?
Here's the full code:
namespace ExploringThreads
{
/// <summary>
/// Description of BasicThreads_TestThreadSafety_v1a
/// </summary>
class ThreadWorker_TestThreadSafety_v1a
{
private Thread thread;
public static StringBuilder strBuilderBuffer = new StringBuilder("", 7500);
public static StringBuilder strBuilderLog = new StringBuilder("", 7500);
public bool IsAlive
{
get { return thread.IsAlive; }
}
public ThreadWorker_TestThreadSafety_v1a(string strThreadName)
{
// It is possible to have a thread begin execution as soon as it is created.
// In the case of MyThread this is done by instantiating a Thread object inside MyThread's constructor.
thread = new Thread(new ThreadStart(this.threadRunMethod));
thread.Name = strThreadName;
thread.Start();
}
public ThreadWorker_TestThreadSafety_v1a() : this("")
{
// NOTE: constructor overloading ^|^
}
//Entry point of thread.
public void threadRunMethod()
{
Console.WriteLine("[ThreadWorker_TestThreadSafety_v1a threadRunMethod()]");
Console.WriteLine(thread.Name + " starting.");
int intSomeCounter = 0;
string strSomeData = "";
do
{
Console.WriteLine("[ThreadWorker_TestThreadSafety_v1a threadRunMethod()] running.");
intSomeCounter++;
strSomeData = "abcdef" + intSomeCounter.ToString() + "|||";
strBuilderBuffer.Append(strSomeData);
strBuilderLog.Append(strSomeData);
Thread.Sleep(200);
} while(intSomeCounter < 15);
Console.WriteLine(thread.Name + " terminating.");
}
}
/// <summary>
/// Description of BasicThreads_TestThreadSafety_v1a.
/// </summary>
public class BasicThreads_TestThreadSafety_v1a
{
public BasicThreads_TestThreadSafety_v1a()
{
}
public void BufferAnalyze()
{
string strCurrentBuffer = ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.ToString();
Console.WriteLine("[BufferAnalyze()] ||<< Thread 'Randomly' Slept due to comp resource scheduling");
Thread.Sleep(1000); // Simulate poor timing of thread resourcing...
ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.Length = 0;
Console.WriteLine("[BufferAnalyze()]\r\nstrCurrentBuffer[{0}] == {1}", strCurrentBuffer.Length.ToString(), strCurrentBuffer);
}
public void TestBasicThreads_TestThreadSafety_v1a()
{
Console.Write("Starting TestBasicThreads_TestThreadSafety_v1a >>> Press any key to continue . . . ");
Console.Read();
// First, construct a MyThread object.
ThreadWorker_TestThreadSafety_v1a threadWorker_TestThreadSafety_v1a = new ThreadWorker_TestThreadSafety_v1a("threadWorker_TestThreadSafety_v1a Child");
do
{
Console.WriteLine("[TestBasicThreads_TestThreadSafety_v1a()]");
Thread.Sleep(750);
BufferAnalyze();
//} while (ThreadWorker_TestThreadSafety_v1a.thread.IsAlive);
} while (threadWorker_TestThreadSafety_v1a.IsAlive);
BufferAnalyze();
Thread.Sleep(1250);
Console.WriteLine("[TestBasicThreads_TestThreadSafety_v1a()]");
Console.WriteLine("ThreadWorker_TestThreadSafety_v1a.strBuilderLog[{0}] == {1}", ThreadWorker_TestThreadSafety_v1a.strBuilderLog.Length.ToString(), ThreadWorker_TestThreadSafety_v1a.strBuilderLog);
Console.Write("Completed TestBasicThreads_TestThreadSafety_v1a >>> Press any key to continue . . . ");
Console.Read();
}
}
}

Download the Reactive Extensions backport for 3.5 here. There is also a NuGet package for it. After you have it downloaded then just reference System.Threading.dll in your project.
Now you can use all of the new concurrent collections standard in .NET 4.0 within .NET 3.5 as well. The best one for your situation is the BlockingCollection. It is basically a buffer that allows threads to enqueue items and dequeue them like a normal queue. Except that the dequeue operation blocks until an item is available.
There is no need to use the StringBuilder class at all now. Here is how I would refactor your code. I tried to keep my example short so that it is easier to understand.
public class Example
{
private BlockingCollection<string> buffer = new BlockingCollection<string>();
public Example()
{
new Thread(ReadFromExternalDevice).Start();
new Thread(BufferAnalyze).Start();
}
private void ReadFromExteneralDevice()
{
while (true)
{
string data = GetFromExternalDevice();
buffer.Add(data);
Thread.Sleep(200);
}
}
private void BufferAnalyze()
{
while (true)
{
string data = buffer.Take(); // This blocks if nothing is in the queue.
Console.WriteLine(data);
}
}
}
For future reference the BufferBlock<T> class from the TPL Data Flow library will do basically the same thing as BlockingCollection. It will be available in .NET 4.5.

Using StringBuffer is not thread safe, but you can switch to ConcurrentQueue<char>.
In case you need other data structure, there are other thread-safe collections in .NET 4, see http://msdn.microsoft.com/en-us/library/dd997305.aspx.
Edit: in .NET 3.5 there are less synchronization primitives. You can make a simple solution by adding a lock around Queue<char>, though it will be less efficient than the .NET 4's ConcurrentQueue. Or use the same StrignBuffer, again with locking reading/writing operations:
public static StringBuilder strBuilderBuffer = new StringBuilder("", 7500);
private object BufferLock = new object();
...
lock (BufferLock)
strBuilderBuffer.Append(strSomeData);
...
string strCurrentBuffer;
lock (BufferLock)
{
strCurrentBuffer = ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.ToString();
ThreadWorker_TestThreadSafety_v1a.strBuilderBuffer.Clear();
}
Console.WriteLine("[BufferAnalyze()] ||<< Thread 'Randomly' Slept ...");
Thread.Sleep(1000); // Simulate poor timing of thread resourcing...
Edit:
You cannot guarantee that the OS won't suspend your working thread which is holding the lock. However the lock guarantees that the other threads will be unable to interfere and change the buffer as long as one thread is processing it.
That's why your time of holding the lock should be as short as possible:
taken the lock, added data, released the lock, -or-
taken the lock, copied data, emptied the buffer, released the lock, started processing the copied data.

If you are doing a lot of reads out of the buffer, perhaps this will help:
http://msdn.microsoft.com/en-us/library/system.threading.readerwriterlock.aspx
Multiple readers are possible, but only one writer.
It is available in .NET 1.X and up...

AutoResetEvent Reset Method

super simple question, but I just wanted some clarification. I want to be able to restart a thread using AutoResetEvent, so I call the following sequence of methods to my AutoResetEvent.
setupEvent.Reset();
setupEvent.Set();
I know it's really obvious, but MSDN doesn't state in their documentation that the Reset method restarts the thread, just that it sets the state of the event to non-signaled.
UPDATE:
Yes the other thread is waiting at WaitOne(), I'm assuming when it gets called it will resume at the exact point it left off, which is what I don't want, I want it to restart from the beginning. The following example from this valuable resource illustrates this:
static void Main()
{
new Thread (Work).Start();
_ready.WaitOne(); // First wait until worker is ready
lock (_locker) _message = "ooo";
_go.Set(); // Tell worker to go
_ready.WaitOne();
lock (_locker) _message = "ahhh"; // Give the worker another message
_go.Set();
_ready.WaitOne();
lock (_locker) _message = null; // Signal the worker to exit
_go.Set();
}
static void Work()
{
while (true)
{
_ready.Set(); // Indicate that we're ready
_go.WaitOne(); // Wait to be kicked off...
lock (_locker)
{
if (_message == null) return; // Gracefully exit
Console.WriteLine (_message);
}
}
}
If I understand this example correctly, notice how the Main thread will resume where it left off when the Work thread signals it, but in my case, I would want the Main thread to restart from the beginning.
UPDATE 2:
#Jaroslav Jandek - It's quite involved, but basically I have a CopyDetection thread that runs a FileSystemWatcher to monitor a folder for any new files that are moved or copied into it. My second thread is responsible for replicating the structure of that particular folder into another folder. So my CopyDetection thread has to block that thread from working while a copy/move operation is in progress. When the operation completes, the CopyDetection thread restarts the second thread so it can re-duplicate the folder structure with the newly added files.
UPDATE 3:
#SwDevMan81 - I actually didn't think about that and that would work save for one caveat. In my program, the source folder that is being duplicated is emptied once the duplication process is complete. That's why I have to block and restart the second thread when new items are added to the source folder, so it can have a chance to re-parse the folder's new structure properly.
To address this, I'm thinking of maybe adding a flag that signals that it is safe to delete the source folder's contents. Guess I could put the delete operation on it's own Cleanup thread.
#Jaroslav Jandek - My apologies, I thought it would be a simple matter to restart a thread on a whim. To answer your questions, I'm not deleting the source folder, only it's content, it's a requirement by my employer that unfortunately I cannot change. Files in the source folder are getting moved, but not all of them, only files that are properly validated by another process, the rest must be purged, i.e. the source folder is emptied. Also, the reason for replicating the source folder structure is that some of the files are contained within a very strict sub-folder hierarchy that must be preserved in the destination directory. Again sorry for making it complicated. All of these mechanisms are in place, have been tested and are working, which is why I didn't feel the need to elaborate on them. I only need to detect when new files are added so I may properly halt the other processes while the copy/move operation is in progress, then I can safely replicate the source folder structure and resume processing.

So thread 1 monitors and thread 2 replicates while other processes modify the monitored files.
Concurrent file access aside, you can't continue replicating after a change. So a successful replication only occurs when there is long enough delay between modifications. Replication cannot be stopped immediately since you replicate in chunks.
So the result of monitoring should be a command (file copy, file delete, file move, etc.).
The result of a successful replication should be an execution of a command.
Considering multiple operations can occur, you need a queue (or queued dictionary - to only perform 1 command on a file) of commands.
// T1:
somethingChanged(string path, CT commandType)
{
commandQueue.AddCommand(path, commandType);
}
// T2:
while (whatever)
{
var command = commandQueue.Peek();
if (command.Execute()) commandQueue.Remove();
else // operation failed, do what you like.
}
Now you may ask how to create a thread-safe query, but that probably belongs to another question (there are many implementations on the web).
EDIT (queue-less version with whole dir replication - can be used with query):
If you do not need multiple operations (eg. always replication the whole directory) and expect the replication to always finish or fail and cancel, you can do:
private volatile bool shouldStop = true;
// T1:
directoryChanged()
{
// StopReplicating
shouldStop = true;
workerReady.WaitOne(); // Wait for the worker to stop replicating.
// StartReplicating
shouldStop = false;
replicationStarter.Set();
}
// T2:
while (whatever)
{
replicationStarter.WaitOne();
... // prepare, throw some shouldStops so worker does not have to work too much.
if (!shouldStop)
{
foreach (var file in files)
{
if (shouldStop) break;
// Copy the file or whatever.
}
}
workerReady.Set();
}

I think this example clarifies (to me anyway) how reset events work:
var resetEvent = new ManualResetEvent(false);
var myclass = new MyAsyncClass();
myclass.MethodFinished += delegate
{
resetEvent.Set();
};
myclass.StartAsyncMethod();
resetEvent.WaitOne(); //We want to wait until the event fires to go on
Assume that MyAsyncClass runs the method on a another thread and fires the event when complete.
This basically turns the asynchronous "StartAsyncMethod" into a synchronous one. Many times I find a real-life example more useful.
The main difference between AutoResetEvent and ManualResetEvent, is that using AutoResetEvent doesn't require you to call Reset(), but automatically sets the state to "false". The next call to WaitOne() blocks when the state is "false" or Reset() has been called.

You just need to make it loop like the other Thread does. Is this what you are looking for?
class Program
{
static AutoResetEvent _ready = new AutoResetEvent(false);
static AutoResetEvent _go = new AutoResetEvent(false);
static Object _locker = new Object();
static string _message = "Start";
static AutoResetEvent _exitClient = new AutoResetEvent(false);
static AutoResetEvent _exitWork = new AutoResetEvent(false);
static void Main()
{
new Thread(Work).Start();
new Thread(Client).Start();
Thread.Sleep(3000); // Run for 3 seconds then finish up
_exitClient.Set();
_exitWork.Set();
_ready.Set(); // Make sure were not blocking still
_go.Set();
}
static void Client()
{
List<string> messages = new List<string>() { "ooo", "ahhh", null };
int i = 0;
while (!_exitClient.WaitOne(0)) // Gracefully exit if triggered
{
_ready.WaitOne(); // First wait until worker is ready
lock (_locker) _message = messages[i++];
_go.Set(); // Tell worker to go
if (i == 3) { i = 0; }
}
}
static void Work()
{
while (!_exitWork.WaitOne(0)) // Gracefully exit if triggered
{
_ready.Set(); // Indicate that we're ready
_go.WaitOne(); // Wait to be kicked off...
lock (_locker)
{
if (_message != null)
{
Console.WriteLine(_message);
}
}
}
}
}

Multi-threaded application interaction with logger thread

Here I am again with questions about multi-threading and an exercise of my Concurrent Programming class.
I have a multi-threaded server - implemented using .NET Asynchronous Programming Model - with GET (download) and PUT (upload) file services. This part is done and tested.
It happens that the statement of the problem says this server must have logging activity with the minimum impact on the server response time, and it should be supported by a low priority thread - logger thread - created for this effect. All logging messages shall be passed by the threads that produce them to this logger thread, using a communication mechanism that may not lock the thread that invokes it (besides the necessary locking to ensure mutual exclusion) and assuming that some logging messages may be ignored.
Here is my current solution, please help validating if this stands as a solution to the stated problem:
using System;
using System.IO;
using System.Threading;
// Multi-threaded Logger
public class Logger {
// textwriter to use as logging output
protected readonly TextWriter _output;
// logger thread
protected Thread _loggerThread;
// logger thread wait timeout
protected int _timeOut = 500; //500ms
// amount of log requests attended
protected volatile int reqNr = 0;
// logging queue
protected readonly object[] _queue;
protected struct LogObj {
public DateTime _start;
public string _msg;
public LogObj(string msg) {
_start = DateTime.Now;
_msg = msg;
}
public LogObj(DateTime start, string msg) {
_start = start;
_msg = msg;
}
public override string ToString() {
return String.Format("{0}: {1}", _start, _msg);
}
}
public Logger(int dimension,TextWriter output) {
/// initialize queue with parameterized dimension
this._queue = new object[dimension];
// initialize logging output
this._output = output;
// initialize logger thread
Start();
}
public Logger() {
// initialize queue with 10 positions
this._queue = new object[10];
// initialize logging output to use console output
this._output = Console.Out;
// initialize logger thread
Start();
}
public void Log(string msg) {
lock (this) {
for (int i = 0; i < _queue.Length; i++) {
// seek for the first available position on queue
if (_queue[i] == null) {
// insert pending log into queue position
_queue[i] = new LogObj(DateTime.Now, msg);
// notify logger thread for a pending log on the queue
Monitor.Pulse(this);
break;
}
// if there aren't any available positions on logging queue, this
// log is not considered and the thread returns
}
}
}
public void GetLog() {
lock (this) {
while(true) {
for (int i = 0; i < _queue.Length; i++) {
// seek all occupied positions on queue (those who have logs)
if (_queue[i] != null) {
// log
LogObj obj = (LogObj)_queue[i];
// makes this position available
_queue[i] = null;
// print log into output stream
_output.WriteLine(String.Format("[Thread #{0} | {1}ms] {2}",
Thread.CurrentThread.ManagedThreadId,
DateTime.Now.Subtract(obj._start).TotalMilliseconds,
obj.ToString()));
}
}
// after printing all pending log's (or if there aren't any pending log's),
// the thread waits until another log arrives
//Monitor.Wait(this, _timeOut);
Monitor.Wait(this);
}
}
}
// Starts logger thread activity
public void Start() {
// Create the thread object, passing in the Logger.Start method
// via a ThreadStart delegate. This does not start the thread.
_loggerThread = new Thread(this.GetLog);
_loggerThread.Priority = ThreadPriority.Lowest;
_loggerThread.Start();
}
// Stops logger thread activity
public void Stop() {
_loggerThread.Abort();
_loggerThread = null;
}
// Increments number of attended log requests
public void IncReq() { reqNr++; }
}
Basically, here are the main points of this code:
Start a low priority thread that loops the logging queue and prints pending logs to the output. After this, the thread is suspended till new log arrives;
When a log arrives, the logger thread is awaken and does it's work.
Is this solution thread-safe? I have been reading Producers-Consumers problem and solution algorithm, but in this problem although I have multiple producers, I only have one reader.

It seems it should be working. Producers-Consumers shouldn't change greatly in case of single consumer. Little nitpicks:
acquiring lock may be an expensive operation (as #Vitaliy Lipchinsky says). I'd recommend to benchmark your logger against naive 'write-through' logger and logger using interlocked operations. Another alternative would be exchanging existing queue with empty one in GetLog and leaving critical section immediately. This way none of producers won't be blocked by long operations in consumers.
make LogObj reference type (class). There's no point in making it struct since you are boxing it anyway. or else make _queue field to be of type LogObj[] (that's better anyway).
make your thread background so that it won't prevent closing your program if Stop won't be called.
Flush your TextWriter. Or else you are risking to lose even those records that managed to fit queue (10 items is a bit small IMHO)
Implement IDisposable and/or finalizer. Your logger owns thread and text writer and those should be freed (and flushed - see above).

While it appears to be thread-safe, I don't believe it is particularly optimal. I would suggest a solution along these lines
NOTE: just read the other responses. What follows is a fairly optimal, optimistic locking solution based on your own. Major differences is locking on an internal class, minimizing 'critical sections', and providing graceful thread termination. If you want to avoid locking altogether, then you can try some of that volatile "non-locking" linked list stuff as #Vitaliy Lipchinsky suggests.
using System.Collections.Generic;
using System.Linq;
using System.Threading;
...
public class Logger
{
// BEST PRACTICE: private synchronization object.
// lock on _syncRoot - you should have one for each critical
// section - to avoid locking on public 'this' instance
private readonly object _syncRoot = new object ();
// synchronization device for stopping our log thread.
// initialized to unsignaled state - when set to signaled
// we stop!
private readonly AutoResetEvent _isStopping =
new AutoResetEvent (false);
// use a Queue<>, cleaner and less error prone than
// manipulating an array. btw, check your indexing
// on your array queue, while starvation will not
// occur in your full pass, ordering is not preserved
private readonly Queue<LogObj> _queue = new Queue<LogObj>();
...
public void Log (string message)
{
// you want to lock ONLY when absolutely necessary
// which in this case is accessing the ONE resource
// of _queue.
lock (_syncRoot)
{
_queue.Enqueue (new LogObj (DateTime.Now, message));
}
}
public void GetLog ()
{
// while not stopping
//
// NOTE: _loggerThread is polling. to increase poll
// interval, increase wait period. for a more event
// driven approach, consider using another
// AutoResetEvent at end of loop, and signal it
// from Log() method above
for (; !_isStopping.WaitOne(1); )
{
List<LogObj> logs = null;
// again lock ONLY when you need to. because our log
// operations may be time-intensive, we do not want
// to block pessimistically. what we really want is
// to dequeue all available messages and release the
// shared resource.
lock (_syncRoot)
{
// copy messages for local scope processing!
//
// NOTE: .Net3.5 extension method. if not available
// logs = new List<LogObj> (_queue);
logs = _queue.ToList ();
// clear the queue for new messages
_queue.Clear ();
// release!
}
foreach (LogObj log in logs)
{
// do your thang
...
}
}
}
}
...
public void Stop ()
{
// graceful thread termination. give threads a chance!
_isStopping.Set ();
_loggerThread.Join (100);
if (_loggerThread.IsAlive)
{
_loggerThread.Abort ();
}
_loggerThread = null;
}

Actually, you ARE introducing locking here. You have locking while pushing a log entry to the queue (Log method): if 10 threads simultaneously pushed 10 items into queue and woke up the Logger thread, then 11th thread will wait until the logger thread log all items...
If you want something really scalable - implement lock-free queue (example is below). With lock-free queue synchronization mechanism will be really straightaway (you can even use single wait handle for notifications).
If you won't manage to find lock-free queue implementation in the web, here is an idea how to do this:
Use linked list for an implementation. Each node in linked list contains a value and a volatile reference to the next node. therefore for operations enqueue and dequeue you can use Interlocked.CompareExchange method. I hope, the idea is clear. If not - let me know and I'll provide more details.

I'm just doing a thought experiment here, since I don't have time to actually try code right now, but I think you can do this without locks at all if you're creative.
Have your logging class contain a method that allocates a queue and a semaphore each time it's called (and another that deallocates the queue and semaphore when the thread is done). The threads that want to do logging will call this method when they start. When they want to log, they push the message onto their own queue and set the semaphore. The logger thread has a big loop that runs through the queues and checks the associated semaphores. If the semaphore associated with the queue is greater than zero, then the queue gets popped off and the semaphore decremented.
Because you're not attempting to pop things off the queue until after the semaphore is set, and you're not setting the semaphore until after you've pushed things onto the queue, I think this will be safe. According to the MSDN documentation for the queue class, if you are enumerating the queue and another thread modifies the collection, an exception is thrown. Catch that exception and you should be good.