We Keep Coding

C# How to let processes with different speeds work together

In the bellow test scenario i like to trigger some task by using multiple timers. Some event can trigger another event.
An event must finish the process, before a new process can be started. Events that gets triggered, while another event is processing, shall queue up and start once nothing is processing. The timer doesn't need to be accurate.
Once a line has executed the code, which takes just few seconds, the line cant take any new orders for minutes. Thats the purpose im using timers.
The current problem on the code bellow, is that things are getting mixed up in the real App. Line2 starts processing, while Line still hasn't finished. How to make the orders queue up properly and process it?
In the real App MyTask will start to run the first lines of code back and forth, after a while the last lines of the MyTask code will be executed.
Im a beginner, so please be patient.
public partial class Form1 : Form
{
readonly System.Windows.Forms.Timer myTimer1 = new System.Windows.Forms.Timer();
readonly System.Windows.Forms.Timer myTimer2 = new System.Windows.Forms.Timer();
int leadTime1 = 100;
int leadTime2 = 100;
public Form1()
{
InitializeComponent();
TaskStarter();
}
private void TaskStarter()
{
myTimer1.Tick += new EventHandler(myEventTimer1);
myTimer2.Tick += new EventHandler(myEventTimer2);
myTimer1.Interval = leadTime1;
myTimer2.Interval = leadTime2;
myTimer1.Start();
}
private void myEventTimer1(object source, EventArgs e)
{
myTimer1.Stop();
Console.WriteLine("Line1 Processing ");
MyTask();
Console.Write(" Line1 Completed");
leadTime1.Interval = 5000; // this leadtime is variable and will show how long the line cant be used again, after the code is executed
myTimer2.Start();
myTimer1.Enabled = true;
}
private void myEventTimer2(object source, EventArgs e)
{
myTimer2.Stop();
Console.WriteLine("Line2 Processing ");
MyTask();
Console.Write(" Line2 Completed");
leadTime2.Interval = 5000; // this leadtime is variable
myTimer2.Enabled = true;
}
private void MyTask()
{
Random rnd = new Random();
int timeExecuteCode = rnd.Next(1000, 5000); // This leadtime does reflect the execution of the real code
Thread.Sleep(timeExecuteCode );
}
}
Update
Thanks to the input i was able to sort the problems, which made me remove all the timers as they were causing the asynchronous task processing. I not just lock the Lines to a while loop till all orders are completed. All is done in a single Thread. I think for the most Pro my code will look very ugly. This solution is understandable with my 4 weeks C# experience :)
The 2 List i use and the properties
public class Orders
{
public string OrderID { get ; set ; }
public Orders(string orderID) { OrderID = orderID; }
}
public class LineData
{
string lineID;
public string LineID { get { return lineID; } set { lineID = value; } }
private string orderId;
public string OrderID { get { return orderId; } set { orderId = value; } }
public string ID { get { return lineID + OrderID; } private set {; } }
public double TaskTime { get; set; }
}
Creating the Line data with the lead times per Line and Part
Adding some sample orders
while loop till all orders are completed
public class Production
{
readonly static List<LineData> listLineData = new List<LineData>();
readonly static List<Orders> listOrders = new List<Orders>();
static void Main()
{
// List Line Processing Master Data
listLineData.Add(new LineData { LineID = "Line1", OrderID = "SubPart1", TaskTime = 3 });
listLineData.Add(new LineData { LineID = "Line1", OrderID = "SubPart2", TaskTime = 3 });
listLineData.Add(new LineData { LineID = "Line2", OrderID = "Part1", TaskTime = 1 });
listLineData.Add(new LineData { LineID = "Line3", OrderID = "Part1", TaskTime = 1 });
listLineData.Add(new LineData { LineID = "Line3", OrderID = "Part2", TaskTime = 2 });
// Create Order Book
listOrders.Add(new Orders("SubPart1"));
listOrders.Add(new Orders("SubPart2"));
listOrders.Add(new Orders("Part1"));
listOrders.Add(new Orders("Part2"));
listOrders.Add(new Orders("SubPart1"));
listOrders.Add(new Orders("SubPart2"));
listOrders.Add(new Orders("Part1"));
listOrders.Add(new Orders("Part2"));
listOrders.Add(new Orders("SubPart1"));
listOrders.Add(new Orders("SubPart2"));
listOrders.Add(new Orders("Part1"));
listOrders.Add(new Orders("Part2"));
while (listOrders.Count > 0)
{
CheckProductionLines();
Thread.Sleep(100)
}
}
Picking orders from the listOrder and assign them to the correct Line.
Using DateTime.Now and add the taskTime to determine whether a line is busy or not
Sending the orders to void InitializeProduction(int indexOrder, string line) to process the order.
In a later step im going to make a function for Line1-Linex, as it is repetitive.
static DateTime timeLine1Busy = new DateTime();
static DateTime timeLine2Busy = new DateTime();
static DateTime timeLine3Busy = new DateTime();
static void CheckProductionLines()
{
// Line 1
int indexOrderLine1 = listOrders.FindIndex(x => x.OrderID == "SubPart1" || x.OrderID == "SubPart2");
if (indexOrderLine1 >= 0 && timeLine1Busy < DateTime.Now)
{
string id = "Line1" + listOrders[indexOrderLine1].OrderID.ToString();// Construct LineID (Line + Part) for Task
int indexTasktime = listLineData.FindIndex(x => x.ID == id); // Get Index LineData where the tasktime is stored
double taskTime = (listLineData[indexTasktime].TaskTime); // Get the Task Time for the current order (min.)
InitializeProduction(indexOrderLine1, "Line1"); // Push the start button to run the task
timeLine1Busy = DateTime.Now.AddSeconds(taskTime); // Set the Line to busy
}
// Line2
int indexOrderLine2 = listOrders.FindIndex(x => x.OrderID == "Part1"); // Pick order Line2
if (indexOrderLine2 >= 0 && timeLine2Busy < DateTime.Now)
{
string id = "Line2" + listOrders[indexOrderLine2].OrderID.ToString(); // Line2 + Order is unique ID in listLineData List
int indexTasktime = listLineData.FindIndex(x => x.ID == id);// Get Index LineData where the tasktime is stored
double taskTime = (listLineData[indexTasktime].TaskTime); // Get the Task Time for the current order (min.)
InitializeProduction(indexOrderLine2, "Line2"); // Push the start button to run the task
timeLine2Busy = DateTime.Now.AddSeconds(taskTime); // Set the Line to busy
}
// Line 3
int indexOrderLine3 = listOrders.FindIndex(x => x.OrderID == "Part1" || x.OrderID == "Part2"); // Pick order
if (indexOrderLine3 >= 0 && timeLine3Busy < DateTime.Now)
{
string id = "Line3" + listOrders[indexOrderLine3].OrderID.ToString(); // Line3 + Order is unique ID in listLineData List
int indexTasktime = listLineData.FindIndex(x => x.ID == id);// Get Index LineData where the tasktime is stored
double taskTime = (listLineData[indexTasktime].TaskTime); // Get the Task Time for the current order (min.)
InitializeProduction(indexOrderLine3, "Line3"); // Push the start button to run the task
timeLine3Busy = DateTime.Now.AddSeconds(taskTime); // Set the Line to busy
}
}
Here i InitializeProduction the production
Remove the order from listOrders
in real here will be processed many tasks
static void InitializeProduction(int indexOrder, string line)
{
Thread.Sleep(1000); //simulates the inizialsation code
Debug.WriteLine($"{line} {listOrders[indexOrder].OrderID} Completed ");
listOrders.RemoveAt(indexOrder); //Remove Order from List
}
}
Im sure you will see a lot of space for improvement. If simple things can or even must be applied, im listening :)

Addition after comments at the end
Your problem screams for a producer-consumer pattern. This lesser known pattern has a producer who produces things that a consumer consumes.
The speed in which the producer produces items can be different than the speed in which the consumer can consume. Sometimes the producer produces faster, sometimes the producer produces slower.
In your case, the producer produces "requests to execute a task". The consumer will execute a task one at a time.
For this I use Nuget package: Microsoft.Tpl.Dataflow. It can do a lot more, but in your case, usage is simple.
Normally there are a lot of multi-threading issues you have to think about, like critical sections in the send-receive buffer. TPL will handle them for your.
If the Producer is started, it produces requests to do something, to execute and await an Action<Task>. The producer will these requests in a BufferBlock<Action<Task>>. It will produce as fast a possible.
First a factory, that will create Action<Task> with random execution time. Note that every created action is not executed yet, thus the task is not running!
class ActionFactory
{
private readonly Random rnd = new Random();
public Action<Task> Create()
{
TimeSpan timeExecuteCode = TimeSpan.FromMilliseconds(rnd.Next(1000, 5000));
return _ => Task.Delay(timeExecuteCode);
// if you want, you can use Thread.Sleep
}
}
The producer is fairly simple:
class Producer
{
private readonly BufferBlock<Action<Task>> buffer = new BufferBlock<Action<Task>>();
public TaskFactory TaskFactory {get; set;}
public ISourceBlock<Action<Task> ProducedActions => buffer;
public async Task ProduceAsync()
{
// Create several tasks and put them on the buffer
for (int i=0; i<10; ++i)
{
Action<Task> createdAction = this.TaskFactory.Create();
await this.buffer.SendAsync(createdAction);
}
// notice listeners to my output that I won't produce anything anymore
this.buffer.Complete();
}
If you want, you can optimize this: while SendAsync, you could create the next action. then await SendAsync task, before sending the next action. For simplicity I didn't do this.
The Consumer needs an input, that accepts Action<Task> objects. It will read this input, execute the action and wait until the action is completed before fetching the next input from the buffer.
class Consumer
{
public ISourceBlock<Action<Task>> ActionsToConsume {get; set;}
public async Task ConsumeAsync()
{
// wait until the producer has produced something,
// or says that nothing will be produced anymore
while (await this.ActionsToConsume.OutputAvailableAsync())
{
// the Producer has produced something; fetch it
Action<Task> actionToExecute = this.ActionsToConsume.ReceiveAsync();
// execute the action, and await the eturned Task
await actionToExecute();
// wait until Producer produces a new action.
}
// if here: producer notifies completion: nothing is expected anymore
}
Put it all together:
TaskFactory factory = new TaskFactory();
Producer producer = new Producer
{
TaskFactory = factory;
}
Consumer consumer = new Consumer
{
Buffer = producer.ProducedActions;
}
// Start Producing and Consuming and wait until everything is ready
var taskProduce = producer.ProduceAsync();
var taskConsume = consumer.ConsumeAsync();
// now producer is happily producing actions and sending them to the consumer.
// the consumer is waiting for actions to consume
// await until both tasks are finished:
await Task.WhenAll(new Task[] {taskProduce, taskConsume});
Addition after comment: do it with less code
The above seems a lot of work. I created separate classes, so you could see who is responsible for what. If you want, you can do it all with one buffer and two methods: a method that produces and a method that consumes:
private readonly BufferBlock<Action<Task>> buffer = new BufferBlock<Action<Task>>();
public async Task ProduceTasksAsync()
{
// Create several tasks and put them on the buffer
for (int i=0; i<10; ++i)
{
Action<Task> createdAction = ...
await this.buffer.SendAsync(createdAction);
}
// producer will not produce anything anymore:
buffer.Complete();
}
async Task ConsumeAsync()
{
while (await this.ActionsToConsume.OutputAvailableAsync())
{
// the Producer has produced something; fetch it, execute it
Action<Task> actionToExecute = this.ActionsToConsume.ReceiveAsync();
await actionToExecute();
}
}
Usage:
async Task ProduceAndConsumeAsync()
{
var taskProduce = producer.ProduceAsync();
var taskConsume = consumer.ConsumeAsync();
await Task.WhenAll(new Task[] {taskProduce, taskConsume});
}

Your problem is that both timers run on the same UI event loop. that means that while timer1 is doing it's event no other events are executed on that thread. The solution to this is to use tha async await pattern that runs code in the background in your case you can do something like this:
private async void myEventTimer1(object source, EventArgs e)
{
myTimer1.Stop();
Console.WriteLine("Line1 Processing ");
await MyTask();
Console.Write(" Line1 Completed");
myTimer1.Interval = 5000; // this leadtime is variable
myTimer2.Start();
myTimer1.Enabled = true;
}
private async void myEventTimer2(object source, EventArgs e)
{
myTimer2.Stop();
Console.WriteLine("Line2 Processing ");
await MyTask();
Console.Write(" Line2 Completed");
myTimer2.Interval = 5000; // this leadtime is variable
myTimer2.Enabled = true;
}
private async Task MyTask()
{
Random rnd = new Random();
int tleadtime = rnd.Next(1000, 5000);
await Task.Delay(tleadtime);
}
This runs MyTask (really just the Delay part) in the background, but continues in the foreground once it is complete.

Now to be clear, this isn't technically an answer to your question as you have asked it, but I believe it will produce the underlying behavior that you asking for (in comments), and I believe in helping people.
We have three classes, Order, Line and Factory written in Console Application as an example.
Order is straight forward, it has two properties, an identifying name, and a leadtime in seconds.
public class Order
{
public string OrderName { get; set; }
public int LeadTimeSeconds { get; set; }
public Order(string orderName, int leadTimeSeconds)
{
OrderName = orderName;
LeadTimeSeconds = leadTimeSeconds;
}
}
Line inherits from a BackgroundWorker MSDN - BackgroundWorker. I won't go into detail here as there are many posts on the subject, but you may delegate to the DoWork event that is invoked asynchronously. They allow you to do something continuously (or prolonged periods) without blocking behaviors since they expose a CancelAsync() method. Line also has reference to your Queue<Order>. A Queue<T> is a nice collection as it allows you to easily Dequeue() the next item in line. Within the constructor, Line calls RunWorkerAsync(), invoking the DoWork event, and in turn the handler Line_ProcessOrder.
public class Line: BackgroundWorker
{
public string LineName { get; set; }
public Queue<Order> OrderQueue { get; set; }
public Line (string lineName, Queue<Order> orderQueue)
{
LineName = lineName;
OrderQueue = orderQueue;
DoWork += Line_ProcessOrder;
RunWorkerAsync();
}
private void Line_ProcessOrder(object sender, DoWorkEventArgs e)
{
Order targetOrder;
BackgroundWorker worker = sender as BackgroundWorker;
while (true)
{
if (worker.CancellationPending == true)
{
e.Cancel = true;
break;
}
else
{
if (OrderQueue.Count > 0)
{
targetOrder = OrderQueue.Dequeue();
Console.WriteLine($"{LineName} is processing {targetOrder.OrderName}");
Thread.Sleep(targetOrder.LeadTimeSeconds * 1000);
Console.WriteLine($"{LineName} finished {targetOrder.OrderName}");
}
}
}
}
}
Finally, Factory brings this all together. We can have any number of Lines, sharing a Queue<Order>, created from any IEnumerable<Queue> that you may of otherwise had. Note that the Lines start working immediately on their construction. You may wish to add Start() and Stop() methods for example.
public class Factory
{
static void Main(string[] args)
{
List<Order> Orders = new List<Order>()
{
new Order("Order1",10),
new Order("Order2",8),
new Order("Order3",5),
new Order("Order4",15)
};
Queue<Order> OrderQueue = new Queue<Order>(Orders);
Line Line1 = new Line("Line1", OrderQueue);
Line Line2 = new Line("Line2", OrderQueue);
while (true) { }
}
}
This may not be exactly what you needed, but I hope it can take you away from the timer approach towards asynchronous programming.

Read file while updating progress bar with coroutines in Unity

I'm trying to read a file line by line while I update a progress bar (two GUI textures with a float expansion in one of its width (maxWidth * currentPercentage)).
I have two implementations:
public static string ThreadedFileRead(string path, Action<float> percAction)
{
FileInfo fileInfo = new FileInfo(path);
StringBuilder sb = new StringBuilder();
float length = fileInfo.Length;
int currentLength = 0;
using (StreamReader sr = new StreamReader(path))
{
while (!sr.EndOfStream)
{
string str = sr.ReadLine();
sb.AppendLine(str);
// yield return str;
percAction(currentLength / length);
currentLength += str.Length;
Interlocked.Add(ref currentLength, str.Length);
}
percAction(1f);
return sb.ToString();
}
}
Using the following implementation:
// Inside a MonoBehaviour
public void Start()
{
string fileContents = "";
StartCoroutine(LoadFileAsync(Application.dataPath + "/Data/file.txt", (s) => fileContents = s));
}
public IEnumerator LoadFileAsync(string path, Action<string> fin)
{
string contents = "";
lock (contents)
{
var task = Task.Factory.StartNew(() =>
{
contents = F.ThreadedFileRead(path, (f) => currentLoadProgress = f);
});
while (!task.IsCompleted)
yield return new WaitForEndOfFrame();
fin?.Invoke(contents);
}
}
But this blocks the current GUI (I don't know why).
I also used this:
// Thanks to: https://stackoverflow.com/questions/41296957/wait-while-file-load-in-unity
// Thanks to: https://stackoverflow.com/a/34378847/3286975
[MustBeReviewed]
public static IEnumerator LoadFileAsync(string pathOrUrl, Action<float> updatePerc, Action<string> finishedReading)
{
FileInfo fileInfo = new FileInfo(pathOrUrl);
float length = fileInfo.Length;
// Application.isEditor && ??? // Must review
if (Path.IsPathRooted(pathOrUrl))
pathOrUrl = "file:///" + pathOrUrl;
/*
using (var www = new UnityWebRequest(pathOrUrl))
{
www.downloadHandler = new DownloadHandlerBuffer();
CityBenchmarkData.StartBenchmark(CityBenchmark.SendWebRequest);
yield return www.SendWebRequest();
CityBenchmarkData.StopBenchmark(CityBenchmark.SendWebRequest);
while (!www.isDone)
{
// www.downloadProgress
updatePerc?.Invoke(www.downloadedBytes / length); // currentLength / length
yield return new WaitForEndOfFrame();
}
finishedReading?.Invoke(www.downloadHandler.text);
}
*/
using (var www = new WWW(pathOrUrl))
{
while (!www.isDone)
{
// www.downloadProgress
updatePerc?.Invoke(www.bytesDownloaded / length); // currentLength / length
yield return new WaitForEndOfFrame();
}
finishedReading?.Invoke(www.text);
}
}
With the following implementation:
public IEnumerator LoadFileAsync(string path, Action<string> fin)
{
yield return F.LoadFileAsync(path, (f) => currentLoadProgress = f, fin);
}
The last code I shared has two parts:
The commented part blocks also the main thread.
The WWW class I used (it will be deprecated in a future) doesn't block the main thread, but it only displays two steps on the progress bar (like 25% and 70%).
I don't why this is happening and if there is a better approach for this.
So, any help (guidance) for this is welcome.

The commented part blocks also the main thread.
It is not "blocking" the main thread. If it is blocking the main thread, the Editor would freeze too until the loading or download is complete. It's simply waiting for www.SendWebRequest() to finish and while waiting, other scripts in your project are still running normally and every frame.
The problem is that some people don't understand how www.isDone is used and when to use it. One example is this post and when people find such code, they run into issues.
Two ways to use UnityWebRequest:
1. Download or make a request then forget it until it's done. You do this when you don't need to know the status of the download such as the UnityWebRequest.downloadedBytes, UnityWebRequest.uploadedBytes and UnityWebRequest.downloadProgress and UnityWebRequest.uploadProgress values.
To do this, you yield the UnityWebRequest.SendWebRequest() function with the yield return keyword. It will wait on the UnityWebRequest.SendWebRequest() function until the download is complete but it's not blocking your program. The code in your Update function and other scripts should still be running. That wait is only done in your LoadFileAsync coroutine function.
Example (Notice the use of yield return www.SendWebRequest() and no UnityWebRequest.isDone):
using (var www = new UnityWebRequest(pathOrUrl))
{
www.downloadHandler = new DownloadHandlerBuffer();
yield return www.SendWebRequest();
if (www.isHttpError || www.isNetworkError)
{
Debug.Log("Error while downloading data: " + www.error);
}
else
{
finishedReading(www.downloadHandler.text);
}
}
2. Download or make a request then wait every frame until UnityWebRequest.isDone is true. You do this when you need to know the status of the download such as the UnityWebRequest.downloadedBytes, UnityWebRequest.uploadedBytes and UnityWebRequest.downloadProgress and UnityWebRequest.uploadProgress values and these can be check while waiting for UnityWebRequest.isDone to be true in a loop.
You cannot wait or yield the UnityWebRequest.SendWebRequest() function in this case since yielding it will wait there until the request is complete making it impossible to check the download status. Just call the UnityWebRequest.SendWebRequest() function like a normal function then do the waiting in a while loop with UnityWebRequest.isDone.
The waiting for a frame is done in a loop with yield return null or yield return new WaitForEndOfFrame() but it's recommended to use yield return null since that doesn't create GC.
Example (Notice the use of UnityWebRequest.isDone and no yield return www.SendWebRequest()):
using (var www = new UnityWebRequest(pathOrUrl))
{
www.downloadHandler = new DownloadHandlerBuffer();
//Do NOT yield the SendWebRequest function when using www.isDone
www.SendWebRequest();
while (!www.isDone)
{
updatePerc.Invoke(www.downloadedBytes / length); // currentLength / length
yield return null;
}
if (www.isHttpError || www.isNetworkError)
{
Debug.Log("Error while downloading data: " + www.error);
}
else
{
finishedReading(www.downloadHandler.text);
}
}
You are mixing both #1 and #2. In your case, you need to use #2. Notice in both cases, I checked for error after the downloading and before using the loaded data with if (www.isHttpError || www.isNetworkError). You must do this.
The WWW class I used (it will be deprecated in a future) doesn't block
the main thread, but it only displays two steps on the progress bar
(like 25% and 70%).
If that's the true, then it is likely that after fixing the issue I talked about with UnityWebRequest, the UnityWebRequest API will likely give you 25% and 70% like the WWW API too.
The reason you see 25% and 70% is because the file size is very small so the Unity API is loading it quickly that it skips some percentage values. This is normal with the Unity's API. Just use any C# System.IO API to read the file to workaround this. Make your ThreadedFileRead function return the result via Action just like your LoadFileAsync function. This will make it easier to implement the Thread.
Grab the UnityThread script which is used to make a callback to the main Thread. You do this so that you can use the values returned from the callback with the Unity API on the main Thread.
Initialize the Thread callback script in the Awake function then use ThreadPool.QueueUserWorkItem or Task.Factory.StartNew to handle the file loading. To send the value back to the main Thread, use UnityThread.executeInUpdate to make the call.
void Awake()
{
UnityThread.initUnityThread();
}
public static void ThreadedFileRead(string path, Action<float> percAction, Action<string> finishedReading)
{
/* Replace Task.Factory with ThreadPool when using .NET <= 3.5
*
* ThreadPool.QueueUserWorkItem(state =>
*
* */
var task = Task.Factory.StartNew(() =>
{
FileInfo fileInfo = new FileInfo(path);
StringBuilder sb = new StringBuilder();
float length = fileInfo.Length;
int currentLength = 0;
using (StreamReader sr = new StreamReader(path))
{
while (!sr.EndOfStream)
{
string str = sr.ReadLine();
sb.AppendLine(str);
// yield return str;
//Call on main Thread
UnityThread.executeInUpdate(() =>
{
percAction(currentLength / length);
});
currentLength += str.Length;
//Interlocked.Add(ref currentLength, str.Length);
}
//Call on main Thread
UnityThread.executeInUpdate(() =>
{
finishedReading(sb.ToString());
});
}
});
}
Usage:
ThreadedFileRead(path, (percent) =>
{
Debug.Log("Update: " + percent);
}, (result) =>
{
Debug.Log("Done: " + result);
});

parsing functions from text file without third party

I’ve been struggling wrapping my head around parsers and lexers, which I’m not even sure is the best way to tackle my challenge. I don’t want to use any third party libraries because of the unnecessary overhead, project size, and possible license issues.
Also there won’t be any arithmetic’s, loops, while’s, for’s or foreach’s
It’s a very simple parser that adds or instantiates objects from a text file.
For instance,
buildings.addWithMaterials([buildable:mansion], {{[materials: brick], [materials: wood]}, {[materials: wood], [materials: brick]}});
Parsing this text would add a Mansion made of two pieces of brick and wood to the buildings Collection.
The object buildable contains the properties Name which in this case is Mansion
and some building components which in this case are the materials Brick and Wood.
Any tip/direction to search for doing this?
I've looked and searched within stackoverflow, most entries I've stumbled upon refer to third parties
like Sprache and more.
If I missed an article, :/ sorry, please point it out to me
Thanks and kind regards, Nick

Providing all of the items in your script are in the same format, you can use something like this to parse it:
public static class ScriptParser
{
public static void Parse(string filename)
{
string[] script = null;
using (StreamReader reader = File.OpenText(filename))
{
// read the whole file, remove all line breaks and split
// by semicolons to get individual commands
script = reader.ReadToEnd().Replace("\r", string.Empty.Replace("\n", string.Empty)).Split(';');
}
foreach (string command in script)
HandleCommand(command);
}
// as arguments seem to be grouped you can't just split
// by commas so this will get the arguments recursively
private static object[] ParseArgs(string argsString)
{
int startPos = 0;
int depth = 0;
List<object> args = new List<object>();
Action<int> addArg = pos =>
{
string arg = argsString.Substring(startPos, pos - startPos).Trim();
if (arg.StartsWith("{") && arg.EndsWith("}"))
{
arg = arg.Substring(1, arg.Length - 2);
args.Add(ParseArgs(arg));
}
else
{
args.Add(arg);
}
}
for (int i = 0; i < argsString.Length; i++)
{
switch (argsString[i])
{
case ',':
if (depth == 0)
{
addArg(i);
startPos = i + 1;
}
break;
case '{':
// increase depth of parsing so that commas
// within braces are ignored
depth++;
break;
case '}':
// decrease depth when exiting braces
depth--;
break;
}
}
// as there is no final comma
addArg(argsString.Length);
return args.ToArray();
}
private static void HandleCommand(string commandString)
{
Command command = new Command();
string prefix = commandString.Substring(0, commandString.IndexOf("("));
command.Collection = prefix.Split('.')[0];
command.Action = prefix.Split('.')[1];
string commandArgs = commandString.Substring(commandString.IndexOf("("));
commandArgs = commandArgs.Substring(1, commandArgs.Length - 2);
command.Args = ParseArgs(commandArgs);
command.Execute();
}
private class Command
{
public string Collection;
public string Action;
public object[] Args;
public void Execute()
{
// need to handle any expected commands
if (Collection == "buildings")
{
if (Action == "addWithMaterials")
{
buildings.AddWithMaterials((string)Args[0], (object[])Args[1]);
}
}
}
}
//Eg.
public static BuildingCollection buildings = new BuildingCollection();
public class BuildingCollection
{
public void AddWithMaterials(string building, object[] materials)
{
Building building = new Building();
foreach (object[] materialsPart in materials)
// more stuff for you to work out
// you will need to match the strings to material classes
}
}
}
Usage:
ScriptParser.Parse("scriptfile.txt");
Note: This lacks error handling, which you will definitely want when parsing a file as it could contain anything.

Most efficient structure for quick access to a random element + the last element on one criteria

We have a function that is informed that we received an item for a specific timestamp.
The purpose of this is to wait that for one specific timestamp, we wait that we receive every item that we are expecting, then push the notification further once we are "synchronized" with all items.
Currently, we have a Dictionary<DateTime, TimeSlot> to store the non-synchronized TimeSlot(TimeSlot = list of all items we received for a specific timestamp).
//Let's assume that this method is not called concurrently, and only once per "MyItem"
public void HandleItemReceived(DateTime timestamp, MyItem item){
TimeSlot slot;
//_pendingTimeSlot is a Dictionary<DateTime,TimeSlot>
if(!_pendingTimeSlot.TryGetValue(timestamp, out slot)){
slot = new TimeSlot(timestamp);
_pendingTimeSlot.Add(timestamp,slot );
//Sometimes we don't receive all the items for one timestamps, which may leads to some ghost-incomplete TimeSlot
if(_pendingTimeSlot.Count>_capacity){
TimeSlot oldestTimeSlot = _pendingTimeSlot.OrderBy(t=>t.Key).Select(t=>t.Value).First();
_pendingTimeSlot.Remove(oldestTimeSlot.TimeStamp);
//Additional work here to log/handle this case
}
}
slot.HandleItemReceived(item);
if(slot.IsComplete){
PushTimeSlotSyncronized(slot);
_pendingTimeSlot.Remove(slot.TimeStamp);
}
}
We have severals instances of this "Synchronizer" in parallels for differents group of items.
It's working fine, except when the system is under heavy loads, we have more incomplete TimeSlot, and the application uses a lot more CPU. The profiler seems to indicate that the Compare of the LINQ query is taking a lot of time(most of the time). So I'm trying to find some structure to hold those references(replace the dictionary)
Here are some metrics:
We have several(variable, but between 10 to 20) instances of this Synchronizer
The current maximum capacity(_capacity) of the synchronizer is 500 items
The shortest interval that we can have between two different timestamp is 100ms(so 10 new Dictionary entry per seconds for each Synchronizer)(most case are more 1 item/second)
For each timestamp, we expect to receive 300-500 items.
So we will do, for one Synchronizer, per second(worst case):
1 Add
500 Get
3-5 Sorts
What would be my best move? I thought to the SortedDictionary But I didn't find any documentation showing me how to take the first element according to the key.

The first thing you can try is eliminating the OrderBy - all you need is the minimum key, no need to sort for getting that:
if (_pendingTimeSlot.Count > _capacity) {
// No Enumerable.Min(DateTime), so doing it manually
var oldestTimeStamp = DateTime.MaxValue;
foreach (var key in _pendingTimeSlot.Keys)
if (oldestTimeStamp > key) oldestTimestamp = key;
_pendingTimeSlot.Remove(oldestTimeStamp);
//Additional work here to log/handle this case
}
What about SortedDictionary, it is an option for sure, although it will consume much more memory. Since it's sorted, you can use simply sortedDictionary.First() to take the key value pair with the minimum key (hence the oldest element in your case).
UPDATE: Here is a hybrid approach using dictionary for fast lookups and ordered double linked list for the other scenarios.
class MyItem
{
// ...
}
class TimeSlot
{
public readonly DateTime TimeStamp;
public TimeSlot(DateTime timeStamp)
{
TimeStamp = timeStamp;
// ...
}
public bool IsComplete = false;
public void HandleItemReceived(MyItem item)
{
// ...
}
// Dedicated members
public TimeSlot PrevPending, NextPending;
}
class Synhronizer
{
const int _capacity = 500;
Dictionary<DateTime, TimeSlot> pendingSlotMap = new Dictionary<DateTime, TimeSlot>(_capacity + 1);
TimeSlot firstPending, lastPending;
//Let's assume that this method is not called concurrently, and only once per "MyItem"
public void HandleItemReceived(DateTime timeStamp, MyItem item)
{
TimeSlot slot;
if (!pendingSlotMap.TryGetValue(timeStamp, out slot))
{
slot = new TimeSlot(timeStamp);
Add(slot);
//Sometimes we don't receive all the items for one timestamps, which may leads to some ghost-incomplete TimeSlot
if (pendingSlotMap.Count > _capacity)
{
// Remove the oldest, which in this case is the first
var oldestSlot = firstPending;
Remove(oldestSlot);
//Additional work here to log/handle this case
}
}
slot.HandleItemReceived(item);
if (slot.IsComplete)
{
PushTimeSlotSyncronized(slot);
Remove(slot);
}
}
void Add(TimeSlot slot)
{
pendingSlotMap.Add(slot.TimeStamp, slot);
// Starting from the end, search for a first slot having TimeStamp < slot.TimeStamp
// If the TimeStamps almost come in order, this is O(1) op.
var after = lastPending;
while (after != null && after.TimeStamp > slot.TimeStamp)
after = after.PrevPending;
// Insert the new slot after the found one (if any).
if (after != null)
{
slot.PrevPending = after;
slot.NextPending = after.NextPending;
after.NextPending = slot;
if (slot.NextPending == null) lastPending = slot;
}
else
{
if (firstPending == null)
firstPending = lastPending = slot;
else
{
slot.NextPending = firstPending;
firstPending.PrevPending = slot;
firstPending = slot;
}
}
}
void Remove(TimeSlot slot)
{
pendingSlotMap.Remove(slot.TimeStamp);
if (slot.NextPending != null)
slot.NextPending.PrevPending = slot.PrevPending;
else
lastPending = slot.PrevPending;
if (slot.PrevPending != null)
slot.PrevPending.NextPending = slot.NextPending;
else
firstPending = slot;
slot.PrevPending = slot.NextPending = null;
}
void PushTimeSlotSyncronized(TimeSlot slot)
{
// ...
}
}
Some additional usages:
Iterating from oldest to newest:
for (var slot = firstPending; slot != null; slot = slot.NextPending)
{
// do something
}
Iterating from oldest to newest and removing items based on a criteria:
for (TimeSlot slot = firstPending, nextSlot; slot != null; slot = nextSlot)
{
nextSlot = slot.NextPending;
if (ShouldRemove(slot))
Remove(slot);
}
Same for reverse scenarios, but using lastPending and PrevPending members instead.

Here is simple sample. The insert method in a list eliminates swapping elements.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
List<Data> inputs = new List<Data>() {
new Data() { date = DateTime.Parse("10/22/15 6:00AM"), data = "abc"},
new Data() { date = DateTime.Parse("10/22/15 4:00AM"), data = "def"},
new Data() { date = DateTime.Parse("10/22/15 6:30AM"), data = "ghi"},
new Data() { date = DateTime.Parse("10/22/15 12:00AM"), data = "jkl"},
new Data() { date = DateTime.Parse("10/22/15 3:00AM"), data = "mno"},
new Data() { date = DateTime.Parse("10/22/15 2:00AM"), data = "pqr"},
};
Data data = new Data();
foreach (Data input in inputs)
{
data.Add(input);
}
}
}
public class Data
{
public static List<Data> sortedData = new List<Data>();
public DateTime date { get; set; }
public string data { get; set;}
public void Add(Data newData)
{
if(sortedData.Count == 0)
{
sortedData.Add(newData);
}
else
{
Boolean added = false;
for(int index = sortedData.Count - 1; index >= 0; index--)
{
if(newData.date > sortedData[index].date)
{
sortedData.Insert(index + 1, newData);
added = true;
break;
}
}
if (added == false)
{
sortedData.Insert(0, newData);
}
}
}
}
}​

Parallel.For loop for spell checking using NHunspell and c#

I have a list of string and if I run spell checking using NHunspell in sequential manner then everything works fine; but if I use Parallel.For loop against the List the application stops working in the middle( some address violation error )
public static bool IsSpellingRight(string inputword, byte[] frDic, byte[] frAff, byte[] enDic, byte[] enAff)
{
if (inputword.Length != 0)
{
bool correct;
if (IsEnglish(inputword))
{
using (var hunspell = new Hunspell(enAff, enDic))
{
correct = hunspell.Spell(inputword);
}
}
else
{
using (var hunspell = new Hunspell(frAff, frDic))
{
correct = hunspell.Spell(inputword);
}
}
return correct ;
}
return false;
}
Edit:
var tokenSource = new CancellationTokenSource();
CancellationToken ct = tokenSource.Token;
var poptions = new ParallelOptions();
// Keep one core/CPU free...
poptions.MaxDegreeOfParallelism = Environment.ProcessorCount - 1;
Task task = Task.Factory.StartNew(delegate
{
Parallel.For(0, total, poptions, i =>
{
if (words[i] != "")
{
_totalWords++;
if (IsSpellingRight(words[i],dictFileBytes,
affFileBytes,dictFileBytesE,affFileBytesE))
{
// do something
}
else
{
BeginInvoke((Action) (() =>
{
//do something on UI thread
}));
}
}
});
}, tokenSource.Token);
task.ContinueWith((t) => BeginInvoke((Action) (() =>
{
MessaageBox.Show("Done");
})));

I think you should forget about your parallel loop implement the things right.
Are you aware of the fact that this code loads and constructs the dictionary:
using (var hunspell = new Hunspell(enAff, enDic))
{
correct = hunspell.Spell(inputword);
}
Your are loading and construction the dictionary over and over again with your code. This is awfully slow! Load Your dictionary once and check all words, then dispose it. And don't do this in parallel because Hunspell objects are not thread safe.
Pseodocode:
Hunspell hunspell = null;
try
{
hunspell = new Hunspell(enAff, enDic)
for( ... )
{
hunspell.Spell(inputword[i]);
}
}
}
finally
{
if( hunspell != null ) hunspell.Dispose();
}
If you need to check words massive in parallel consider to read this article:
http://www.codeproject.com/Articles/43769/Spell-Check-Hyphenation-and-Thesaurus-for-NET-with

Ok, now I can see a potential problem. In line
_totalWords++;
you're incrementing a value, that (I suppose) is declared somewhere outside the loop. Use locking mechanism.
edit:
Also, you could use Interlocked.Increment(ref val);, which would be faster, than simple locking.
edit2:
Here's how the locking I described in comment should look like for the problem you encounter:
static object Locker = new object(); //anywhere in the class
//then in your method
if (inputword.Length != 0)
{
bool correct;
bool isEnglish;
lock(Locker) {isEnglish = IsEnglish(inputword);}
if(isEnglish)
{
//..do your stuff
}
//rest of your function
}