We Keep Coding

Logging data from structs in interval 10-15 ms in C#

I have to prepare Logger class which will be saving data from 3 structs in interval of 10-15 ms. My approach to this problem is below:
public class Logger
{
// Variables
private Task loggerTask;
public bool IsActive { get; set; }
// Constructor
public Logger()
{
}
private void Logging()
{
#if DEBUG
System.Diagnostics.Debug.WriteLine("Logging has been started.");
#endif
FileStream fs = new FileStream($"./log {DateTime.Now.ToString("dd.MM HH.mm.ss")}.txt", FileMode.Create, FileAccess.Write);
StreamWriter sw = new StreamWriter(fs, Encoding.Default);
try
{
Queue<double> times = new Queue<double>();
Queue<Attitude> attitudes = new Queue<Attitude>();
Queue<LocalPositionNed> positions = new Queue<LocalPositionNed>();
Queue<SetPositionTargetLocalNed> setpoints = new Queue<SetPositionTargetLocalNed>();
// Logs data
DateTime start = DateTime.Now;
DateTime last = start;
DateTime now;
while (IsActive)
{
now = DateTime.Now;
if ((now - last).TotalMilliseconds < 16)
continue;
last = now;
times.Enqueue((now - start).TotalMilliseconds);
attitudes.Enqueue(GCS.Instance.Drone.Attitude);
positions.Enqueue(GCS.Instance.Drone.LocalPositionNed);
setpoints.Enqueue(GCS.Instance.Offboard.SetPoint);
}
// Save data
for(int i = 0; i < times.Count; i++)
{
sw.WriteLine($"{times.ElementAt(i)}\t" +
$"{attitudes.ElementAt(i).Pitch}\t" +
$"{attitudes.ElementAt(i).Roll}\t" +
$"{attitudes.ElementAt(i).Yaw}\t" +
$"{attitudes.ElementAt(i).Pitchspeed}\t" +
$"{attitudes.ElementAt(i).Rollspeed}\t" +
$"{attitudes.ElementAt(i).Yawspeed}\t" +
$"{positions.ElementAt(i).X}\t" +
$"{positions.ElementAt(i).Y}\t" +
$"{positions.ElementAt(i).Z}\t" +
$"{positions.ElementAt(i).Vx}\t" +
$"{positions.ElementAt(i).Vy}\t" +
$"{positions.ElementAt(i).Vz}\t" +
$"{setpoints.ElementAt(i).Vx}\t" +
$"{setpoints.ElementAt(i).Vy}\t" +
$"{setpoints.ElementAt(i).Vz}\t");
}
}
catch (Exception ex)
{
#if DEBUG
System.Diagnostics.Debug.WriteLine($"Logging exception: {ex.Message}");
#endif
}
finally
{
sw.Dispose();
fs.Dispose();
}
#if DEBUG
System.Diagnostics.Debug.WriteLine("Logging has been ended.");
#endif
}
// Static method
public void Start()
{
IsActive = true;
loggerTask = new Task(Logging);
loggerTask.Start();
}
public void Stop()
{
IsActive = false;
}
}
I have problem with intervals, because they are varying about 5-8 ms. My project requires maximum varying of 1-2 ms. Does anyone have idea how I can improve my approach.
Thank you for your responses.

The biggest issue is probably using DateTime.Now, this has poor resolution and is not appropriate for this kind of task.
A simple alternative that would be more appropriate is a stopwatch.
var delay = 16;
var stopwatch = Stopwatch.StartNew();
long current = 0;
long previous;
var next = stopwatch.ElapsedMilliseconds + delay;
for (int i = 0; i < 100; i++)
{
while (next > stopwatch.ElapsedMilliseconds)
{
// Spin
}
next = stopwatch.ElapsedMilliseconds + delay;
previous = current;
current = stopwatch.ElapsedMilliseconds;
var delta = current - previous;
Console.WriteLine("Delta: " + delta);
}
As mentioned in the comments, this does not provide any strong guarantees, merely a best effort. There are also better ways to do this that does not waste an entire thread doing nothing, for example using multi media timers. But it might be adequate for testing/debugging purposes as long as the load on the CPU is light.

I'm curious about the logic that checks only once at a certain time

I would like help.
There are some problems with the code.
I want to do only one inspection at a certain time every day.
In particular, the problem is the most serious in DateTime.Now.Hour == 11 part.
I am having difficulty checking certain times. I need to write code that can be checked only once at 11:00 in the whlie statement.
I created a license file and checked the date of the file.
public static CResult Dailytime(string date)
{
CResult result = new CResult();
if(result.nRet == 0)
{
while (true)
{
if (result.nRet == 1 || result.nRet == 2)
{
return result;
}
if (DateTime.Now.Hour == 11)
{
result = DailyCheckData(date);
if(result.nRet == 1 || result.nRet == 2)
{
return result;
}
}
System.Threading.Thread.Sleep(60 * 30 * 1000);
}
}
return result;
}
public static CResult DailyCheckData(string data)
{
CResult result = new CResult();
DateTime licenseDate = Convert.ToDateTime(data);
string dateNow = DateTime.Now.ToString("yyyy-MM-dd");
int compareDate = DateTime.Compare(Convert.ToDateTime(data), DateTime.Now);
if (licenseDate.ToString("yyyy-MM-dd") == dateNow)
{
result = ExpirationCertificate();
Console.WriteLine("Result = " + result.Result + " Msg = " + result.Msg + " nRet = " + result.nRet);
return result;
}
else
{
if (compareDate > 0)
{
result = TrueCertificate();
Console.WriteLine("Result = " + result.Result + " Msg = " + result.Msg + " nRet = " + result.nRet);
}
else if (compareDate <= 0)
{
result = ExpirationCertificate();
Console.WriteLine("Result = " + result.Result + " Msg = " + result.Msg + " nRet = " + result.nRet);
}
return result;
}
}
CResult class
nRet= 0 or 1 or 2
0 = fasle date
1 = false file
2 = true
Suggest or suggest ways to improve.

Can you try create a variable for DateTime.Now, after a line of code that value change.
DateTime licenseDate = Convert.ToDateTime(data);
string dateNow = DateTime.Now.ToString("yyyy-MM-dd");
int compareDate = DateTime.Compare(Convert.ToDateTime(data), DateTime.Now);
To
DateTime licenseDate = Convert.ToDateTime(data);
var now = DateTime.Now;
string dateNow = now.ToString("yyyy-MM-dd");
int compareDate = DateTime.Compare(licenseDate, now);

you shouldn't use Thread.Sleep() method for such a long duration. It is poor programming logic to make the thread sleep for such a long period.
What you can do to solve this is, is to make a Timer. There's an example attached in the link. A simple snippet to match your logic would be:
licenseStart = //setYours
lastCheck = DateTime.Now;
nextCheck = now.AddDays(1); // now.AddHours(1)
var timer = new Timer(o=>{/* Do work*/}, null, TimeSpan.Zero, nextCheck);
Hope it helps!

I asked about logic that can only loop once at 11 o'clock.
But I could not find the right answer and I found the answer.
I do not speak English well.
That's why others do not understand the intent of the question.
bool bOnce = true;
//bool s0nce = true;
while (true)
{
if (DateTime.Now.Hour == 11 && bOnce)
{
//run code
}
if (bOnce == true)
bOnce = false;
else
bOnce = true;
Thread.Sleep(10 * 60 * 1000);
}

Optimizing code to easily edit

In my game there is a text that outputs code based on if certain Upgrades are bought I started out coding it like this
if (UpgradeManager.Instance.HasUpgrade("basicFunction") && !UpgradeManager.Instance.HasUpgrade("basicCounter"))
{
codeString = "protected void GainUnits(){ units += " + gains + ";}";
}
else if (UpgradeManager.Instance.HasUpgrade("basicCounter") && UpgradeManager.Instance.HasUpgrade("basicFunction"))
{
codeString = "private timer = 20; private void Update(){ if(timer > 0){GainUnits();}" + "protected void GainUnits(){ units += " + gains + ";}";
}
else if (UpgradeManager.Instance.HasUpgrade("basicCounter") && !UpgradeManager.Instance.HasUpgrade("basicFunction"))
{
codeString = "private timer = 20; private void Update(){ if(timer > 0){GainUnits();}" + "units += " + gains;
}
else
{
codeString = "units += " + gains;
}
as i add more and more upgrades i realize that i have to add more and more rules on top of whats already there creating an exponential amount of work as more is added to the game
So my question is how can i optimize this to lighten the workload?

You can try that approach that can be more maintainable:
// arrange a list of bool predicates and corresponding handlers
var policies = new KeyValuePair<Func<bool>, Action>[]
{
new KeyValuePair<Func<bool>, Action>(
() => {return UpgradeManager.Instance.HasUpgrade("basicFunction") && !UpgradeManager.Instance.HasUpgrade("basicCounter");},
() => {codeString = "protected void GainUnits(){ units += " + gains + ";}";}),
new KeyValuePair<Func<bool>, Action>(
() => {return UpgradeManager.Instance.HasUpgrade("basicCounter") && UpgradeManager.Instance.HasUpgrade("basicFunction");},
() => {codeString = "private timer = 20; private void Update(){ if(timer > 0){GainUnits();}" + "protected void GainUnits(){ units += " + gains + ";}";}),
new KeyValuePair<Func<bool>, Action>(
() => {return UpgradeManager.Instance.HasUpgrade("basicCounter") && !UpgradeManager.Instance.HasUpgrade("basicFunction");},
() => {codeString = "private timer = 20; private void Update(){ if(timer > 0){GainUnits();}" + "units += " + gains;}),
new KeyValuePair<Func<bool>, Action>(
() => {return true;}, // last one should be always true
() => {codeString = "units += " + gains;}),
};
// now let iterate over the policies
foreach(var policy in policies)
{
if (policy.Key()) // evaluate predicates one-by-one
{
policy.Value();
break; // if predicated matched, do the action and break out of the loop
}
}
Depending on the logic necessary, you can employ different containers (that can give you better performance or flexibility, etc.)

else if (UpgradeManager.Instance.HasUpgrade("basicCounter") && !UpgradeManager.Instance.HasUpgrade("basicFunction"))
{
codeString = "private timer = 20; private void Update(){ if(timer > 0){GainUnits();}" + "units += " + gains;
}
Doesn't look like valid code there. I assume you want to increment it directly in the if-block, like this?
"if(timer > 0){ units += " + gains + "; }"
You can start by isolating what part of code that each flag modifies:
basicCounter: adds timer and implements Update() that increments units
basicFunction: encapsulates how units is incremented
With that in mind, it is easier to deconstruct the code:
increasing units:
GainUnits(); // if basicFunction
units += __GAINS__; // if !basicFunction
counter:
private timer = 20;
private void Update() {
if(timer > 0) {
/* use the code from <increasing units> */
}
}
core implementation:
/* use code from <counter> */ // if basicCounter
/* use code from <increasing units> */ // if !basicCounter
And, putting it back together:
string GenerateCode()
{
return string.Join("\n", GetImplementationCode());
}
IEnumerable<string> GetImplementationCode()
{
return UpgradeManager.Instance.HasUpgrade("basicCounter")
? GetCounterCode()
: GetIncreaseUnitsCode();
}
IEnumerable<string> GetCounterCode()
{
yield return "private timer = 20;";
yield return "private void Update() {";
yield return "if(timer > 0) {";
foreach (var loc in GetIncreaseUnitsCode())
{
yield return loc;
}
yield return "}";
yield return "}";
}
IEnumerable<string> GetIncreaseUnitsCode()
{
if (UpgradeManager.Instance.HasUpgrade("basicFunction"))
{
yield return "GainUnits();";
}
else
{
var gains = /* insert logics here */;
yield return $"units += {gains};";
}
}

write file out of memory c#

I get some problems with c# windows form.
My goal is to slice a big file(maybe>5GB) into files,and each file contains a million lines.
According to the code below,I have no idea why it will be out of memory.
Thanks.
StreamReader readfile = new StreamReader(...);
StreamWriter writefile = new StreamWriter(...);
string content;
while ((content = readfile.ReadLine()) != null)
{
writefile.Write(content + "\r\n");
i++;
if (i % 1000000 == 0)
{
index++;
writefile.Close();
writefile.Dispose();
writefile = new StreamWriter(...);
}
label5.Text = i.ToString();
label5.Update();
}

The error is probably in the
label5.Text = i.ToString();
label5.Update();
just to make a test I've written something like:
for (int i = 0; i < int.MaxValue; i++)
{
label1.Text = i.ToString();
label1.Update();
}
The app freezes around 16000-18000 (Windows 7 Pro SP1 x64, the app running both x86 and x64).
What probably happens is that by running your long operation in the main thread of the app, you stall the message queue of the window, and at a certain point it freezes. You can see that this is the problem by adding a
Application.DoEvents();
instead of the
label5.Update();
But even this is a false solution. The correct solution is moving the copying on another thread and updating the control every x milliseconds, using the Invoke method (because you are on a secondary thread),
For example:
public void Copy(string source, string dest)
{
const int updateMilliseconds = 100;
int index = 0;
int i = 0;
StreamWriter writefile = null;
try
{
using (StreamReader readfile = new StreamReader(source))
{
writefile = new StreamWriter(dest + index);
// Initial value "back in time". Forces initial update
int milliseconds = unchecked(Environment.TickCount - updateMilliseconds);
string content;
while ((content = readfile.ReadLine()) != null)
{
writefile.Write(content);
writefile.Write("\r\n"); // Splitted to remove a string concatenation
i++;
if (i % 1000000 == 0)
{
index++;
writefile.Dispose();
writefile = new StreamWriter(dest + index);
// Force update
milliseconds = unchecked(milliseconds - updateMilliseconds);
}
int milliseconds2 = Environment.TickCount;
int diff = unchecked(milliseconds2 - milliseconds);
if (diff >= updateMilliseconds)
{
milliseconds = milliseconds2;
Invoke((Action)(() => label5.Text = string.Format("File {0}, line {1}", index, i)));
}
}
}
}
finally
{
if (writefile != null)
{
writefile.Dispose();
}
}
// Last update
Invoke((Action)(() => label5.Text = string.Format("File {0}, line {1} Finished", index, i)));
}
and call it with:
var thread = new Thread(() => Copy(#"C:\Temp\lst.txt", #"C:\Temp\output"));
thread.Start();
Note how it will write the label5 every 100 milliseconds, plus once at the beginning (by setting the initial value of milliseconds "back in time"), each time the output file is changed (by setting the value of milliseconds "back in time") and after having disposed everything.
An even more correct example can be written by using the BackgroundWorker class, that exists explicitly for this scenario. It has an event, ProgressChanged, that can be subscribed to update the window.
Something like this:
private void button1_Click(object sender, EventArgs e)
{
BackgroundWorker backgroundWorker = new BackgroundWorker();
backgroundWorker.WorkerReportsProgress = true;
backgroundWorker.ProgressChanged += backgroundWorker_ProgressChanged;
backgroundWorker.RunWorkerCompleted += backgroundWorker_RunWorkerCompleted;
backgroundWorker.DoWork += backgroundWorker_DoWork;
backgroundWorker.RunWorkerAsync(new string[] { #"C:\Temp\lst.txt", #"C:\Temp\output" });
}
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker worker = sender as BackgroundWorker;
string[] arguments = (string[])e.Argument;
string source = arguments[0];
string dest = arguments[1];
const int updateMilliseconds = 100;
int index = 0;
int i = 0;
StreamWriter writefile = null;
try
{
using (StreamReader readfile = new StreamReader(source))
{
writefile = new StreamWriter(dest + index);
// Initial value "back in time". Forces initial update
int milliseconds = unchecked(Environment.TickCount - updateMilliseconds);
string content;
while ((content = readfile.ReadLine()) != null)
{
writefile.Write(content);
writefile.Write("\r\n"); // Splitted to remove a string concatenation
i++;
if (i % 1000000 == 0)
{
index++;
writefile.Dispose();
writefile = new StreamWriter(dest + index);
// Force update
milliseconds = unchecked(milliseconds - updateMilliseconds);
}
int milliseconds2 = Environment.TickCount;
int diff = unchecked(milliseconds2 - milliseconds);
if (diff >= updateMilliseconds)
{
milliseconds = milliseconds2;
worker.ReportProgress(0, new int[] { index, i });
}
}
}
}
finally
{
if (writefile != null)
{
writefile.Dispose();
}
}
// For the RunWorkerCompleted
e.Result = new int[] { index, i };
}
void backgroundWorker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
int[] state = (int[])e.UserState;
label5.Text = string.Format("File {0}, line {1}", state[0], state[1]);
}
void backgroundWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
int[] state = (int[])e.Result;
label5.Text = string.Format("File {0}, line {1} Finished", state[0], state[1]);
}

Timing Manager and Worker Threads

I am having trouble getting the synchronization of my managers and workers to go through correctly. The problem is that the SyncManager is stopping before the SyncWorker is done... And I don't know why it is continuing to execute without waiting for the SyncWorker to finish.
SyncManager
public void ExecuteTask()
{
/*Code to set up Variables*/
while (fileMetadata.Count > 0 && Status != WorkerStatus.ShouldStop)
{
if (DateTime.Now.Subtract(lastUpdate).Minutes > 1)
_serviceLog.WriteEntry(ProgressString());
if (Status == WorkerStatus.ShouldStop)
break;
SyncFile(fileMetadata[0]);
//Possible solution
//Wait for Sync File to Finish before stepping (although logically it should do that)
if (DateTime.Now.Subtract(lastUpdate).Minutes > 1)
_serviceLog.WriteEntry(ProgressString());
//Code That I think is causing the error
fileMetadata.RemoveAt(0);
}
/*Code to preform clean up actions and notify user of all transfers*/
}
public void SyncFile(FileMetadata fmd)
{
lock (executionLock)
{
attempts++;
long waitTime = 0;
switch (Type)
{
case (SyncManagerType.UploadLarge):
waitTime = 8192;
break;
case (SyncManagerType.UploadMedium):
waitTime = 4096;
break;
case (SyncManagerType.DownloadLarge):
waitTime = 8192;
break;
case (SyncManagerType.DownloadMedium):
waitTime = 4096;
break;
default:
waitTime = 30;
break;
}
syncWorker = new FileSyncWorker(GenerateSyncWorkerID(), _serviceLog, fmd, ref sb);
syncThread = new Thread(new ThreadStart(syncWorker.ExecuteTask));
syncThread.Start();
DateTime start = DateTime.Now;
DateTime finish;
while (DateTime.Now.Subtract(start).TotalSeconds < waitTime && (syncWorker.Status == WorkerStatus.Working))
{
if (DateTime.Now.Subtract(lastUpdate).Minutes > 1)
_serviceLog.WriteEntry(ProgressString());
if (Status == WorkerStatus.ShouldStop)
waitTime = 15;
Thread.Sleep(1000);
}
finish = DateTime.Now;
Thread.Sleep(1000);
if (finish.Subtract(start).TotalSeconds < waitTime)
{
if (syncWorker.Status == WorkerStatus.CompletedSuccess)
{
successes++;
}
}
else
{
syncThread.Abort();
sb.AppendLine("Sync Manager (" + ID + ") - FSW (" + syncWorker.ID + ") - Transfer timed out after " + waitTime + " seconds.");
}
//Possible Solution
//FileMetadata toRemove = fileMetadata.Find(delegate(FileMetadata f) { return f.Equals(syncWorker.FileData); });
//fileMetadata.Remove(toRemove);
activeIDs.RemoveAll(delegate(int i) { return i == syncWorker.ID; });
syncWorker = null;
Thread.Sleep(1000);
}
}

If you want the parent thread to wait for the worker make sure you put a Thread.Join before exiting:
if (finish.Subtract(start).TotalSeconds < waitTime)
{
if (syncWorker.Status == WorkerStatus.CompletedSuccess)
{
successes++;
}
}
else
{
syncThread.Abort();
syncThread.Join();
sb.AppendLine("Sync Manager (" + ID + ") - FSW (" + syncWorker.ID + ") - Transfer timed out after " + waitTime + " seconds.");
}