Calculate rolling window Max DrawDown in C# - c#

The requirement is to compute the maximum drawdown for a rolling window in C# for a timeseries of e.g. returns.
I.e. at each new observation, we recompute the maximum drawdown for the new time window.

Hello people.
This is quite a complex problem if you want to solve this in a computationally efficient way for a rolling window.
I have gone ahead and written a solution to this in C#.
I want to share this as the effort required to replicate this work is quite high.
First, here are the results:
here we take a simple drawdown implementation and re-calculate for the full window each time
test1 - simple drawdown test with 30 period rolling window. run 100 times.
total seconds 0.8060461
test2 - simple drawdown test with 60 period rolling window. run 100 times.
total seconds 1.416081
test3 - simple drawdown test with 180 period rolling window. run 100 times.
total seconds 3.6602093
test4 - simple drawdown test with 360 period rolling window. run 100 times.
total seconds 6.696383
test5 - simple drawdown test with 500 period rolling window. run 100 times.
total seconds 8.9815137
here we compare to the results generated from my efficient rolling window algorithm where only the latest observation is added and then it does it's magic
test6 - running drawdown test with 30 period rolling window. run 100 times.
total seconds 0.2940168
test7 - running drawdown test with 60 period rolling window. run 100 times.
total seconds 0.3050175
test8 - running drawdown test with 180 period rolling window. run 100 times.
total seconds 0.3780216
test9 - running drawdown test with 360 period rolling window. run 100 times.
total seconds 0.4560261
test10 - running drawdown test with 500 period rolling window. run 100 times.
total seconds 0.5050288
At at 500 period window. We are achieving about a 20:1 improvement in calculation time.
Here is the code of the simple drawdown class used for the comparisons:
public class SimpleDrawDown
{
public double Peak { get; set; }
public double Trough { get; set; }
public double MaxDrawDown { get; set; }
public SimpleDrawDown()
{
Peak = double.NegativeInfinity;
Trough = double.PositiveInfinity;
MaxDrawDown = 0;
}
public void Calculate(double newValue)
{
if (newValue > Peak)
{
Peak = newValue;
Trough = Peak;
}
else if (newValue < Trough)
{
Trough = newValue;
var tmpDrawDown = Peak - Trough;
if (tmpDrawDown > MaxDrawDown)
MaxDrawDown = tmpDrawDown;
}
}
}
And here is the code for the full efficient implementation. Hopefully the code comments make sense.
internal class DrawDown
{
int _n;
int _startIndex, _endIndex, _troughIndex;
public int Count { get; set; }
LinkedList<double> _values;
public double Peak { get; set; }
public double Trough { get; set; }
public bool SkipMoveBackDoubleCalc { get; set; }
public int PeakIndex
{
get
{
return _startIndex;
}
}
public int TroughIndex
{
get
{
return _troughIndex;
}
}
//peak to trough return
public double DrawDownAmount
{
get
{
return Peak - Trough;
}
}
/// <summary>
///
/// </summary>
/// <param name="n">max window for drawdown period</param>
/// <param name="peak">drawdown peak i.e. start value</param>
public DrawDown(int n, double peak)
{
_n = n - 1;
_startIndex = _n;
_endIndex = _n;
_troughIndex = _n;
Count = 1;
_values = new LinkedList<double>();
_values.AddLast(peak);
Peak = peak;
Trough = peak;
}
/// <summary>
/// adds a new observation on the drawdown curve
/// </summary>
/// <param name="newValue"></param>
public void Add(double newValue)
{
//push the start of this drawdown backwards
//_startIndex--;
//the end of the drawdown is the current period end
_endIndex = _n;
//the total periods increases with a new observation
Count++;
//track what all point values are in the drawdown curve
_values.AddLast(newValue);
//update if we have a new trough
if (newValue < Trough)
{
Trough = newValue;
_troughIndex = _endIndex;
}
}
/// <summary>
/// Shift this Drawdown backwards in the observation window
/// </summary>
/// <param name="trackingNewPeak">whether we are already tracking a new peak or not</param>
/// <returns>a new drawdown to track if a new peak becomes active</returns>
public DrawDown MoveBack(bool trackingNewPeak, bool recomputeWindow = true)
{
if (!SkipMoveBackDoubleCalc)
{
_startIndex--;
_endIndex--;
_troughIndex--;
if (recomputeWindow)
return RecomputeDrawdownToWindowSize(trackingNewPeak);
}
else
SkipMoveBackDoubleCalc = false;
return null;
}
private DrawDown RecomputeDrawdownToWindowSize(bool trackingNewPeak)
{
//the start of this drawdown has fallen out of the start of our observation window, so we have to recalculate the peak of the drawdown
if (_startIndex < 0)
{
Peak = double.NegativeInfinity;
_values.RemoveFirst();
Count--;
//there is the possibility now that there is a higher peak, within the current drawdown curve, than our first observation
//when we find it, remove all data points prior to this point
//the new peak must be before the current known trough point
int iObservation = 0, iNewPeak = 0, iNewTrough = _troughIndex, iTmpNewPeak = 0, iTempTrough = 0;
double newDrawDown = 0, tmpPeak = 0, tmpTrough = double.NegativeInfinity;
DrawDown newDrawDownObj = null;
foreach (var pointOnDrawDown in _values)
{
if (iObservation < _troughIndex)
{
if (pointOnDrawDown > Peak)
{
iNewPeak = iObservation;
Peak = pointOnDrawDown;
}
}
else if (iObservation == _troughIndex)
{
newDrawDown = Peak - Trough;
tmpPeak = Peak;
}
else
{
//now continue on through the remaining points, to determine if there is a nested-drawdown, that is now larger than the newDrawDown
//e.g. higher peak beyond _troughIndex, with higher trough than that at _troughIndex, but where new peak minus new trough is > newDrawDown
if (pointOnDrawDown > tmpPeak)
{
tmpPeak = pointOnDrawDown;
tmpTrough = tmpPeak;
iTmpNewPeak = iObservation;
//we need a new drawdown object, as we have a new higher peak
if (!trackingNewPeak)
newDrawDownObj = new DrawDown(_n + 1, tmpPeak);
}
else
{
if (!trackingNewPeak && newDrawDownObj != null)
{
newDrawDownObj.MoveBack(true, false); //recomputeWindow is irrelevant for this as it will never fall before period 0 in this usage scenario
newDrawDownObj.Add(pointOnDrawDown); //keep tracking this new drawdown peak
}
if (pointOnDrawDown < tmpTrough)
{
tmpTrough = pointOnDrawDown;
iTempTrough = iObservation;
var tmpDrawDown = tmpPeak - tmpTrough;
if (tmpDrawDown > newDrawDown)
{
newDrawDown = tmpDrawDown;
iNewPeak = iTmpNewPeak;
iNewTrough = iTempTrough;
Peak = tmpPeak;
Trough = tmpTrough;
}
}
}
}
iObservation++;
}
_startIndex = iNewPeak; //our drawdown now starts from here in our observation window
_troughIndex = iNewTrough;
for (int i = 0; i < _startIndex; i++)
{
_values.RemoveFirst(); //get rid of the data points prior to this new drawdown peak
Count--;
}
return newDrawDownObj;
}
return null;
}
}
public class RunningDrawDown
{
int _n;
List<DrawDown> _drawdownObjs;
DrawDown _currentDrawDown;
DrawDown _maxDrawDownObj;
/// <summary>
/// The Peak of the MaxDrawDown
/// </summary>
public double DrawDownPeak
{
get
{
if (_maxDrawDownObj == null) return double.NegativeInfinity;
return _maxDrawDownObj.Peak;
}
}
/// <summary>
/// The Trough of the Max DrawDown
/// </summary>
public double DrawDownTrough
{
get
{
if (_maxDrawDownObj == null) return double.PositiveInfinity;
return _maxDrawDownObj.Trough;
}
}
/// <summary>
/// The Size of the DrawDown - Peak to Trough
/// </summary>
public double DrawDown
{
get
{
if (_maxDrawDownObj == null) return 0;
return _maxDrawDownObj.DrawDownAmount;
}
}
/// <summary>
/// The Index into the Window that the Peak of the DrawDown is seen
/// </summary>
public int PeakIndex
{
get
{
if (_maxDrawDownObj == null) return 0;
return _maxDrawDownObj.PeakIndex;
}
}
/// <summary>
/// The Index into the Window that the Trough of the DrawDown is seen
/// </summary>
public int TroughIndex
{
get
{
if (_maxDrawDownObj == null) return 0;
return _maxDrawDownObj.TroughIndex;
}
}
/// <summary>
/// Creates a running window for the calculation of MaxDrawDown within the window
/// </summary>
/// <param name="n">the number of periods within the window</param>
public RunningDrawDown(int n)
{
_n = n;
_currentDrawDown = null;
_drawdownObjs = new List<DrawDown>();
}
/// <summary>
/// The new value to add onto the end of the current window (the first value will drop off)
/// </summary>
/// <param name="newValue">the new point on the curve</param>
public void Calculate(double newValue)
{
if (double.IsNaN(newValue)) return;
if (_currentDrawDown == null)
{
var drawDown = new DrawDown(_n, newValue);
_currentDrawDown = drawDown;
_maxDrawDownObj = drawDown;
}
else
{
//shift current drawdown back one. and if the first observation falling outside the window means we encounter a new peak after the current trough, we start tracking a new drawdown
var drawDownFromNewPeak = _currentDrawDown.MoveBack(false);
//this is a special case, where a new lower peak (now the highest) is created due to the drop of of the pre-existing highest peak, and we are not yet tracking a new peak
if (drawDownFromNewPeak != null)
{
_drawdownObjs.Add(_currentDrawDown); //record this drawdown into our running drawdowns list)
_currentDrawDown.SkipMoveBackDoubleCalc = true; //MoveBack() is calculated again below in _drawdownObjs collection, so we make sure that is skipped this first time
_currentDrawDown = drawDownFromNewPeak;
_currentDrawDown.MoveBack(true);
}
if (newValue > _currentDrawDown.Peak)
{
//we need a new drawdown object, as we have a new higher peak
var drawDown = new DrawDown(_n, newValue);
//do we have an existing drawdown object, and does it have more than 1 observation
if (_currentDrawDown.Count > 1)
{
_drawdownObjs.Add(_currentDrawDown); //record this drawdown into our running drawdowns list)
_currentDrawDown.SkipMoveBackDoubleCalc = true; //MoveBack() is calculated again below in _drawdownObjs collection, so we make sure that is skipped this first time
}
_currentDrawDown = drawDown;
}
else
{
//add the new observation to the current drawdown
_currentDrawDown.Add(newValue);
}
}
//does our new drawdown surpass any of the previous drawdowns?
//if so, we can drop the old drawdowns, as for the remainer of the old drawdowns lives in our lookup window, they will be smaller than the new one
var newDrawDown = _currentDrawDown.DrawDownAmount;
_maxDrawDownObj = _currentDrawDown;
var maxDrawDown = newDrawDown;
var keepDrawDownsList = new List<DrawDown>();
foreach (var drawDownObj in _drawdownObjs)
{
drawDownObj.MoveBack(true);
if (drawDownObj.DrawDownAmount > newDrawDown)
{
keepDrawDownsList.Add(drawDownObj);
}
//also calculate our max drawdown here
if (drawDownObj.DrawDownAmount > maxDrawDown)
{
maxDrawDown = drawDownObj.DrawDownAmount;
_maxDrawDownObj = drawDownObj;
}
}
_drawdownObjs = keepDrawDownsList;
}
}
Example usage:
RunningDrawDown rd = new RunningDrawDown(500);
foreach (var input in data)
{
rd.Calculate(input);
Console.WriteLine(string.Format("max draw {0:0.00000}, peak {1:0.00000}, trough {2:0.00000}, drawstart {3:0.00000}, drawend {4:0.00000}",
rd.DrawDown, rd.DrawDownPeak, rd.DrawDownTrough, rd.PeakIndex, rd.TroughIndex));
}

This algorithm correctly calculates the maximum percentual drawdown, as it's commonly used in finance.
The example by DaManJ calculates the maximum numeric dropdown. While this can still be useful, it's more uncommon. I also added a function for this.
/// <summary>
/// Calculate the maximum percentual drawdown in a collection of values. This is the most common method.
/// </summary>
public static MaxDrawDown CalculateMaxDrawDownByPercentage(IEnumerable<double> values) => CalculateMaxDrawDown(values, true);
/// <summary>
/// Calculate the maximum numeric drawdown in a collection of values. This method is less common but can sometimes be more relevant.
/// </summary>
public static MaxDrawDown CalculateMaxDrawDownByAmount(IEnumerable<double> values) => CalculateMaxDrawDown(values, false);
private static MaxDrawDown CalculateMaxDrawDown(IEnumerable<double> values, bool byPercentage)
{
double peakChange = double.NaN;
double maxDrop = double.MinValue, maxDrawdown = double.MaxValue;
double maxDropByPercentage=0;
foreach (var change in values)
{
if (double.IsNaN(peakChange))
{
peakChange = change;
continue;
}
var diff = peakChange - change;
peakChange = diff < 0 ? change : peakChange;
double newDrawdown;
if (maxDrop < diff) //new low drop in amount
maxDrop = diff;
newDrawdown = (maxDrop / peakChange) * -1d;
if ( !byPercentage || newDrawdown < maxDrawdown ) { //new low drop in percent
maxDrawdown = newDrawdown;
maxDropByPercentage = maxDrop;
}
}
double percentage = maxDrawdown != double.MaxValue && maxDrawdown < 0 ? Math.Abs(maxDrawdown) : 0;
maxDrop = maxDrop != double.MinValue ? maxDrop : 0;
if (byPercentage)
return new MaxDrawDown() { DrawDownPercentage = percentage, DrawDownAmount = maxDropByPercentage };
else
return new MaxDrawDown() { DrawDownPercentage = percentage, DrawDownAmount = maxDrop };
}
public class MaxDrawDown { public double DrawDownPercentage { get; set; } public double DrawDownAmount { get; set; } }
[Test]
public void TestAmountVsPercentage()
{
var increasingPeaks = new List<double>() {
60,
20, //lowest percentually
100,
50, //lowest by amount
//60 to 20 is still the max percentual drawdown (40 units, 66%), not 100 to 50 (50 units, 50%)
};
var rd = CalculateMaxDrawDownByPercentage(increasingPeaks);
Assert.That(rd.DrawDownAmount, Is.EqualTo(40));
Assert.That(rd.DrawDownPercentage, Is.EqualTo(0.666).Within(0.001));
var rdAmount = CalculateMaxDrawDownByAmount(increasingPeaks);
Assert.That(rdAmount.DrawDownAmount, Is.EqualTo(50));
Assert.That(rdAmount.DrawDownPercentage, Is.EqualTo(0.5));
}

Related

Grading Form Using C#

I was trying to make a .NET grade form using C# and the issue that I am having is that the average that I am getting is wrong. Even though I tried to calculate the total in many different loops it just does not get the sum correct. Sometime it is too high sometimes it is too low. When I tried calculating the total marks in the validation loop it shows some weird sum of high value and when I tried to calculate the total in calculate function the sum comes out too low. Please help!
namespace SemesterGradesForm
{
public partial class formSemesterGrades : Form
{
TextBox[] inputTextBoxes;
TextBox[] outputTextBoxes;
double totalMarks;
public formSemesterGrades()
{
InitializeComponent();
inputTextBoxes = new TextBox[] { textBoxCourse1Marks, textBoxCourse2Marks, textBoxCourse3Marks, textBoxCourse4Marks, textBoxCourse5Marks, textBoxCourse6Marks, textBoxCourse7Marks };
outputTextBoxes = new TextBox[] { textBoxCourse1LetterGrade, textBoxCourse2LetterGrade, textBoxCourse3LetterGrade, textBoxCourse4LetterGrade, textBoxCourse5LetterGrade, textBoxCourse6LetterGrade, textBoxCourse7LetterGrade };
}
/// <summary>
/// Compare a given numeric grade value to an array of grades to determine a letter representing that grade.
/// </summary>
/// <param name="numericGrade"> A grde between 0 and 100</param>
/// <returns>Letter grade as a short string</returns>
private string GetLetterGrade(double numericGrade)
{
// Declare arrays for the grade values and letter values that corresponds.
double[] gradeValues = { 0D, 50D, 52D, 58D, 60D, 62D, 68D, 70D, 72D, 78D, 80D, 82D, 90D };
string[] gradeLetters = { "F", "D-", "D", "D+", "C-", "C", "C+", "B-", "B", "B+", "A-", "A", "A+" };
// Default the return letter to F
string returnLetter = "F";
// Count through the array comparing grades to the input grade.
for (int counter = 0; counter < gradeValues.Length; counter++)
{
// if the niput grade is bigger than the value in the array, assign the letter grade.
if (numericGrade > gradeValues[counter])
{
returnLetter = gradeLetters[counter];
}
// If the input grade is not bigger than the value in the arraym return the last assigned letter grade
else
{
return returnLetter;
}
}
return returnLetter;
}
/// <summary>
/// Check if a passed TexBox is a numeric grade between 0 and 100
/// </summary>
/// <param name="boxToCheck"> A textbox to check for a valid numeric grade value</param>
/// <returns> true if valid </returns>
private bool IsTextBoxValid(TextBox boxToCheck)
{
const double MinimumGrade = 0.0;
const double MaximumGrade = 100.0;
double gradeValue = 0;
if (double.TryParse(boxToCheck.Text, out gradeValue))
{
if (gradeValue >= MinimumGrade && gradeValue <= MaximumGrade)
{
return true;
}
else
{
return false;
}
}
else
{
return false;
}
}
/// <summary>
/// Clears fields and sets the form to its default state
/// </summary>
private void SetDefaults()
{
// Clear All input controls.
ClearControls(inputTextBoxes);
//Clear all output controls.
ClearControls(outputTextBoxes);
textBoxSemesterMarks.Clear();
textBoxSemesterLetterGrade.Clear();
textBoxOutput.Clear();
// Reset variable
totalMarks = 0;
//Set focus in some useful way
textBoxCourse1Marks.Focus();
// Re-enable controls
buttonCalculate.Enabled = true;
SetControlsEnabled(inputTextBoxes, true);
}
/// <summary>
/// Mass clears the text boxes
/// </summary>
/// <param name="controlArray">An array of controls with a text property to clear</param>
private void ClearControls(Control[] controlArray)
{
foreach (Control controlToClear in controlArray)
{
controlToClear.Text = String.Empty;
}
}
/// <summary>
/// TODO: You should comment this - what does it do?
/// </summary>
/// <param name="controlArray">An array of controls to enable or disable</param>
/// <param name="enabledStatus">true to enable, false to disable</param>
private void SetControlsEnabled(Control[] controlArray, bool enabledStatus)
{
foreach (Control controlToSet in controlArray)
{
controlToSet.Enabled = enabledStatus;
}
}
/// <summary>
/// When youn leave one of the textboxes
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void LeaveInputTextbox (object sender, EventArgs e)
{
// Count through all of the textboxes.
for( int inputCounter = 0; inputCounter < inputTextBoxes.Length; inputCounter++ )
{
// Determine if the textbox's contents are valid.
if (IsTextBoxValid(inputTextBoxes[inputCounter]))
{
double grade;
// Get the letter grade from this textbox
grade = double.Parse(inputTextBoxes[inputCounter].Text);
// Assign this letter grade to the corresponding label.
outputTextBoxes[inputCounter].Text = GetLetterGrade(grade);
}
}
}
private void formSemesterGrades_Load(object sender, EventArgs e)
{
}
private void buttonReset_Click(object sender, EventArgs e)
{
SetDefaults();
}
private void buttonCalculate_Click(object sender, EventArgs e)
{
double averageMarks = 0;
int invalidBox = 0;
int validBox = 0;
for (int inputCounter = 0; inputCounter < inputTextBoxes.Length; inputCounter++)
{
if (IsTextBoxValid(inputTextBoxes[inputCounter]))
{
totalMarks += double.Parse(inputTextBoxes[inputCounter].Text);
// Increase counter
inputCounter++;
// If the textbox is valid, count it. If not, just don't.
validBox++;
}
else
{
// If the box is not blank, increment the number of invalid boxes by one.
if(String.IsNullOrEmpty(inputTextBoxes[inputCounter].Text) == false)
{
// Focus on invalid input
inputTextBoxes[inputCounter].Focus();
textBoxOutput.Text = "Please enter VALID Values!";
// Increase invalid counter
invalidBox++;
}
}
}
// If number of valid boxes == 1 && number of invalid boxes == 0
if (validBox >= 1 && invalidBox == 0)
{
// Calculate and output the average
averageMarks = Math.Round(totalMarks / inputTextBoxes.Length, 2);
// Display the Average marks and grade
textBoxSemesterMarks.Text = averageMarks.ToString();
// Assign this letter grade to the corresponding label.
double grade;
grade = double.Parse(textBoxSemesterMarks.Text);
textBoxSemesterLetterGrade.Text = GetLetterGrade(grade);
// Disable input controls until the form is reset.
buttonCalculate.Enabled = false;
SetControlsEnabled(inputTextBoxes, false);
buttonReset.Focus();
}
else
{
textBoxOutput.Text = "Please enter VALID Values!";
}
}
private void buttonExit_Click(object sender, EventArgs e)
{
Close();
}
}
}
I think you have posted too much code, as the question pertains to the calculation of the class average grade.
Fundamentally, you are storing the grades as strings inside UI elements (such as text boxes) and therefore cannot directly interact with the data unless you keep converting from strings to values all the time.
I suggest you create a C# object (a class) to store the class grades named Class and have it handle all the logic. In this case, you can use the built-in .Average() method as part of System.Linq which returns the average of any collection of numbers.
In my example below I produces the following output:
Class: Art I
---
Student Score Grade
Alex 48 F
Beatrice 56 D
Claire 65 C
Dennis 78 B+
Eugene 82 A
Forest 88 A
Gwen 98 A+
---
Average 73.6 B
from the following sample code:
static void Main(string[] args)
{
var art = new Class("Art I",
"Alex", "Beatrice", "Claire",
"Dennis", "Eugene", "Forest",
"Gwen");
art.SetGrade("Alex", 48m);
art.SetGrade("Beatrice", 56m);
art.SetGrade("Claire", 65m);
art.SetGrade("Dennis", 78m);
art.SetGrade("Eugene", 82m);
art.SetGrade("Forest", 88m);
art.SetGrade("Gwen", 98m);
Console.WriteLine($"Class: {art.Title}");
Console.WriteLine("---");
Console.WriteLine($"{"Student",12} {"Score",8} {"Grade",8}");
foreach (var grade in art.Grades)
{
Console.WriteLine($"{grade.Key,12} {grade.Value,8} {Class.GetLetterGrade(grade.Value),8}");
}
Console.WriteLine("---");
Console.WriteLine($"{"Average",12} {art.AverageScore,8} {Class.GetLetterGrade(art.AveragScore),8}");
}
The key here is the methods Class.AverageScore and the letter scores which depend on the static function Class.GetLetterGrade().
The actual logic is handled by the Class object defined as
public class Class
{
public Class(string title, params string[] students)
{
Title = title;
grades = new Dictionary<string, decimal>();
foreach (var student in students)
{
grades[student] = 0m;
}
}
public string Title { get; }
public IReadOnlyCollection<string> Students { get => grades.Keys.ToList(); }
public void SetGrade(string student, decimal score)
{
if (grades.ContainsKey(student))
{
this.grades[student] = score;
}
else
{
throw new ArgumentException("Student not found", nameof(student));
}
}
readonly Dictionary<string, decimal> grades;
public IReadOnlyDictionary<string, decimal> Grades { get => grades; }
public decimal AverageScore { get => Math.Round(grades.Values.Average(),1); }
#region Grading
// Declare arrays for the grade values and letter values that corresponds.
static readonly decimal[] gradeValues = { 0, 50, 52, 58, 60, 62, 68, 70, 72, 78, 80, 82, 90 };
static readonly string[] gradeLetters = { "F", "D-", "D", "D+", "C-", "C", "C+", "B-", "B", "B+", "A-", "A", "A+" };
public static string GetLetterGrade(decimal score)
{
// use rounding rules
score = Math.Round(score, 0, MidpointRounding.AwayFromZero);
if (score > 100m) { score = 100m; } // max 100
int index = Array.IndexOf(gradeValues, gradeValues.LastOrDefault((x) => score >= x));
if (index < 0) { index = 0; } // default "F"
return gradeLetters[index];
}
public static decimal GetScoreFromLetter(string grade)
{
int index = Array.IndexOf(gradeLetters, grade);
if (index <= 0) { index = 0; } // default "0"
return gradeValues[index];
}
#endregion
}
Notice that I chose to use decimal to score grades instead of double as the nature of double makes it harder for comparisons and display. This avoids results like 84.9999999999997 instead of 85.0.

Clear(refresh) just one line but in two different tasks

When I run with 2 tasks, sometimes write datas in 3 (have to write always in 2 lines max)lines but idk why. Why is it do that? How can I fix it? Otherwise if I run with 1 task it is working well. I tried to commented the code.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace LifeBar
{
class Program
{
public static readonly int FullHealth = 200;
static void Main(string[] args)
{
List<Task> tasks = new List<Task>();
Console.Write("What is the player ID: "); int playerId = int.Parse(Console.ReadLine());
//playerID is equals with the line where is writed
var t1 = Task.Run(() =>
{
for (int i = FullHealth; i >= 0; i--)
{
WhatDraw(i, playerId);
System.Threading.Thread.Sleep(150);
}
});
//tasks.Add(t1);
//var t2 = Task.Run(() =>
//{
// for (int i = 200; i >= 0; i--)
// {
// WhatDraw(i, playerId + 1); (+1 cus I would like to write it to the next line)
// System.Threading.Thread.Sleep(200);
// }
//});
//tasks.Add(t2);
Task.WaitAll(tasks.ToArray());
Console.ReadKey();
}
this is a line deleter
public static void ClearCurrentConsoleLine(int playerId)
{
Task.Factory.StartNew(() =>
{
Console.SetCursorPosition(playerId, Console.CursorTop);
Console.Write(new string(' ', Console.WindowWidth));
Console.SetCursorPosition(playerId, Console.CursorTop);
});
}
they are retunrnig with the lifebar
/// <summary>
/// Draw healt bar
/// </summary>
/// <param name = life> i </param>
/// <returns>Health bar value</returns>
static string DrawHealth(int life)
{
string health = String.Empty;
if (life == 0) return "Frank dead";
else if (life < 10) return "< 10";
for (int i = 0; i < life / 10; i++)
{
health += " |"; // 1 amout of | equal with 10 health
}
return health;
}
/// <summary>
/// Draw armour bar
/// </summary>
/// <param name="life"> (i)</param>
/// <returns>Armour bar value</returns>
static string DrawArmour(int life)
{
string armour = "| | | | | | | | | |";
for (int i = 0; i < life / 10; i++)
{
armour += " |:|"; // 1 amout of |:| equal with 10 armour
}
return armour;
}
this is a depender
/// <summary>
/// Health or Armour draw
/// </summary>
/// <param name="fullLife">(i)</param>
/// <param name="playerId">playerId</param>
static void WhatDraw(int fullLife, int playerId)
{
Console.SetCursorPosition(0, playerId);
if (fullLife > 100)
{
double percent = fullLife / Convert.ToDouble(FullHealth);
Console.WriteLine($"Frank ({Math.Round(percent * 100)})% " + DrawArmour(fullLife - 100));
if (fullLife % 10 == 0)
{
Console.SetCursorPosition(playerId, Console.CursorTop-1);
ClearCurrentConsoleLine(playerId);
}
}
else
{
double percent = fullLife / Convert.ToDouble(FullHealth);
Console.WriteLine($"Frank ({Math.Round(percent * 100)}%) " + DrawHealth(fullLife));
if (fullLife % 10 == 0 && fullLife!=0)
{
Console.SetCursorPosition(playerId, Console.CursorTop-1);
ClearCurrentConsoleLine(playerId);
}
}
}
}
}
You are hitting a race condition. Consider this order of events:
// Thread 1
/* a */ Console.SetCursorPosition
/* b */
/* c */ Console.Write
// Thread 2
/* d */ Console.SetCursorPosition
/* e */
/* f */ Console.Write
a - Your first background task/thread runs and it sets the current cursor position
b - It's not yet run the next command, so we can consider it "sitting" at line b.
d - The second task sets the cursor position
c - The first task writes to the console.
Well, the second task has changed the position!! All of the properties on the Console class are static so the first task is no longer writing to the location where it moved the cursor, but rather where the second task did.
The timing could be even worse; for example the cursor position could be moved at the same time (a and d execute simultaneously) or the second task could move the cursor position while the first task is writing (d executes after c starts but before it completes). If both lines c and f are invoked at the same time, you might expect garbled output.
Of course this is a dumbed down explanation that applies more generally when concurrency is involved. The Console class itself is synchronized to prevent some of these cases, but it does not/cannot when a single "operation" involves multiple commands (set position + write). This coordination is up to you.
We can achieve this easily with C#'s lock statement (a shorthand for Monitor.Enter/Monitor.Exit) to ensure that our operations against the console cannot overlap:
private static readonly object lockobj = new object();
public static void ClearCurrentConsoleLine(int playerId)
{
lock(lockobj)
{
Console.SetCursorPosition(playerId, Console.CursorTop);
Console.Write(new string(' ', Console.WindowWidth));
Console.SetCursorPosition(playerId, Console.CursorTop);
}
}
static void WhatDraw(int fullLife, int playerId)
{
lock(lockobj)
{
Console.SetCursorPosition(0, playerId);
if (fullLife > 100)
{
double percent = fullLife / Convert.ToDouble(FullHealth);
Console.WriteLine($"Frank ({Math.Round(percent * 100)})% " + DrawArmour(fullLife - 100));
if (fullLife % 10 == 0)
{
Console.SetCursorPosition(playerId, Console.CursorTop-1);
ClearCurrentConsoleLine(playerId);
}
}
else
{
double percent = fullLife / Convert.ToDouble(FullHealth);
Console.WriteLine($"Frank ({Math.Round(percent * 100)}%) " + DrawHealth(fullLife));
if (fullLife % 10 == 0 && fullLife!=0)
{
Console.SetCursorPosition(playerId, Console.CursorTop-1);
ClearCurrentConsoleLine(playerId);
}
}
}
}
Because we need to synchronize a set of operations against the Console we need to surround all of those operations in a lock. This will force any concurrent operations to wait until the resource is no longer in use. The lock is reentrant on the same thread so when WhatDraw has the lock it can safely call ClearCurrentConsoleLine without needing to wait. Once you've locked access to a resource, best practices dictate that you should always lock around it, even for a single operation:
lock(lockobj)
Console.WriteLine("Single write");
I've also removed the Task.Run from the ClearCurrentConsoleLine method as it's pointless and adds unnecessary complexity, especially since we need to synchronize these operations. Remember that a Task is a promise to complete at some later point. By the time that actually happens you may not actually want to clear the line anymore! Better to just clear it synchronously when and as necessary.
Also note that lock cannot be used safely in async contexts and you'll need to use other synchronization primitives such as a SemaphoreSlim.

Proper Threading Technique in C# (using Timer + Receive Handler + BackgroundWorker)

I'm new to threading, and I am trying to use it in a program (front end for a hardware test), but running into heavy time delays - clearly I'm doing something wrong.
My Setup:
I have a hardware controller hooked up to my PC. I am communicating with it via serial to read values from 4 sensors. This communication sits within receiveHandler,
private void receiveHandler(object sender, DataStreamEventArgs e)
{
byte[] dataBytes = e.Response;
//throws it into a string
//parses the string with delimiters
//adds all items to a linked-list "queue"
}
I have a timer (System.Timers.Timer) that will count the duration of the hardware test, do one calculation and ask the controller for the updated state of each sensor.
private void OnTimedEvent(object sender, EventArgs e)
{
test.duration = ++tickCounter;
test.ampHoursOut = (test.ampHoursOut * 3600 + test.amperage * 1) / 3600;
sendToController("CH2.GETANALOG"); //repeat this line once for each sensor
}
I currently have a BackgroundWorker that will take the data collected from the controller (that I stuck in a "queue" in receiveHandler) and comprehend it + do some calculations + update the UI.
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker worker = (BackgroundWorker)sender;
while (!worker.CancellationPending)
{
//do a whole lot of logic
}
}
My Problem:
One of my controller inputs is reading the state of a hardware switch. I notice that there is a 5 second delay between when I hit the switch and when it's state changes on the UI.
There must be a heavy delay in my processing. I imagine BackgroundWorker isn't catching up with all of the information coming in. I'm asking for information every second (via the timer), so maybe this needs to slow down - but I would like to have information update every second to have an accurate test.
Am I using the timer, backgroundworker, and receive handler correctly? What can be done to speed up my communication + processing?
I'm a self-taught programmer and don't necessarily (as you can probably see) know all of the basic concepts. Any advice would be appreciated :)
Thanks, Julia
EDIT: Code:
//variables used (added relevant ones so code is better understood)
private static System.Timers.Timer timer;
LinkedList<string> incomingData = new LinkedList<string>();
string lastDatum = "";
/// <summary>
/// Method that is called when serial data is received from the controller.
/// The data is received in bytes, converted to a string and parsed at each "::" which represents a new message.
/// The last 'item' from the data is saved and added to the front of the next set of data, in case it is incomplete.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void receiveHandler(object sender, DataStreamEventArgs e)
{
byte[] dataBytes = e.Response;
string data = lastDatum + System.Text.Encoding.Default.GetString(dataBytes); // reads the next available line coming from the serial port
UpdateDataTextbox(data); // prints it out for debuging purposes
lastDatum = ""; // just in case
char[] delimiters = new char[] { ':' }; // splits the data at the ":" symbol, deleting empty entries.
Queue<string> dataQueue = new Queue<string>(data.Split(delimiters,
StringSplitOptions.RemoveEmptyEntries));
while(dataQueue.Count > 1)
{
string str = dataQueue.Dequeue().Replace("\r", string.Empty).Replace("\n", string.Empty).Replace(":", string.Empty).Trim(); // remove all useless characters
if (str != "")
{
incomingData.AddLast(str); // add to a queue that can be accessed by the background worker for processing & outputting.
}
}
lastDatum = dataQueue.Dequeue(); // Last data item in a transmission may be incomplete "CH1.GETDA" and thus it is appended to the front of the next list of data.
}
/// <summary>
/// Background Worker thread will be used to continue testing the connection to the controller,
/// process messages in the incoming message queue (actually a linked list),
/// and sends new messages for updated data.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker worker = (BackgroundWorker)sender;
while (!worker.CancellationPending)
{
if (!relayBoard.OpenConn())
{
MessageBox.Show("Connection to controller has been lost.");
testInterrupted = "Lost connection. Time = " + test.duration;
UpdateStatusText(false);
ShutErDown();
}
//update test time & Ah out
timeSpan = TimeSpan.FromSeconds(tickCounter);
UpdateDurationTextbox(timeSpan.ToString()); // update display with duration
UpdateAhTextbox(test.ampHoursOut.ToString());
/////////////////////////////////////////////////////
if (incomingData.Count > 0)
{
string dataLine = "";
try
{
dataLine = incomingData.First();
}
catch (System.InvalidOperationException emai)
{
break; //data hasn't come in yet, it will by the time this runs next.
}
incomingData.RemoveFirst();
if (dataLine.Contains("GET")) // some sensor values (analog/temp/digital) has come in
{
if (dataLine.Contains("GETANALOG")) // an analog value has come in
{
int index = dataLine.IndexOf("CH");
int pin = (int)Char.GetNumericValue(dataLine[index + 2]);
double value = 0;
int dataLineLength = dataLine.Length;
if (dataLineLength > 13) // value is appended to end of line
{
try
{
value = Convert.ToDouble(dataLine.Substring(13));
}
catch // can't convert to double
{
int index2 = dataLine.IndexOf("CH", 3);
if (index2 != -1) // there happen to be two sets of commands stuck together into one
{
string secondHalf = dataLine.Substring(index2);
incomingData.AddFirst(secondHalf);
}
}
}
else // value is on the next line
{
try
{
value = Convert.ToDouble(incomingData.First());
incomingData.RemoveFirst();
}
catch // can't convert to double
{
MessageBox.Show("Error occured: " + dataLine);
}
}
switch (pin)
{
case 1:
ReadVoltage(value);
break;
case 2:
ReadAmperage(value);
break;
}
}
else if (dataLine.Contains("GETTEMP")) // received reply with temperature data
{
int index = dataLine.IndexOf("CH");
int pin = (int)Char.GetNumericValue(dataLine[index + 2]); // using index of CH, retrieve which pin this message is coming from
double value = 0;
int dataLineLength = dataLine.Length;
if (dataLineLength > 11) // value is appended to end of line
{
try
{
value = Convert.ToDouble(dataLine.Substring(11));
}
catch // can't convert to double
{
int index2 = dataLine.IndexOf("CH", 3);
if (index2 != -1) // there happen to be two sets of commands stuck together into one
{
string secondHalf = dataLine.Substring(index2);
incomingData.AddFirst(secondHalf);
}
}
}
else // value is on the next line
{
value = Convert.ToDouble(incomingData.First());
incomingData.RemoveFirst();
}
ReadTemperature(pin, value);
}
else // must be CH3.GET
{
int index = dataLine.IndexOf("CH");
int pin = (int)Char.GetNumericValue(dataLine[index + 2]); // using index of CH, retrieve which pin this message is coming from
if (pin == 3) // only care if it's pin 3 (BMS), otherwise it's a mistake
{
double value = 0;
int dataLineLength = dataLine.Length;
if (dataLineLength > 7) // value is appended to end of line
{
try
{
value = Convert.ToDouble(dataLine.Substring(7));
}
catch // can't convert to double
{
int index2 = dataLine.IndexOf("CH", 3);
if (index2 != -1) // there happen to be two sets of commands stuck together into one
{
string secondHalf = dataLine.Substring(index2);
incomingData.AddFirst(secondHalf);
}
}
}
else // value is on the next line
{
value = Convert.ToDouble(incomingData.First());
incomingData.RemoveFirst();
}
ReadBMS(value);
}
}
}
else if (dataLine.Contains("REL")) // received reply about relay turning on or off.
{
if (dataLine.Contains("RELS")) // all relays turning on/off
{
if (dataLine.Contains("ON"))
{
for (int pin = 1; pin <= 4; pin++)
{
test.contactors[pin] = true;
}
}
else // "OFF"
{
for (int pin = 1; pin <= 4; pin++)
{
test.contactors[pin] = false;
}
}
}
else // single relay is turning on/off
{
int index = dataLine.IndexOf("REL");
int pin = (int)Char.GetNumericValue(dataLine[index + 3]);
if (dataLine.Contains("ON"))
{
test.contactors[pin] = true;
}
else if (dataLine.Contains("OFF"))
{
test.contactors[pin] = false;
}
else if (dataLine.Contains("GET"))
{
if (Convert.ToInt32(incomingData.First()) == 1)
test.contactors[pin] = true;
else
test.contactors[pin] = false;
incomingData.RemoveFirst();
}
}
}
}
}
}
/// <summary>
/// Main timer used to log the duration of the test, calculate amp hours
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void OnTimedEvent(object sender, EventArgs e)
{
test.duration = ++tickCounter;
// calculate Ah
test.ampHoursOut = (test.ampHoursOut * 3600 + test.amperage * 1) / 3600;
//read & output v, a, bms state
//sendToController("CH1.GETANALOG"); // get voltage
sendToController("CH2.GETANALOG"); // get amperage
sendToController("CH3.GET"); // get BMS state
//read & output temperature
sendToController("CH4.GETTEMP"); // get temperature
sendToController("CH5.GETTEMP");
}

Suppressing Frequencies From FFT

What I am trying to do is to retrieve the frequencies from some song and suppress all the frequencies that do not appear in the human vocal range or in general any range. Here is my suppress function.
public void SupressAndWrite(Func<FrequencyUnit, bool> func)
{
this.WaveManipulated = true;
while (this.mainWave.WAVFile.NumSamplesRemaining > 0)
{
FrequencyUnit[] freqUnits = this.mainWave.NextFrequencyUnits();
Complex[] compUnits = (from item
in freqUnits
select (func(item)
? new Complex(item.Frequency, 0) :Complex.Zero))
.ToArray();
FourierTransform.FFT(compUnits, FourierTransform.Direction.Backward);
short[] shorts = (from item
in compUnits
select (short)item.Real).ToArray();
foreach (short item in shorts)
{
this.ManipulatedFile.AddSample16bit(item);
}
}
this.ManipulatedFile.Close();
}
Here is my class for my wave.
public sealed class ComplexWave
{
public readonly WAVFile WAVFile;
public readonly Int32 SampleSize;
private FourierTransform.Direction fourierDirection { get; set; }
private long position;
/// <param name="file"></param>
/// <param name="sampleSize in BLOCKS"></param>
public ComplexWave(WAVFile file, int sampleSize)
{
file.NullReferenceExceptionCheck();
this.WAVFile = file;
this.SampleSize = sampleSize;
if (this.SampleSize % 8 != 0)
{
if (this.SampleSize % 16 != 0)
{
throw new ArgumentException("Sample Size");
}
}
if (!MathTools.IsPowerOf2(sampleSize))
{
throw new ArgumentException("Sample Size");
}
this.fourierDirection = FourierTransform.Direction.Forward;
}
public Complex[] NextSampleFourierTransform()
{
short[] newInput = this.GetNextSample();
Complex[] data = newInput.CopyToComplex();
if (newInput.Any((x) => x != 0))
{
Debug.Write("done");
}
FourierTransform.FFT(data, this.fourierDirection);
return data;
}
public FrequencyUnit[] NextFrequencyUnits()
{
Complex[] cm = this.NextSampleFourierTransform();
FrequencyUnit[] freqUn = new FrequencyUnit[(cm.Length / 2)];
int max = (cm.Length / 2);
for (int i = 0; i < max; i++)
{
freqUn[i] = new FrequencyUnit(cm[i], this.WAVFile.SampleRateHz, i, cm.Length);
}
Array.Sort(freqUn);
return freqUn;
}
private short[] GetNextSample()
{
short[] retval = new short[this.SampleSize];
for (int i = 0; i < this.SampleSize; i++)
{
if (this.WAVFile.NumSamplesRemaining > 0)
{
retval[i] = this.WAVFile.GetNextSampleAs16Bit();
this.position++;
}
}
return retval;
}
}
Both FFT forward and FFT backwards work correctly. Could you please tell me what my error is.
Unfortunately, human voice, even when singing, isn't in 'frequency range'. It usually has one main frequency and multitude of harmonics that follow it, depending on the phoneme.
Use this https://play.google.com/store/apps/details?id=radonsoft.net.spectralview&hl=en or some similar app to see what I mean - and then re-define your strategy. Also google 'karaoke' effect.
NEXT:
It's not obvious from your example, but you should scan whole file in windows (google 'fft windowing') to process it whole.

How to speed up this code to add chars to an string and extend it to desired length (all permutation)?

I have the following code: given a string with length m, add '-' to any possible position (including between different chars of original string) to extend it to length n. For example, given "ABC", extend it to 6. It will be "---ABC", "--A-BC", "--AB-C", "--ABC-", ......, "AB---C", "ABC---".
class Program
{
// <summary>
/// Grow a string to desired length with time limitation
/// </summary>
/// <param name="s"></param>
/// <param name="length"></param>
/// <param name="pad"></param>
/// <param name="Padded"></param>
public static bool ExtendToLen(string s, int length, char pad, ref List<string> Padded, ref Stopwatch timer, int timeOut)
{
if (s.Length == length)
{
Padded.Add(s);
return true;
}
else if (s.Length > length)
{
return true;
}
else
{
List<int> pos = GetExceptPos(s, pad.ToString());
pos.Sort();
int count = -1;
foreach (int p in pos)
{
if (timer.Elapsed.TotalSeconds > timeOut)
{
return false;
}
//Debug.WriteLine(string.Format("pos:{0}", p), "PadToLength");
count++;
// Pad left
string leftPadStr = s.Substring(0, p) + pad + s.Substring(p);
//Debug.WriteLine(string.Format("\tLeftPadStr:{0}", leftPadStr));
bool go = ExtendToLen(leftPadStr, length, pad, ref Padded, ref timer, timeOut);
if (go == false) { return false; }
// Pad right at the last pos
if (count == pos.Count - 1)
{
string rightPadStr = s + pad;
go = ExtendToLen(rightPadStr, length, pad, ref Padded, ref timer, timeOut);
if (go == false) { return false; }
}
}
return true;
}
}
/// <summary>
/// Find indexes of elements different from target str
/// </summary>
/// <param name="str"></param>
/// <param name="excludeStr"></param>
/// <returns></returns>
private static List<int> GetExceptPos(string str, string excludeStr)
{
List<int> allIndexes = new List<int>();
for (int i = 0; i < str.Length; i++)
{
allIndexes.Add(i);
}
return allIndexes.Except(str.IndexesOf(excludeStr)).ToList();
}
static void Main(string[] args)
{
string old = "ACGUA";
List<string> newList = new List<string>();
Stopwatch timer = new Stopwatch();
timer.Start();
bool isGood = ExtendToLen(old, 12, '-', ref newList, ref timer, 100);
timer.Stop();
foreach (string s in newList)
{
Console.WriteLine(s);
}
Console.WriteLine("Elapsed time: {0}", timer.Elapsed);
Console.ReadLine();
}
}
public static class ExtensionMethods
{
/// <summary>
/// Return all indeces
/// </summary>
/// <param name="haystack"></param>
/// <param name="needle"></param>
/// <returns></returns>
public static IEnumerable<int> IndexesOf(this string haystack, string needle)
{
int lastIndex = 0;
while (true)
{
int index = haystack.IndexOf(needle, lastIndex);
if (index == -1)
{
yield break;
}
yield return index;
lastIndex = index + needle.Length;
}
}
}
It runs slowly, for example, if I want to extend a string (len = 5) to 20, it runs long long long time. And the result seems to be redundant.
So the question is how to speed it up and remove those redundancy.
Any suggestion will be appreciated.
This is a very rough hack and should get you started.
It basically moves the string backwards through your padding one character at a time.
static unsafe void performPaddedBubbleSort(string s, char c, int padCount, IList<string> list) {
s = new string(c, padCount) + s;
bool complete = false;
int index = 0;
int count = 0;
fixed (char* p = s) {
while (count < s.Length && *p == c) {
while (index < s.Length) {
if (*(p + index) != c) {
// flip them
char tempChar = *(p + index);
if (index != 0)
*((p + index) - 1) = tempChar;
*(p + index) = c;
list.Add(new string(p));
}
index++;
}
index = 0;
count++;
}
}
}
Input of "ABC" with padding of 3 (to make it 6 chars wide) is:
--A-BC
--AB-C
--ABC-
-A-BC-
-AB-C-
-ABC--
A-BC--
AB-C--
ABC---
Elapsed time: 00:00:00.0008663
Not sure if that's exactly what you were after.. but its a start I guess. As you can see it executes in a fraction of a second.

Categories