We Keep Coding

Odd C# behavior

I'm a hobby programmer.
I tried to ask this question earlier on a very unstructured way (Sorry again), now I try to ask on the proper way.
I wrote the following code that seems to work unreliably.
The code was written like this for several reasons. I know it's messy but it should still work. To explain why I wrote it like this would mean that I need to explain several weeks' of work that is quite extensive. Please accept that this is at least the least worse option I could figure out. In the below sample I removed all sections of the code that are not needed to reproduce the error.
What this program does in a nutshell:
The purpose is to check a large number of parameter combinations for a program that receives streaming data. I simulate the original process to test parameter combinations.
First data is read from files that represents recorded streaming data.
Then the data is aggregated.
Then I build a list of parameters to test for.
Finally I run the code for each parameter combination in parallel.
Inside the parallel part I calculate a financial indicator called the bollinger bands. This is a moving average with adding +/- standard deviation. This means the upper line and the lower line should only be equal when variable bBandDelta = 0. However sometimes it happens that CandleList[slot, w][ctr].bollingerUp is equal to CandleList[slot, w][ctr].bollingerDown even when bBandDelta is not 0.
As a result I don't understand how can line 277 kick in. It seems that sometimes the program fails to write to the CandleList[slot, w][ctr]. However this should not be possible because (1) I lock the list and (2) I use ConcurrentBag. Could I have some help please?
Source files are here.
The code is:
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Collections.Concurrent;
namespace Justfortest
{
class tick : IComparable<tick> //Data element to represent a tick
{
public string disp_name; //ticker ID
public DateTime? trd_date; //trade date
public TimeSpan? trdtim_1; //trade time
public decimal trdprc_1; //price
public int? trdvol_1; //tick volume
public int CompareTo(tick other)
{
if (this.trdprc_1 == other.trdprc_1)
{
return other.trdprc_1.CompareTo(this.trdprc_1); //Return the later item
}
return this.trdprc_1.CompareTo(other.trdprc_1); //Return the earlier item
}
}
class candle : IComparable<candle> //Data element to represent a candle and all chart data calculated on candle level
{
public int id = 0;
public DateTime? openDate;
public TimeSpan? openTime;
public DateTime? closeDate;
public TimeSpan? closeTime;
public decimal open = 0;
public decimal high = 0;
public decimal low = 0;
public decimal close = 0;
public int? volume = 0;
public decimal totalPrice = 0;
public decimal bollingerUp = 0; //Bollinger upper line
public decimal bollingerDown = 0; //Bollinger below line
public int CompareTo(candle other)
{
if (totalPrice == other.totalPrice)
{
return other.totalPrice.CompareTo(totalPrice); //Return the later item
}
return totalPrice.CompareTo(other.totalPrice); //Return the earlier item
}
}
class param : IComparable<param> //Data element represent a trade event signal
{
public int par1;
public int bollPar;
public int par2;
public int par3;
public int par4;
public int par5;
public int par6;
public decimal par7;
public decimal par8;
public decimal par9;
public decimal par10;
int IComparable<param>.CompareTo(param other)
{
throw new NotImplementedException();
}
}
class programCLass
{
void myProgram()
{
Console.WriteLine("Hello");
Console.WindowWidth = 180;
string[] sources = new string[]
{
#"C:\test\source\sourceW1.csv",
#"C:\test\source\sourceW2.csv",
};
List<candle>[] sourceCandleList = new List<candle>[sources.Count()];
List<param> paramList = new List<param>(10000000);
var csvAnalyzer = new StringBuilder();
{
List<tick>[] updatelist = new List<tick>[sources.Count()];
Console.WriteLine("START LOAD");
for (var i = 0; i < sources.Count(); i++)
{
var file = sources[i];
updatelist[i] = new List<tick>();
// ---------- Read CSV file ----------
var reader = new StreamReader(File.OpenRead(file));
while (!reader.EndOfStream)
{
var line = reader.ReadLine();
var values = line.Split(',');
tick update = new tick();
update.disp_name = values[0].ToString();
update.trd_date = Convert.ToDateTime(values[1]);
update.trdtim_1 = TimeSpan.Parse(values[2]);
update.trdprc_1 = Convert.ToDecimal(values[3]);
update.trdvol_1 = Convert.ToInt32(values[4]);
updatelist[i].Add(update);
}
Console.WriteLine(i);
}
Console.WriteLine("END LOAD"); // All files are in the memory
// Aggreagate
Console.WriteLine("AGGREGATE START");
int tickAggr = 500;
for (var w = 0; w < sources.Count(); w++)
{
sourceCandleList[w] = new List<candle>();
List<tick> FuturesList = new List<tick>();
foreach (var update in updatelist[w])
{
tick t = new tick();
t.disp_name = update.disp_name.ToString();
t.trd_date = update.trd_date;
t.trdtim_1 = update.trdtim_1;
t.trdprc_1 = Convert.ToDecimal(update.trdprc_1);
t.trdvol_1 = update.trdvol_1;
// Add new tick to the list
FuturesList.Add(t);
if (FuturesList.Count == Math.Truncate(FuturesList.Count / (decimal)tickAggr) * tickAggr)
{
candle c = new candle();
c.openDate = FuturesList[FuturesList.Count - tickAggr].trd_date;
c.openTime = FuturesList[FuturesList.Count - tickAggr].trdtim_1;
c.closeDate = FuturesList.Last().trd_date;
c.closeTime = FuturesList.Last().trdtim_1;
c.open = FuturesList[FuturesList.Count - tickAggr].trdprc_1;
c.high = FuturesList.GetRange(FuturesList.Count - tickAggr, tickAggr).Max().trdprc_1;
c.low = FuturesList.GetRange(FuturesList.Count - tickAggr, tickAggr).Min().trdprc_1;
c.close = FuturesList.Last().trdprc_1;
c.volume = FuturesList.GetRange(FuturesList.Count - tickAggr, tickAggr).Sum(tick => tick.trdvol_1);
c.totalPrice = (c.open + c.high + c.low + c.close) / 4;
sourceCandleList[w].Add(c);
if (sourceCandleList[w].Count == 1)
{
c.id = 0;
}
else
{
c.id = sourceCandleList[w][sourceCandleList[w].Count - 2].id + 1;
}
}
}
FuturesList.Clear();
}
Console.WriteLine("AGGREGATE END");
for (var i = 0; i < sources.Count(); i++)
{
updatelist[i].Clear();
}
}
Console.WriteLine("BUILD PARAMLIST");
for (int par1 = 8; par1 <= 20; par1 += 4) // parameter deployed
{
for (int bollPar = 10; bollPar <= 25; bollPar += 5) // parameter deployed
{
for (int par2 = 6; par2 <= 18; par2 += 4) // parameter deployed
{
for (int par3 = 14; par3 <= 20; par3 += 3) // parameter deployed
{
for (int par4 = 10; par4 <= 20; par4 += 5) // parameter deployed
{
for (int par5 = 4; par5 <= 10; par5 += 2) // parameter deployed
{
for (int par6 = 5; par6 <= 30; par6 += 5)
{
for (decimal par7 = 1.0005M; par7 <= 1.002M; par7 += 0.0005M)
{
for (decimal par8 = 1.002M; par8 <= 1.0048M; par8 += 0.0007M)
{
for (decimal par9 = 0.2M; par9 <= 0.5M; par9 += 0.1M)
{
for (decimal par10 = 0.5M; par10 <= 2; par10 += 0.5M)
{
param p = new param();
p.par1 = par1;
p.bollPar = bollPar;
p.par2 = par2;
p.par3 = par3;
p.par4 = par4;
p.par5 = par5;
p.par6 = par6;
p.par7 = par7;
p.par8 = par8;
p.par9 = par9;
p.par10 = par10;
paramList.Add(p);
}
}
}
}
}
}
}
}
}
}
}
Console.WriteLine("END BUILD PARAMLIST, scenarios to test:{0}", paramList.Count);
var sourceCount = sources.Count();
sources = null;
Console.WriteLine("Start building pools");
int maxThreads = 64;
ConcurrentBag<int> pool = new ConcurrentBag<int>();
List<candle>[,] CandleList = new List<candle>[maxThreads, sourceCount];
for (int i = 0; i <= maxThreads - 1; i++)
{
pool.Add(i);
for (int w = 0; w <= sourceCount - 1; w++)
{
CandleList[i, w] = sourceCandleList[w].ConvertAll(p => p);
}
}
Console.WriteLine("End building pools");
int pItemsProcessed = 0;
Parallel.ForEach(paramList,
new ParallelOptions { MaxDegreeOfParallelism = maxThreads },
p =>
{
int slot = 1000;
while (!pool.TryTake(out slot));
var bollPar = p.bollPar;
decimal bollingerMiddle = 0;
double bBandDeltaX = 0;
for (var w = 0; w < sourceCount; w++)
{
lock (CandleList[slot, w])
{
for (var ctr = 0; ctr < CandleList[slot, w].Count; ctr++)
{
CandleList[slot, w][ctr].bollingerUp = 0; //Bollinger upper line
CandleList[slot, w][ctr].bollingerDown = 0; //Bollinger below line
//Bollinger Bands Calculation
if (ctr + 1 >= bollPar)
{
bollingerMiddle = 0;
bBandDeltaX = 0;
for (int i = 0; i <= bollPar - 1; i++)
{
bollingerMiddle = bollingerMiddle + CandleList[slot, w][ctr - i].totalPrice;
}
bollingerMiddle = bollingerMiddle / bollPar; //average
for (int i = 0; i <= bollPar - 1; i++)
{
bBandDeltaX = bBandDeltaX + (double)Math.Pow(System.Convert.ToDouble(CandleList[slot, w][ctr - i].totalPrice) - System.Convert.ToDouble(bollingerMiddle), 2);
}
bBandDeltaX = bBandDeltaX / bollPar;
decimal bBandDelta = (decimal)Math.Sqrt(System.Convert.ToDouble(bBandDeltaX));
CandleList[slot, w][ctr].bollingerUp = bollingerMiddle + 2 * bBandDelta;
CandleList[slot, w][ctr].bollingerDown = bollingerMiddle - 2 * bBandDelta;
if (CandleList[slot, w][ctr].bollingerUp == CandleList[slot, w][ctr].bollingerDown)
{
Console.WriteLine("?! Items processed=" + pItemsProcessed + " bollPar=" + bollPar + " ctr=" + ctr + " bollingerMiddle=" + bollingerMiddle + " bBandDeltaX=" + bBandDeltaX + " bBandDelta=" + bBandDelta + " bollingerUp=" + CandleList[slot, w][ctr].bollingerUp + " bollingerDown=" + CandleList[slot, w][ctr].bollingerDown);
}
}
// REMOVED Further calculations happen here
}
// REMOVED Some evaluations happen here
}
}
// REMOVED Some more evaluations happen here
Interlocked.Increment(ref pItemsProcessed);
pool.Add(slot);
});
}
static void Main(string[] args)
{
var P = new programCLass();
P.myProgram();
}
}
}

In C# arrays, how not to create duplicated random numbers?

I'm a beginner in C#, trying to make a lottery form applicaton.
There are types, first when you have 5 tips ( otos bool ) and 5 tips ( hatos bool ).
And there are many types of how many numbers will be raffled (tiz, harminc, kilencven, negyvenot).
I tried to scan the numbers after the raffle with Array.Equals with this code:
for (int i = 0; i <= 4; i++)
{
for (int y = 0; y <= 4; y++)
{
if (Array.Equals(lottoszamok[i], lottoszamok[y]))
lottoszamok[i] = r.Next (1, ?);
}
}
but at this the number will be scanned with itself too, so it will be always equal.
here is my code by the way:
if (otos == true)
{
for (int i = 0; i <= 5; i++)
{
if (tiz == true)
{
lottoszamok[i] = r.Next(1, 10);
}
else if (harminc == true)
{
lottoszamok[i] = r.Next(1, 30);
}
else if (kilencven == true)
{
lottoszamok[i] = r.Next(1, 90);
}
else if (negyvenot == true)
{
lottoszamok[i] = r.Next(1, 45);
}
else if (egyeni == true)
{
lottoszamok[i] = r.Next(1, (egyeniertek + 1));
}
}
}
if (hatos == true)
{
for (int i = 0; i <= 6; i++)
{
if (tiz == true)
{
lottoszamok[i] = r.Next(1, 10);
}
else if (harminc == true)
{
lottoszamok[i] = r.Next(1, 30);
}
else if (kilencven == true)
{
lottoszamok[i] = r.Next(1, 90);
}
else if (negyvenot == true)
{
lottoszamok[i] = r.Next(1, 45);
}
else if (egyeni == true)
{
lottoszamok[i] = r.Next(1, (egyeniertek + 1));
}
}
}

If you're trying to pick numbers from a range 1..n without repetitions, you need to "shuffle" the numbers out:
int[] allPossibleNumbers = Enumerable.Range(1, maxNumber).ToArray();
int[] picked = new int[numberToPick];
for (int i = 0; i < numberToPick; i++)
{
int index = r.Next(i, maxNumber);
picked[i] = allPossibleNumbers[index];
allPossibleNumbers[index] = allPossibleNumbers[i];
}
where numberToPick is 5 if otos or 6 if hatos, and maxNumber depends on tiz, harminc, kilencven, negyvenot, egyeni and egyeniertek.
If your maxNumber is huge and you only want to pick a few numbers, the following doesn't require the whole range to be in memory at once:
Dictionary<int, int> outOfPlace = new Dictionary<int,int>();
int[] picked = new int[numberToPick];
for (int i = 0; i < numberToPick; i++)
{
int shuffleOut = outOfPlace.ContainsKey(i) ? outOfPlace[i] : i;
int index = r.Next(i, maxNumber);
picked[i] = 1 + (outOfPlace.ContainsKey(index) ? outOfPlace[index] : index);
outOfPlace[index] = shuffleOut;
outOfPlace.Remove(i);
}

Try this one!
if (i!=y && Array.Equals(lottoszamok[i], lottoszamok[y]))

I made it this way, if you want you could put swapping like method.
static void SwapInts(int[] array, int position1, int position2)
{
// Swaps elements in an array.
int temp = array[position1]; // Copy the first position's element
array[position1] = array[position2]; // Assign to the second element
array[position2] = temp; // Assign to the first element
}
static void Main()
{
Random rng = new Random();
int n = int.Parse(Console.ReadLine());
int[] intarray = new int[n];
for (int i = 0; i < n; i++)
{
// Initialize array
intarray[i] = i + 1;
}
// Exchange resultArray[i] with random element in resultArray[i..n-1]
for (int i = 0; i < n; i++)
{
int positionSwapElement1 = i + rng.Next(0, n - i);
SwapInts(intarray, i, positionSwapElement1);
}
for (int i = 0; i < n; i++)
{
Console.Write(intarray[i] + " ");
}
}
}

I spend many time to get this, but i believe i can do it, now it's done, By the Easier way in the word, this kill every think about Random not duplicate,very simply code without any philosophy or difficulty of Developers made ... (welcome to my work) that (BEST OF THE BEST):
Numbers between (1-10) without any duplicate, 1- MY WORK in C#
private void TenNumbersRandomly()
{
int[] a = new int[10];
Random r = new Random();
int x;
for (int i = 0; i < 10; i++)
{
x= r.Next(1, 11);
for (int j = 0; j <= i ; j++)
{
while (a[j] == x)
{
x = r.Next(1, 11);
j = 0;
}
}
a[i] = x;
tb1.Text += a[i]+"\n";
}
}
2- in VB some Different i also have it :
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim x As Integer, i As Integer, j As Integer
x = Int(Rnd() * 10) + 1
Label1.Text = ""
Dim a(9) As Integer
For i = 0 To 9
x = Int(Rnd() * 10) + 1
For j = 0 To i
While (a(j) = x)
x = Int(Rnd() * 10) + 1
j = 0
End While
Next j
a(i) = x
Label1.Text += a(i).ToString() + " "
Next i

Training ff nn using backpropagation

I have encountered some difficulties while training my nn. When I use, lets say, 10 training sets, at the end of training procces neural network is trained just for the last two. I'm entering same values that I have used to train network and I am getting wrong results save for the last two. It seems to me that new nn memory suppresses older memory. I'm using 64 input neurons, 42 neurons in hidden layer and one output neuron. Sigmoid function is used for activating neurons. Training inputs and expected outputs are in 0 to 1 range. Does anyone have any clue what might be causing the problem?
Neuron b = new Neuron();
Fft f = new Fft();
float e = 2.71828f;
float eta = 0.05f;
float alpha = 0.05f;
float[] saw = new float[42];
float[] dh = new float[42];
float error = 0;
float dto = 0;
Random broj = new Random();
TextReader br = new StreamReader("d:/trening.txt");
TextReader ir = new StreamReader("d:\\input.txt");
float NextFloat(Random rng, float min, float max)
{
return (float)(min + (rng.NextDouble() * (max - min)));
}
public void load()//load memory
{
int i, j;
byte[] floatBytes;
BinaryReader br = new BinaryReader(File.Open("d:/memorija.txt", FileMode.Open));
for (j = 0; j <= 41; j++)
{
for (i = 0; i <= 64; i++)
{
floatBytes = br.ReadBytes(4);
b.w12[i][j] = BitConverter.ToSingle(floatBytes, 0);
}
}
for (j = 0; j <= 1; j++)
{
for (i = 0; i <= 41; i++)
{
floatBytes = br.ReadBytes(4);
b.w23[i][j] = BitConverter.ToSingle(floatBytes, 0);
}
}
br.Close();
}
public void trening()//Get training inputs and expected outputs
{ //Calls process methode
int i, n,ct=0;
using (TextReader tr = new StreamReader("d:/trening.txt"))
{
do
{
ct++;
} while (tr.ReadLine() != null);
tr.Close();
}
for (n = 0; n < (ct-1)/65; n++)
{
for (i = 1; i <= 65; i++)
b.input[i] = Convert.ToSingle(br.ReadLine());
process(b.input[65]);
target.Text = ((b.input[65]).ToString());
}
}
public void process(double t)//Trains nn using backpropagation
{
error = 0;
do
{
int i, j, k;
BinaryWriter bw = new BinaryWriter(File.Open("d:\\memorija.txt", FileMode.Create));
i = k = j = 0;
for (j = 1; j <= 41; j++)
{
b.ulaz2[j] = b.w12[0][j];
for (i = 1; i <= 64; i++)
{
b.ulaz2[j] += b.input[i] * b.w12[i][j];
} b.izlaz2[j] = (float)(1.0 / (1.0 + Math.Pow(e, -b.ulaz2[j])));
if (b.izlaz2[j] < 0)
MessageBox.Show(b.izlaz2[j].ToString());
}
for (k = 1; k <= 1; k++)
{
b.ulaz3 = b.w23[0][k];
for (j = 1; j <= 41; j++)
{
b.ulaz3 += b.izlaz2[j] * b.w23[j][k];
} b.izlaz = (float)(1.0 / (1.0 + Math.Pow(e, -b.ulaz3)));
error += (float)(0.5 * (t - b.izlaz) * (t - b.izlaz));
dto = (float)(t - b.izlaz) * b.izlaz * (1 - b.izlaz);
}
for (j = 1; j <= 41; j++)
{
saw[j] = 0;
for (k = 1; k <= 1; k++)
{
saw[j] += dto * b.izlaz2[j];
} dh[j] = saw[j] * b.izlaz2[j] * (1 - b.izlaz2[j]);
}
for (j = 1; j <= 41; j++)
{
b.w12d[0][j] = eta * dh[j] + alpha * b.w12d[0][j];
b.w12[0][j] += b.w12d[0][j];
for (i = 1; i <= 64; i++)
{
b.w12d[i][j] = eta * b.input[i] * dh[j] + alpha * b.w12d[i][j];
b.w12[i][j] += b.w12d[i][j];
}
}
for (k = 1; k <= 1; k++)
{
b.w23d[0][k] = eta * dto + alpha * b.w23d[0][k];
b.w23[0][k] += b.w23d[0][k];
for (j = 1; j <= 41; j++)
{
b.w23d[j][k] = eta * b.izlaz2[j] * dto + alpha * b.w23d[j][k];
b.w23[j][k] += b.w23d[j][k];
}
}
for (j = 0; j <= 41; j++)
{
for (i = 0; i <= 64; i++)
bw.Write(b.w12[i][j]);
}
for (j = 0; j <= 1; j++)
{
for (i = 0; i <= 41; i++)
bw.Write(b.w23[i][j]);
}
bw.Close();
izlazb.Text = Convert.ToString(b.izlaz);
errorl.Text = Convert.ToString(Math.Abs(b.izlaz - b.input[64]));
} while (Math.Abs(b.izlaz - t) > 0.03);
}
public void test()//This methode gets input values and gives output based on previous training
{
int i = 0, j = 0, k = 0;
for (i = 1; i < 65; i++)
b.input[i] = (float)Convert.ToDouble(ir.ReadLine());
for (j = 1; j <= 41; j++)
{
b.ulaz2[j] = b.w12[0][j];
for (i = 1; i <= 64; i++)
{
b.ulaz2[j] += b.input[i] * b.w12[i][j];
} b.izlaz2[j] = (float)(1.0 / (1.0 + Math.Pow(e, -b.ulaz2[j])));
}
for (k = 1; k <= 1; k++)
{
b.ulaz3 = b.w23[0][k];
for (j = 1; j <= 41; j++)
{
b.ulaz3 += b.izlaz2[j] * b.w23[j][k];
} b.izlaz = (float)(1.0 / (1.0 + Math.Pow(e, -b.ulaz3)));
} izlazb.Text = Convert.ToString(b.izlaz);
target.Text = "/";
errorl.Text = "/";
}
public void reset()//Resets memory
{
BinaryWriter fw = new BinaryWriter(File.Open("d:\\memorija.txt", FileMode.Create));
int i = 0;
int j = 0;
Random broj = new Random();
for (j = 0; j <= 41; j++)
{
for (i = 0; i <= 64; i++)
{
b.w12[i][j] = 0;
b.w12[i][j] = 2 * (NextFloat(broj, -0.5f, 0.5f));
fw.Write(b.w12[i][j]);
}
}
for (j = 0; j <= 1; j++)
{
for (i = 0; i <= 41; i++)
{
b.w23[i][j] = 0;
b.w23[i][j] = 2 * (NextFloat(broj, -0.5f, 0.5f));
fw.Write(b.w23[i][j]);
}
}
fw.Close();
}
}
}
And neuron class
public class Neuron
{
public float[][] w12 = new float[65][];//(65, 42);
public float[][] w12d = new float[65][];//(65, 42);
public float[][] w23 = new float[42][];//(42,2);
public float[][] w23d = new float[42][];//(42, 2);
public float[] ulaz2 = new float[42];
public float[] izlaz2 = new float[42];
public float ulaz3;
public float[] input =new float[66];
public static float[] ioutput;
public float izlaz;
public void arrayInit()
{
int i, j;
for (i = 0; i <=64; i++)
{
w12[i] = new float[42];
w12d[i] = new float[42];
}
for (i = 0; i <42; i++)
{
w23[i] = new float[2];
w23d[i] = new float[2];
}
for (j = 0; j < 42; j++)
for (i = 0; i <=64; i++)
{
w12[i][j] = 0;
w12d[i][j] = 0;
}
for (j = 0; j < 2; j++)
for (i = 0; i < 42; i++)
{
w23[i][j] = 0;
w23d[i][j] = 0;
}
}
}

I found out what the problem was. I didn't mix training arrays, I was introducing one array to nn until it was trained for it, instead of introducing all arrays in cyclic manner. I hope this will be useful for someone.

Equations with Array Bidimenensional in C#

I have an array with 310 lines and 120 columns.
I get the data that will populate this array from serial port at intervals of 10 seconds.
How to send this data to the first row of the array and on the next iteration to the line down and it continuously until the operation is complete?
Once completed, take the maximum, minimum and average of the array. And finally, the average, maximum and minimum of a selection of cells in the Array.
This is possible with array in C#?
([7,31]-xxxxx.MIN([$28$5:$45$95])/(xxxxx.MAX[$28$5:$46$95]-xxxxx.MIN[$28$5:$45$95])

The comments you made below your question clarified it a bit, but the screwy nature of comments jumbled this line:
...receive a string like this  0#,22008,21930,00000, n / a, n / a !
But only use the 0#,22008,21930,00000. The...
So I'll just assume for now that those weird characters are an endline. Adapt it otherwise.
A set arraylength of 120 is completely unnecessary, and a 2-dimensional array adds no new methods usable here. We'll just use a jagged array instead. The length of 310 probably has some reason, so we'll keep that.
So on each row, the first 24 "cells" are the input from the stream. I'm unfamiliar with ports, but I can safely assume you can route that RS485/USB to a Stream object.
Cells 0, 4, 8, 12, 16 and 20 are filled with IDs, one of {01, 02, 03, 04, 05, 06} to indicate what LPC it came from.
But a bit later in the comments you specify wanting this:
([32,6]-MIN([28,5]:[41,96]))/(MAX([28,5]:[41,96])-MIN([28,5]:[41,96]))
Even in Excel, this would take the mini/max/average over those IDs as well. We're avoiding that in this code though, it ignores those columns.
class LaserArray
{
private int minimum;
private int maximum;
private double average;
private bool finishedReading;
private StreamReader sr;
public int Minimum
{ get { return finishedReading ? minimum : -1; } }
public int Maximum
{ get { return finishedReading ? maximum : -1; } }
public double Average
{ get { return finishedReading ? average : -1; } }
public bool FinishedReading
{ get { return finishedReading; } }
public bool StreamInitialized
{ get { return sr != null; } }
private int[][] arr;
public LaserArray()
{
arr = new int[310][];
finishedReading = false;
}
public bool InitStream(Stream s)
{
try
{
sr = new StreamReader(s);
/*alternatively, as I have no clue about your Stream:
* sr = new StreamReader(s, bool detectEncodingFromByteOrderMarks)
* sr = new StreamReader(s, Encoding encoding)
* sr = new StreamReader(s, Encoding encoding, bool detectEncodingFromByteOrderMarks)
* sr = new StreamReader(s, Encoding encoding, bool detectEncodingFromByteOrderMarks, int buffersize)
* */
}
catch(Exception)
{
Console.WriteLine("Invalid Stream object.");
sr = null;
return false;
}
return true;
}
public void ReadInData()
{
if (sr == null)
{
Console.WriteLine("Initialize a Stream with UseStream first.");
return;
}
if (finishedReading)
{
Console.WriteLine("The stream is already read.");
return;
}
minimum = int.MaxValue; maximum = 0;
int currentTotal = 0;
for (int rowCounter = 0; rowCounter < 310; rowCounter++)
{
arr[rowCounter] = new int[24];
int indexCounter = 0;
for (int i = 0; i < 6; i++)
{ // 0#,22008,21930,00000, n / a, n / a !
char[] buffer = new char[28]; //123456789012345678901234 5 67 8 makes 28 characters?
try
{
sr.ReadBlock(buffer, 0, 2 + 5 + 1);
}
catch (IOException e)
{
//some error occurred
Console.WriteLine("IOException: " + e.Message);
}
string input = new String(buffer);
arr[rowCounter][indexCounter] = int.Parse(input.Substring(2, 2));
indexCounter++;
int currentNumber;
currentNumber = int.Parse(input.Substring(6, 5));
arr[rowCounter][indexCounter] = currentNumber;
currentTotal += currentNumber;
if (currentNumber > maximum)
maximum = currentNumber;
if (currentNumber < minimum)
maximum = currentNumber;
indexCounter++;
currentNumber = int.Parse(input.Substring(12, 5));
arr[rowCounter][indexCounter] = currentNumber;
currentTotal += currentNumber;
if (currentNumber > maximum)
maximum = currentNumber;
if (currentNumber < minimum)
maximum = currentNumber;
indexCounter++;
currentNumber = int.Parse(input.Substring(18, 5));
arr[rowCounter][indexCounter] = currentNumber;
currentTotal += currentNumber;
if (currentNumber > maximum)
maximum = currentNumber;
if (currentNumber < minimum)
maximum = currentNumber;
indexCounter++;
}
}
average = currentTotal / (double) 310;
//succesfully read in 310 lines of data
finishedReading = true;
}
public int GetMax(int topRow, int leftColumn, int bottomRow, int rightColumn)
{
if (!finishedReading)
{
Console.WriteLine("Use ReadInData first.");
return -1;
}
int max = 0;
for (int i = topRow; i <= bottomRow; i++)
for (int j = leftColumn; j <= rightColumn; j++)
{
if (j == 0 || j == 4 || j == 8 || j == 12 || j == 16 || j == 20 || j == 24)
continue;
if (arr[i][j] > max)
max = arr[i][j];
}
return max;
}
public int GetMin(int topRow, int leftColumn, int bottomRow, int rightColumn)
{
if (!finishedReading)
{
Console.WriteLine("Use ReadInData first.");
return -1;
}
int min = 99999;
for (int i = topRow; i <= bottomRow; i++)
for (int j = leftColumn; j <= rightColumn; j++)
{
if (j == 0 || j == 4 || j == 8 || j == 12 || j == 16 || j == 20 || j == 24)
continue;
if (arr[i][j] < min)
min = arr[i][j];
}
return min;
}
public double GetAverage(int topRow, int leftColumn, int bottomRow, int rightColumn)
{
if (!finishedReading)
{
Console.WriteLine("Use ReadInData first.");
return -1;
}
int total = 0;
int counter = 0;
for (int i = topRow; i <= bottomRow; i++)
for (int j = leftColumn; j <= rightColumn; j++)
{
if (j == 0 || j == 4 || j == 8 || j == 12 || j == 16 || j == 20 || j == 24)
continue;
counter++;
total += arr[i][j];
}
return total / (double) 310;
}
}
This should be self-explanatory. Create a LaserArray, init the stream, readIn the data, and then have fun with the results.
Also, I'm bored and on vacation, so that's why I'm answering a year-old question in such great detail.
Also reputation.

Need to optimise counting positive and negative values

I need to optimise code that counts pos/neg values and remove non-qualified values by time.
I have queue of values with time-stamp attached.
I need to discard values which are 1ms old and count negative and positive values. here is pseudo code
list<val> l;
v = q.dequeue();
deleteold(l, v.time);
l.add(v);
negcount = l.count(i => i.value < 0);
poscount = l.count(i => i.value >= 0);
if(negcount == 10) return -1;
if(poscount == 10) return 1;
I need this code in c# working with max speed. No need to stick to the List. In fact arrays separated for neg and pos values are welcome.
edit: probably unsafe arrays will be the best. any hints?
EDIT: thanks for the heads up.. i quickly tested array version vs list (which i already have) and the list is faster: 35 vs 16 ms for 1 mil iterations...
Here is the code for fairness sake:
class Program
{
static int LEN = 10;
static int LEN1 = 9;
static void Main(string[] args)
{
Var[] data = GenerateData();
Stopwatch sw = new Stopwatch();
for (int i = 0; i < 30; i++)
{
sw.Reset();
ArraysMethod(data, sw);
Console.Write("Array: {0:0.0000}ms ", sw.ElapsedTicks / 10000.0);
sw.Reset();
ListMethod(data, sw);
Console.WriteLine("List: {0:0.0000}ms", sw.ElapsedTicks / 10000.0);
}
Console.ReadLine();
}
private static void ArraysMethod(Var[] data, Stopwatch sw)
{
int signal = 0;
int ni = 0, pi = 0;
Var[] n = new Var[LEN];
Var[] p = new Var[LEN];
for (int i = 0; i < LEN; i++)
{
n[i] = new Var();
p[i] = new Var();
}
sw.Start();
for (int i = 0; i < DATALEN; i++)
{
Var v = data[i];
if (v.val < 0)
{
int x = 0;
ni = 0;
// time is not sequential
for (int j = 0; j < LEN; j++)
{
long diff = v.time - n[j].time;
if (diff < 0)
diff = 0;
// too old
if (diff > 10000)
x = j;
else
ni++;
}
n[x] = v;
if (ni >= LEN1)
signal = -1;
}
else
{
int x = 0;
pi = 0;
// time is not sequential
for (int j = 0; j < LEN; j++)
{
long diff = v.time - p[j].time;
if (diff < 0)
diff = 0;
// too old
if (diff > 10000)
x = j;
else
pi++;
}
p[x] = v;
if (pi >= LEN1)
signal = 1;
}
}
sw.Stop();
}
private static void ListMethod(Var[] data, Stopwatch sw)
{
int signal = 0;
List<Var> d = new List<Var>();
sw.Start();
for (int i = 0; i < DATALEN; i++)
{
Var v = data[i];
d.Add(new Var() { time = v.time, val = v.val < 0 ? -1 : 1 });
// delete expired
for (int j = 0; j < d.Count; j++)
{
if (v.time - d[j].time < 10000)
d.RemoveAt(j--);
else
break;
}
int cnt = 0;
int k = d.Count;
for (int j = 0; j < k; j++)
{
cnt += d[j].val;
}
if ((cnt >= 0 ? cnt : -cnt) >= LEN)
signal = 9;
}
sw.Stop();
}
static int DATALEN = 1000000;
private static Var[] GenerateData()
{
Random r = new Random(DateTime.Now.Millisecond);
Var[] data = new Var[DATALEN];
Var prev = new Var() { val = 0, time = DateTime.Now.TimeOfDay.Ticks};
for (int i = 0; i < DATALEN; i++)
{
int x = r.Next(20);
data[i] = new Var() { val = x - 10, time = prev.time + x * 1000 };
}
return data;
}
class Var
{
public int val;
public long time;
}
}

To get negcount and poscount, you are traversing the entire list twice.
Instead, traverse it once (to compute negcount), and then poscount = l.Count - negcount.

Some ideas:
Only count until max(negcount,poscount) becomes 10, then quit (no need to count the rest). Only works if 10 is the maximum count.
Count negative and positive items in 1 go.
Calculate only negcount and infer poscount from count-negcount which is easier to do than counting them both.
Whether any of them are faster than what you have now, and which is fastest, depends among other things on what the data typically looks like. Is it long? Short?
Some more about 3:
You can use trickery to avoid branches here. You don't have to test whether the item is negative, you can add its negativity to a counter. Supposing the item is x and it is an int, x >> 31 is 0 for positive x and -1 for negative x. So counter -= x >> 31 will give negcount.
Edit: unsafe arrays can be faster, but shouldn't be in this case, because the loop would be of the form
for (int i = 0; i < array.Length; i++)
do something with array[i];
Which is optimized by the JIT compiler.