Finding standard deviation/variance in data values [duplicate] - c#

This question already has answers here:
How do I determine the standard deviation (stddev) of a set of values?
(12 answers)
Closed 9 years ago.
So this is quite a complex problem and I googled alot about it but haven't really come up with anything. So the problem, I need to find the Standard Deviation of some variables within files. So let me state what I need to do: I have the code to find the average value of some numbers extracted from files. What I need to do with that average value is subtract it from the value in the files and then take that new value and square it.
Code to find the average value:
var query5 = from file in fileEntries
doc = XDocument.Load(file)
let x = doc.Descendants("").Single()
let y = doc.Descendants("").Single()
let z = doc.Descendants("")Single()
select new
{
X1 = x.Element("Max").Value,
X2 = x.Element("Min").Value,
Y1 = y.Element("Max").Value,
Y2 = y.Element("Min").Value,
Z1 = z.Element("Max").Value,
Z2 = z.Element("Min").Value
};

Given your above, if you want the population variance and standard deviation, you could do:
// ... code from above
double averageMaximumX = query.Average(t => double.Parse(t.XMax));
double varianceMaximumX = query.Sum(t =>
Math.Pow(double.Parse(t.XMax) - averageMaximumX, 2)));
double stdDevMaximumX = Math.Sqrt(varianceMaximumX);
varianceMaximumX /= query.Count();

Related

The result of calculation deviates after some digits [duplicate]

This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 3 years ago.
Result of calculation shown below should be equal to 0,007306897 (I referred from 2 books) . But when i check result from Watch 1, i see that result is equal to 0,007306882. I split the process into some parts. And the problem is occurring when c is calculating.
///Declarations
double sigma = 1.00000000;
double a,b,e,c;
a = (1 / Math.Sqrt(2 * Math.PI)); //calculated properly
c = -(i * i + j * j) / 2.00000000 * (sigma * sigma); //i and j are equal to -2
e = Math.E; //calculated properly
b = Math.Pow(e, c);
result=a * b;
Unfortunately the double type is not highly accurate. The link below shows that some numbers like 1.05 can not be stored accurately by the double type.
http://www.binaryconvert.com/convert_double.html
For normal usage it is usually accurate enough, if you need accuracy to the level you describe you probably need to use something like binary coded decimal. That will mean the normal math library won't work. You could try asking on some physics sites to see what they use when modelling complex systems to maintain accuracy.

What parameters should I be using for the LogNormal and Normal Distribution in MATH.NET

I've tried several combinations of Mathdotnet's LogNormal and Normal classes: https://numerics.mathdotnet.com/api/MathNet.Numerics.Distributions/LogNormal.
I seem to get a lot closer to the result I'm looking for using the mean and standard deviation as parameters. However, I notice that when I use larger numbers, like numberOfMinutes my results do not deviate past the mean like they do with smaller numbers like numberOfDays do. I know I'm not thinking about this right and could use some help.
Also, I'd like to use the geometric mean vs the mean but I didn't know what parameter to use for the variance given I couldn't pinpoint how to even use it for the mean.
Finally, I hope the answer to this also answers the same issue I'm having with the Normal distribution.
List<double> numberOfDays = new List<double> { 10, 12, 18, 30 };
double mean = numberOfDays.Mean(); // 17.5
double geometricMean = numberOfDays.GeometricMean(); // 15.954
double variance = numberOfDays.Variance(); // 81
double standardDeviation = numberOfDays.StandardDeviation(); // 9
// Do I need a Geometric Standard Deviation or Variance
double numberOfDaysSampleMV = LogNormal.WithMeanVariance(mean, variance).Sample(); // One example sample yielded 40.23
double numberOfDaysSampleMSD = LogNormal.WithMeanVariance(mean, standardDeviation).Sample(); // One example sample yielded 17.33
I believe you are confused about the parameters required. Using conventional notation, you have set X which you believe is LogNormal:
X = { 10, 12, 18, 30 }
mean: m = 17.5
standard deviation: sd = 9
from this you derive set Y which is Normal:
Y = {2.30,2.48,2.89,3.4}
mean: mu = 2.77
standard deviation: sigma = 0.487
Note that mu and sigma are computed from Y, not X. To create sample of the LogNormal data, you use mu and sigma, not m and sd.
double[] sample = new double[100];
LogNormal.Samples(sample, mu, sigma);
This is consistent with the Wikipedia article on the LogNormal distribution. The Numerics documentation is not clear.
Here is my test program which might be useful:
List<double> X = new List<double> { 10, 12, 18, 30 }; // assume to be LogNormal
double m = X.Mean(); // mean of log normal values = 17.5
double sd = X.StandardDeviation(); // standard deviation of log normal values = 9
List<double> Y = new List<double> { };
for (int i = 0; i < 4; i++)
{
Y.Add(Math.Log(X[i]));
}
// Y = {2.30,2.48,2.89,3.4}
double mu = Y.Mean(); // mean of normal values = 2.77
double sigma = Y.StandardDeviation(); // standard deviation of normal values = 0.487
double[] sample = new double[100];
LogNormal.Samples(sample, mu, sigma); // get sample
double sample_m = sample.Mean(); // 17.93, approximates m
double sample_sd = sample.StandardDeviation(); // 8.98, approximates sd
sample = new double[100];
Normal.Samples(sample, mu, sigma); // get sample
double sample_mu = sample.Mean(); //2.77, approximates mu
double sample_sigma = sample.StandardDeviation(); //0.517 approximates sigma
Using your test program above my samples came out like this.
Using LogNormal(mu, sigma)
I'm ultimately concerned about the values greater than 30 and less than 10.
However, by trail and error [accidentally], when I use the following method to get the samples using the original m and sd variable in your test program I get the results I'm looking for. I do not want to go forward with something I accidentally did.
sample = new double[100];
for (int i = 0; i < 100; i++)
{
sample[i] = LogNormal.WithMeanVariance(m, sd).Sample();
}
Using LogNormal.WithMeanVariance(m, sd)
My values are consistently between the Min and Max and concentrated around the Mean.
My example shows pretty clearly how to get a LogNormal sample that has the mean and standard deviation of the original data.
The min/max of 10/30 is unrealistic if you are going create your samples based on the mean and standard deviation of the sample. Suppose you took of random sample of the weights of 4 people out of a population of 1000 people. Would you expect your sample to include both the lightest and heaviest of the population?
LogNormal.WithMeanVariance(m, sd) is wrong. The units are wrong. It's expecting a variance would have the units of ln(days)^2 while sd has units of days.
I suggest you a) use LogNormal(mu,sigma) and discard any values that are outside your min/max range or b) use LogNormal(mu,c*sigma) for some value of c less than one to reduce the variance enough that all the values are in your min/max range. The choice depends on the nature of your project.
The Wikipedia entry on the LogNormal distribution has formulas for computing mu and sigma from m and sd which might be better than calculating from the Y data.

I wrote the code right? [duplicate]

This question already has answers here:
Find Cube root of a number Using System.Math.Pow() method in C#
(5 answers)
Closed 6 years ago.
According to the equation above this, I wrote here such code:
Double x = 16.55 * Math.Pow(10.0, -3);
Double y = -2.75;
Double z = 0.15;
Double kvadrat_koren_3 = Math.Pow(x, 1/3);
Double vozvedenie_v_stepen = Math.Pow(x, y + 2);
Double Summa_v_Skobkax = kvadrat_koren_3 + vozvedenie_v_stepen;
Double Kvadrat_koren_10 = Math.Sqrt(10.0 * Summa_v_Skobkax);
Double ArcSinus = Math.Pow(Math.Asin(z), 2) - Math.Abs(x - y);
Double Beta = Kvadrat_koren_10 * ArcSinus;
Console.WriteLine(Math.Round(Beta, 5));
But the calculation of the result issued: -41.31532, as needed: -40.63069.
Where am I wrong to write the expression?
P.S. I am using the latest version of SharpDevelop, and teach himself programming to change jobs.
Your problem is with the following: Math.Pow(x, 1/3); you're using integers for 1/3, which will give you an integer result of 0, rather than 0.33333etc.
Change the expression to Math.Pow(x, 1.0/3.0);

String to float C#, odd result [duplicate]

This question already has answers here:
C# String to Float Conversion
(5 answers)
Closed 7 years ago.
In my code I want to convert a string to a float. But when I conver something like 49.5 to a float, it gives the output 495 instead of 49.5 how can I solve this?
float.Parse, Single.Parse or Convert.ToSingle(); give all the same result...
String[] splittedLine = new String[25];
splittedLine = foundLine.Split('-');
float Z = float.Parse(splittedLine[2]);
float X = float.Parse(splittedLine[3]);
float Y = Single.Parse(splittedLine[4]);
PointF Center = new PointF(X /2, Y /2);
the values in the X & Y are or a full number (example 207 or 49.5);
foundLine is a line from a textdocument.
So how can I that the value from the text file (49.5) will stay 49.5 instead of 495?
Probably your system has a different format for fractional numbers than what your file was written with.
By default, float.Parse will use your system's locale settings to decide on this. To manually specify a format, you can use another overload:
float.Parse(splittedLine[2], CultureInfo.InvariantCulture);

Issue with .equals while comparing double in c# [duplicate]

This question already has answers here:
Comparing double values in C#
(18 answers)
Closed 8 years ago.
I have a float value that i parsed into double later i rounded off to 2.Also i have another float value to which i did exactly the same as first one.
Here is the sample code..
string pulse = arrvaluedline[2].ToString();
pCost = float.Parse(arrvaluedline[3]);
double d = System.Convert.ToDouble(spCost);
double dd = Math.Round(d,2);
string[] arrpulse = pulse.Split(':');
vodanoofPulse = float.Parse(arrpulse[0]);
calculatedCost = CallCost * Pulse;
double dcalcost = Math.Round(calculatedCost, 2);
Now here i am trying to compare
if (dcalcost.Equals(spCost)){
}
Although my both values dcalcost and spCost are 0.4 Except this ,flow is not going inside the if ..Why..Please help me .,
The Equals method should be used with caution, because two apparently equivalent values can be unequal due to the differing precision of the two values. The following example reports that the Double value .333333 and the Double value returned by dividing 1 by 3 are unequal.
// Initialize two doubles with apparently identical values
double double1 = .33333;
double double2 = 1/3;
// Compare them for equality
Console.WriteLine(double1.Equals(double2)); // displays false
Comparing doubles are not as easy as one might think. Here is an example from MSDN on how you can do it in a better way.
// Initialize two doubles with apparently identical values
double double1 = .333333;
double double2 = (double) 1/3;
// Define the tolerance for variation in their values
double difference = Math.Abs(double1 * .00001);
if (Math.Abs(double1 - double2) <= difference)
Console.WriteLine("double1 and double2 are equal.");
else
Console.WriteLine("double1 and double2 are unequal.");

Categories