We Keep Coding

Finding lookup values in datatable with match and with interpolation

I have a datatable that was imported from a CSV file. There are a multitude of different tables that can be imported.
The datatables have four columns (type=double): LookupColumn L M S
LookupColumn will usually have a unique name (e.g., Length, Height, Weight). The other column names remain the same. This is immaterial as you can just use dt.Column[0] mostly. The lookup column will always be the first column imported.
I need to search the datatable on LookupColumn for a LookupValue passed from the app (from a textbox).
If a LookupValue matches exactly a number in LookupColumn, then, return the values for L, M, S.
If there is no match, then I need to find the rows on either side of where the LookupValue would lie and return the min/max values for each variable in L,M,S.
Once I have those, I can interpolate the values for L, M, S.
For example:
Col_0
L
M
S
45.0
-0.3521
2.441
0.09182
45.5
-0.3521
2.524
0.09153
46.0
-0.3521
2.608
0.09124
46.5
-0.3521
2.691
0.09094
47.0
-0.3521
2.776
0.09065
47.5
-0.3521
2.861
0.09036
48.0
-0.3521
2.948
0.09007
If my LookupValue in Col[0] = 46.5, the program would return L=-0.3521 M=2.691 S=0.09094
These values will be put in textboxes on the form the viewer sees.
If there was no match (assuming LookupValue was within the range LookupColumn min/max) then I need to return the rows on both sides of where the value would lie if it were present--that is, Lmin Lmax, Mmin Mmax, Smin Smax and use those in the following formula to get the interpolated value (IntVal) for LookupColumn (Col_0).
For example, if the LookupValue in (Col_0) = 46.8, the returned results (array?, list?) would be the rows where Col_0 = 46.5 and 47.0:
Col_0
L Values
M Values
S Values
LookupMin = 46.5
Lmin = -0.3521
Mmin = 2.691
Smin = 0.09094
LookupMax = 47.0
Lmax = -0.3521
Mmax = 2.776
Smax = 0.09065
Interpolated Value = LMSmin + (46.8 - LookupMin) * (LMSmax - LMSmin / LookupMax - LookupMin)
Interpolated L = -0.3521 because Lmin = Lmax
Interpolated M = 2.691 + (46.8 - 46.5) * (2.776 - 2.691 / 47.0 - 46.5)
Interpolated M = 2.7418
Interpolated S = 0.09094 + (46.8 - 46.5) * (0.09065 - 0.09094 / 47.0 - 46.5)
Interpolated S = 0.09088
So, given the Min/Max values for Col_0 and either L, M, or S min/max values, I can interpolate any value the user provides that isn't in the lookup even if the LookupValue has more decimals. The interpolated L,M,S values will be put in textboxes for the user.
I have a bit of code that works when there's a match, but, I think there's a better/more concise way either using Linq or Tuples. I realize this isn't the best code and I'm open to suggestions.
I have scoured StackOverflow and found several posts on interpolation and Lookup Tables. It seems that the best practice for the lookup is to use a tuple, but, I'm not very clear on on their use.
For the most part, this question is focused on returning the Min/Max values of the lookup when there's no match. Once I have those, I don't think the interpolation is a big feat as I know the formula. Also, I know that the user could enter values out of range--I will account for those issues later.
Any help is appreciated.
private void tbLookup_Leave(object sender, System.EventArgs e)
{
string colName = tmpDT.Columns[0].ColumnName;
string colSearch = colName + " = '" + tbLookup.Text + "'";
if (tbLookup.Text.Length > 0)
{
// Exact match
while (true)
{
DataRow[] foundRow = tmpDT.Select(colSearch);
if (foundRow.Length == 0)
{
break;
}
foreach (DataRow row in foundRow)
{
string L = row.Field<string>("L");
string M = row.Field<string>("M");
string S = row.Field<string>("S");
tbLkupL.Text = L;
tbLkupM.Text = M;
tbLkupS.Text = S;
}
// No match
// Call interpolation method
}
}
else
{
MessageBox.Show("Please enter a lookup value", "Missing Data");
}

You inquired about possibly using LINQ, so I checked my code chest and found something similar, that I adapted to your needs.
using System.Linq; // Add this at the top of the Program.cs file.
The extension method returns three output parameters, that contain found indices, or -1 if not found.
// An extension methods class must be the first class in a file.
// Add this class inside the namespace of a console app, before the Program class (in the Program.cs file).
public static class ExtensionMethods
{
public static bool GetNearestOrEqual<TSource, TValue>(this System.Collections.Generic.IEnumerable<TSource> source, System.Func<TSource, TValue> valueSelector, TValue referenceValue, out int indexOfLowerMax, out int indexOfEqual, out int indexOfHigherMin)
where TValue : struct, System.IComparable<TValue>
{
using var e = source.GetEnumerator();
var ltCurrent = new TValue?();
var gtCurrent = new TValue?();
indexOfLowerMax = -1;
indexOfEqual = -1;
indexOfHigherMin = -1;
var index = 0;
while (e.MoveNext())
{
var currentValue = valueSelector(e.Current);
switch (currentValue.CompareTo(referenceValue))
{
case int lo when lo < 0:
if (!ltCurrent.HasValue || currentValue.CompareTo(ltCurrent.Value) > 0)
{
indexOfLowerMax = index;
ltCurrent = currentValue;
}
break;
case int hi when hi > 0:
if (!gtCurrent.HasValue || currentValue.CompareTo(gtCurrent.Value) < 0)
{
indexOfHigherMin = index;
gtCurrent = currentValue;
}
break;
default:
indexOfEqual = index;
break;
}
index++;
}
return indexOfLowerMax != -1 || indexOfEqual != -1 || indexOfHigherMin != -1;
}
}
Sample of how you could use it (created a simple console app):
// Replace the Main() inside the Program class of a console app.
static void Main(string[] args)
{
var dt = new System.Data.DataTable();
dt.Columns.Add("Col_0", typeof(double));
dt.Columns.Add("L", typeof(double));
dt.Columns.Add("M", typeof(double));
dt.Columns.Add("S", typeof(double));
dt.Rows.Add(new object[] { 45.0, -0.3521, 2.441, 0.09182 });
dt.Rows.Add(new object[] { 45.5, -0.3521, 2.524, 0.09153 });
dt.Rows.Add(new object[] { 46.0, -0.3521, 2.608, 0.09124 });
dt.Rows.Add(new object[] { 46.5, -0.3521, 2.691, 0.09094 });
dt.Rows.Add(new object[] { 47.0, -0.3521, 2.776, 0.09065 });
dt.Rows.Add(new object[] { 47.5, -0.3521, 2.861, 0.09036 });
dt.Rows.Add(new object[] { 48.0, -0.3521, 2.948, 0.09007 });
var lookupValue = 46.8;
var foundAnything = dt.Rows.Cast<System.Data.DataRow>().GetNearestOrEqual(o => (double)o.ItemArray[0], lookupValue, out var indexOfLowerMax, out var indexOfEqual, out var indexOfHigherMin);
// Assuming example for when both low and high are found...
var dr = dt.NewRow();
var lookuploDiff = lookupValue - (double)dt.Rows[indexOfLowerMax][0];
var hiloDiff = (double)dt.Rows[indexOfHigherMin][0] - (double)dt.Rows[indexOfLowerMax][0];
dr.ItemArray = new object[] {
lookupValue,
(double)dt.Rows[indexOfLowerMax][1] + lookuploDiff * (((double)dt.Rows[indexOfHigherMin][1] - (double)dt.Rows[indexOfLowerMax][1]) / hiloDiff),
(double)dt.Rows[indexOfLowerMax][2] + lookuploDiff * (((double)dt.Rows[indexOfHigherMin][2] - (double)dt.Rows[indexOfLowerMax][2]) / hiloDiff),
(double)dt.Rows[indexOfLowerMax][3] + lookuploDiff * (((double)dt.Rows[indexOfHigherMin][3] - (double)dt.Rows[indexOfLowerMax][3]) / hiloDiff),
};
dt.Rows.InsertAt(dr, indexOfHigherMin);
}
As always, if there are any questions, this is the place. :)

c# csv count a specified data in file or in datagridview

I have a csv file and would like to count the 2. column how many times contains 111.
the csv file has 46 separated columns with separator ; .
"first col" "second col" "....."
abc 111 a
abc 112 b
abc 113 c
abc 111 d
abc 112 e
abc 113 f
i would like to count the 111.
Filled up first the datagridview fom datatable.
dgv.DataSource = dgv_table;
string[] raw_text = File.ReadAllLines("d:\\"+lb_csv.Text);
string[] data_col = null;
int x = 0;
foreach (string text_line in raw_text)
{
// MessageBox.Show(text_line);
data_col = text_line.Split(';');
if (x == 0)
{
for (int i = 0; i <= data_col.Count() - 1; i++)
{
dgv_table.Columns.Add(data_col[i]);
}
//header
x++;
}
else
{
//data
dgv_table.Rows.Add(data_col);
}
I find a lots of solution to count the 2nd columns specified data:111
but all time i had problems.
int xCount = dgv.Rows.Cast<DataGridViewRow>().Select(row => row.Cells["second col"].Value).Where(s => s !=null && Equals(111)).Count();
this.lb_qty.Text = xCount.ToString();
But it gives error for row.Cells["second col"].Value
An unhandled exception of type 'System.ArgumentException' occurred in System.Windows.Forms.dll
Additional information: Column named second col cannot be found.
Can someone help me how to solve this problem and get the needed result?

I would suggest you to skip using DataGridView and use counter variable in your loop, like Arkadiusz suggested.
If you still want to work with DataTable, count values like this:
int xCount = dgv_table.Rows.Cast<DataRow>().Count(r => r["second col"] != null && r["second col"].ToString() == "111");

I would try to read the file into a DataTable and use it as DataSource for the DataGridView.
DataTable d_Table = new DataTable();
//fill the DataTable
this.dgv_table.DataSource = d_Table;
To count the rows wich contains 111 in the second column, you can select the DataTable like this:
DataTable d_Table = new DataTable();
//fill the DataTable
DataRow[] rowCount = d_Table.Select("secondCol = '111'");
this.lb_qty.Text = rowCount.Length.ToString();
Or you can do it in a foreach-loop:
int count = 0;
foreach(DataGridViewRow dgr in this.dgv_table.Rows)
{
if(dgr.Cells["secondCol"].Value.ToString() == "111") count++;
}
this.lb_qty.Text = count.ToString();

you can use this method to save the CSV into List of arrays List
public static List<string[]> readCSV(String filename)
{
List<string[]> result = new List<string[]>();
try
{
string[] line = File.ReadAllLines(filename);
foreach (string l in line)
{
string[] value= vrstica.Split(',');
result.Add(value);
}
}
catch (Exception e)
{
Console.WriteLine("Error: '{0}'", e);
}
return result;
}
every array will represent a column, so you can simply find the frequency of any value using LINQ or even loop:
foreach (var item in tmp[1].GroupBy(c => c))
{
Console.WriteLine("{0} : {1}", item.Key, item.Count());
}

int CountValues(string input, string searchedValue, int ColumnNumber, bool skipFirstLine = false)
{
int numberOfSearchedValue= 0;
string line;
using (StreamReader reader = new StreamReader (input))
{
if(skipFirstLine)
reader.ReadLine();
while ((line = reader.ReadLine()) != null)
{
if(line.Split(';')[ColumnNumber] == searchedValue)
numberOfSearchedValue++;
}
}
return numberOfSearchedValue;
}
Edit:
StreamReader.ReadLine() reads the line but also, using this method we are jumping to second line. If there is no more lines it returns null, so that is our ending condition. Rest of the code is readable, I think
:)
Didn't test that so be careful :)
It might be necessary to use Trim() or ToUpperCase() in some places (as usually when you are searching).

Parsing data from CSV file

Like in title I have problem with parsing data from CVS files. When i choose file with diffrent formating all i get is "Input string was not in a correct format".
My code works with files formatted like that:
16.990750 4.0
17.000250 5.0
17.009750 1.0
17.019250 6.0
But cant handle files formatted like this one:
Series1 - X;Series1 - Y;
285.75;798
285.79;764
285.84;578
285.88;690
This is code responsibile for reading data from file and creating chart from it:
if (openFileDialog1.ShowDialog() == DialogResult.OK)
{
string cos = File.ReadAllText(openFileDialog1.FileName);
string[] rows = cos.Split(new string[] { "\r\n" }, StringSplitOptions.RemoveEmptyEntries);
DataTable table = new DataTable();
table.Columns.Add("xValue", typeof(decimal));
table.Columns.Add("yValue", typeof(decimal));
foreach (string row in rows)
{
string[] values = row.Split(' ');
DataRow ch = table.NewRow();
ch[0] = Decimal.Parse(values[0], NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture);
ch[1] = Decimal.Parse(values[1], NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture);
table.Rows.Add(ch);
}
if (seria == false)
{
wykres.Series.Add("series");
wykres.Series["series"].ChartType = System.Windows.Forms.DataVisualization.Charting.SeriesChartType.Line;
wykres.Series["series"].XValueMember = "xValue";
wykres.Series["series"].YValueMembers = "yValue";
wykres.DataSource = table;
wykres.DataBind();
seria = true;
}
}
EDIT
I changed parsing method to this one:
foreach (string row in rows)
{
var values = row.Split(';');
var ch = table.NewRow();
decimal num = 0;
if (decimal.TryParse(values[0], out num))
ch[0] = num;
if (decimal.TryParse(values[1], out num))
ch[1] = num;
table.Rows.Add(ch);
}
It works okay but with one exception - It can't read decimals only integers from csv file(see the picture below).
View of table in locals
Why is this happening?

I suggest you don't re-invent the wheel, but use some well-tested library to parse the CSV (for example, your implementation does not handle quoted values well. It also doesn't allow the separator as part of a value).
And guess what: .NET includes something that could help you: the TextFieldParser class. Don't worry about the VisualBasicnamespace - it works in C#, too :-)

foreach (string row in rows)
{
var values = row.Split(';');
var ch = table.NewRow();
decimal num = 0;
if (decimal.TryParse(values[0], out num))
ch[0] = num;
if (decimal.TryParse(values[1], out num))
ch[1] = num;
table.Rows.Add(ch);
}
In the second text format the separator is(;) and first row of the text has two strings therefore to convert a string to decimal use decimal.TryParse() instead of decimal.Parse().the return type of method TryParse() is boolean
therefore if it returned true that means the string converted successful .

Casting a String as Integer in a .net DataTable

Disclaimer: This is my very first .net c# project
I am attempting to import a CSV into MSSQL but need to iterate through the CSV values first for sanitization purposes. Some of the columns in the CSV will be integer (will be used for calcuations later) and some are regular varchar.
My script above appears to force all values (that is row column values) in the DataTable as a string which throws an Exception later in my application when SQL cannot write a string as an integer.
Here is my method I am using for the getCSVImport which creates a datatable and populates it.
What I am thinking is to add another condition which checks if the value is an integer and then cast it as an integer (this kind of thing is new to me as PHP would does not handle types so strongly) but I fear that wont work as I am not sure if I can mix the values within a dataTable with various types.
So my question is, is there a way for me to have different values in a datatable as different types? My code below takes the line as a whole and writes it as a string, I need the values to be assigned either as string or as integer.
/*
* getCsvData()
* This method will create a datatable from the CSV file. We'll take the CSV file as is.
* and collect the data as needed:
*
* - Remove those original 4 lines (worthless info)
* - Line 5 starts with the headers, remove any of the brackets around the values
* - Iterate through the rest of the fields and sanitize them before we add it to the datatable
*
*/
private DataTable getCsvData(string csv_file_path)
{
// Create a new csvData tabletable object:
DataTable csvData = new DataTable();
try
{
using (TextFieldParser csvReader = new TextFieldParser(csv_file_path))
{
csvReader.SetDelimiters(new string[] { "," });
csvReader.HasFieldsEnclosedInQuotes = true;
int row = 1;
while (!csvReader.EndOfData)
{
// Read the string and collect the row data
string[] rowData = csvReader.ReadFields();
if (row <= 4)
{
// We want to start on row 5 as first rows are nonsense :)
// Incriment the row so that we can do our magic above
row++;
continue;
} if(row == 5)
{
// Row 5 is the headers, we need to sanitize and continue:
foreach (string column in rowData)
{
// Remove the [ ] from the values:
var col = column.Substring(1, column.Length - 2);
DataColumn datecolumn = new DataColumn(col);
datecolumn.AllowDBNull = true;
csvData.Columns.Add(datecolumn);
}
// Incriment the row so that we can do our magic above
row++;
} else
{
// These are all of the actual rows, sanitize and add the rows:
//Making empty value as null
for (int i = 0; i < rowData.Length; i++)
{
// First remove the brackets:
if (rowData[i].Substring(0,1) == "[")
{
rowData[i] = rowData[i].Substring(1, rowData[i].Length - 2);
}
// Set blank to null:
if (rowData[i] == "" || rowData[i] == "-")
{
rowData[i] = null;
}
// Lastly, we need to do some calculations:
}
// Add the sanitized row to the DataTable:
csvData.Rows.Add(rowData);
}
}
}
}
catch (Exception ex)
{
throw new Exception("Could not parse the CSV file: "+ ex.Message);
}
return csvData;
}

You can cast the string to a int:
int j;
bool parsed=Int32.TryParse("-105", out j))
With TryParse you can check if it succeeded.
Then when you want to save it to the table again, cast it to string. You can simply do <variable>.ToString()

By default, data columns are initialized to a string data type.
There's an overload that allows you to specify the type, so I'd suggest you try that. Since your columns are known beforehand, you can easily handle this in your code.
private DataColumn AddColumn(string columnName, Type columnType)
{
// Remove the [ ] from the values:
var col = column.Substring(1, columnName.Length - 2);
DataColumn dataColumn = new DataColumn(col, columnType);
dataColumn.AllowDBNull = true;
return dataColumn;
}
if (row == 5)
{
csvData.Columns.Add(AddColumn(rowData[0], typeof(string)));
csvData.Columns.Add(AddColumn(rowData[1], typeof(int)));
csvData.Columns.Add(AddColumn(rowData[2], typeof(DateTime)));
csvData.Columns.Add(AddColumn(rowData[3], typeof(string)));
// etc
}
I'm not sure you'll even need to convert the other values before adding them to the DataTable, but if you do, many built-in types have TryParse methods, such as DateTime.TryParse and Int32.TryParse. You can call each of them in succession, and one of the "tries" succeeds, you'll know your type.
Alternatively, since you know the column types beforehand, you can just cast each value.
csvData.Rows.Add(Convert.ToString(rowData[0]),
Convert.ToInt32(rowData[1]),
Convert.ToDateTime(rowData[2]),
Convert.ToString(rowData[3]));

I would use *.TryParse(), ie: With this sample CSV:
*A sample csv file with
*some comment lines at top
-- with different comment
// comment strings.
[charField],[dateField],[intField],[decimalField]
"Sample char data 1",2016/1/2,123,123.45
"Sample char data 2",,2,1.5
"Sample char data 3",,3,
"Sample char data 4",,,
,,,
"Sample char data 6",2016/2/29 10:20,10,20.5
You might use TryParse on those datetime, int, decimal fields:
void Main()
{
var myData = ReadMyCSV(#"c:\MyPath\MyFile.csv");
// do whatever with myData
}
public IEnumerable<MyRow> ReadMyCSV(string fileName)
{
using (TextFieldParser tfp = new TextFieldParser(fileName))
{
tfp.HasFieldsEnclosedInQuotes = true;
tfp.SetDelimiters(new string[] { "," });
//tfp.CommentTokens = new string[] { "*","--","//" };
// instead of using comment tokens we are going to skip 4 lines
for (int j = 0; j < 4; j++)
{
tfp.ReadLine();
}
// header line.
tfp.ReadLine();
DateTime dt;
int i;
decimal d;
while (!tfp.EndOfData)
{
var data = tfp.ReadFields();
yield return new MyRow
{
MyCharData = data[0],
MyDateTime = DateTime.TryParse(data[1], out dt) ? dt : (DateTime?)null,
MyIntData = int.TryParse(data[2], out i) ? i : 0,
MyDecimal = decimal.TryParse(data[3], System.Globalization.NumberStyles.Any, null, out d) ? d : 0M
};
}
}
}
public class MyRow
{
public string MyCharData { get; set; }
public int MyIntData { get; set; }
public DateTime? MyDateTime { get; set; }
public decimal MyDecimal { get; set; }
}
I could further sanitize the data loaded, such as:
myData.Where( d => d.MyIntData != 0 );
Note: I didn't use a DataTable, which I could if I wanted to. For MSSQL loading, I would probably use an intermediate in-memory SQLite instance to save the sanitized data and then push to MSSQL using SqlBulkCopy class. A DataTable is of course an option (I just think it is less flexible).

formatting string builder in a For loop

I have a function for my webform which outputs each item within my cart using a for loop. However, my problem is to format this string so it includes tabs spaces, and a new line for each item in the cart. My code stands as this:
public string Display()
{
CartClass CartList = CartClass.GetCart();
System.Text.StringBuilder sb = new StringBuilder();
for (int i = 0; i < CartList.CartList.Count(); i++)
{
Movies Movie = CartList.CartList[i];
sb.Append(String.Format(i + 1 + "." + "\t"
+ Movie.MovieName + "\t" + "£" + Movie.MovieCost.ToString() + "\n" ));
}
return sb.ToString();
}
also to note that I output this string within a asp:listbox using this function
private void DisplayCart()
{
lstCart.Items.Clear();
lstCart.Items.Add(cart.Display());
}
and my out come is this
I'd like the format to resemble a list. For example
Up £5
Madagascar £5
Finding Nemo £5
how can I solve this?

It is not clear what you want in your line to be displayed, but assuming that you want a progressive number, the MovieName and its cost formatted with the current currency symbol, you could use directly the AppendFormat method and the rules of composite formatting
for (int i = 0; i < CartList.CartList.Count(); i++)
{
Movies Movie = CartList.CartList[i];
sb.AppendFormat("{0}.\t{1}\t{2:C}\n", i+1, Movie.MovieName, Movie.MovieCost);
}
The key point here is that both string.Format and StringBuilder.AppendFormat require a format string with placeholders embedded in curly braces ({0}....{1}) where the arguments following the format specifier will be inserted.
See Composite Formatting
However, your problem is caused by adding the whole stringbuilder as one single item. The newline character doesn't break you string in two, it is simply ignored by the listbox items collection.
You need to add one item at time. ( or look at the answer of Mr. Carey)
private void DisplayCart()
{
lstCart.Items.Clear();
CartClass CartList = CartClass.GetCart();
for (int i = 0; i < CartList.CartList.Count(); i++)
{
Movies Movie = CartList.CartList[i];
lstCart.Items.Add(string.Format("{0}.\t{1}\t{2:C}",
i+1, Movie.MovieName, Movie.MovieCost);
}
}

Assuming you want to segregate each movie from the cart list and add each movie as an item to the ListBox, you can use yield return from within the for loop to output an enumerable instead of a composite string and then you can enumerate the list to add each item to the ListBox.
You can do something like:
public IEnumerable<string> Display()
{
CartClass CartList = CartClass.GetCart();
for (int i = 0; i < CartList.CartList.Count(); i++)
{
Movies Movie = CartList.CartList[i];
yield return String.Format(i + 1 + "." + "\t"
+ Movie.MovieName + "\t" + "£" + Movie.MovieCost.ToString() + "\n" );
}
}
And then in the DisplayCart function:
private void DisplayCart()
{
lstCart.Items.Clear();
foreach (var movie in cart.Display())
{
lstCart.Items.Add(movie);
}
}

Assuming you'r using an ASP.Net ListBox control...
Well, any sequence of 1 or more whitespace characters save is, per the HTML spec, collapsed to a single SP (0x20) character when displayed on an HTML page. So there' not much point in differentiating between HT (tab) and SP.
What you probably want to do is to use data binding to populate your listbox. In which case your code becomes something like this:
CartClass cart = CartClass.GetCart() ;
IEnumerable<Movie> movies = cart.CartList ;
lstCart.DataSource = movies;
lstCart.DataTextField = <property-to-be-displayed-to-user>
lstCart.DataValueField = <property-to-be-used-when-item-is-selected>
lstCart.DataBind();
If you really want to format a string, you might look at doing something like this:
public string Display()
{
CartClass cart = CartClass.GetCart();
StringBuilder sb = new StringBuilder();
for ( int i = 0 ; i < cart.CartList.Count() ; i++ )
{
Movie movie = cart.CartList[i];
sb.AppendFormat("{0}.\t{1}\t{2:C}" , i+1 , movie.MovieName , movie.MovieCost ).AppendLine() ;
}
return sb.ToString();
}
The C format specifier is "currency". Assuming you're in the UK, your default CultureInfo should format the movie cost as a proper UK price/currency value.