I want a 2D array generated from a CSV file with unknown number of rows/columns. The column count is fixed based on the header data. I need to be able to process it as a grid going both across rows and down columns hence needing array.
At the moment, I can split the data into rows, then split each row into components. I then add each row to a list. This all seems to work fine.
What I cant do is convert a list of string arrays into a 2d array.
It currently is failing on the line string[,] newCSV = csvFile.ToArray(); with error Cannot implicitly convert type 'string[][]' to 'string[ * , * ]' so I'm obviously not declaring something properly - I've just no idea what!
List<string[]> csvFile = new List<string[]>();
void Start()
{
// TODO: file picker
TextAsset sourceData = Resources.Load<TextAsset>("CSVData");
if (sourceData != null)
{
// Each piece of data in a CSV has a newline at the end
// Split the base data into an array each time the newline char is found
string[] data = sourceData.text.Split(new char[] {'\n'} );
for (int i = 0; i < data.Length; i ++)
{
string[] row = data[i].Split(new char[] {','} );
Debug.Log(row[0] + " " + row[1]);
csvFile.Add(row);
}
string[,] newCSV = csvFile.ToArray();
} else {
Debug.Log("Can't open source file");
}
Since your data is in the form of a table, I highly suggest using a DataTable instead of a 2d array like you're currently using to model/hold the data from your csv.
There's a ton of pre baked functionality that comes with this data structure that will make working with your data much easier.
If you take this route, you could then also use this which will copy CSV data into a DataTable using the structure of your CSV data to create the DataTable.
It's very easy to configure and use.
Just a small tip, you should always try to use data structures that best fit your task, whenever possible. Think of the data structures and algorithms you use as tools used to build a house, while you could certainly use a screw driver to pound in a nail, it's much easier and more efficient to use a hammer.
You can use this function to get 2d array.
static public string[,] SplitCsvGrid(string csvText)
{
string[] lines = csvText.Split("\n"[0]);
// finds the max width of row
int width = 0;
for (int i = 0; i < lines.Length; i++)
{
string[] row = SplitCsvLine(lines[i]);
width = Mathf.Max(width, row.Length);
}
// creates new 2D string grid to output to
string[,] outputGrid = new string[width + 1, lines.Length + 1];
for (int y = 0; y < lines.Length; y++)
{
string[] row = SplitCsvLine(lines[y]);
for (int x = 0; x < row.Length; x++)
{
outputGrid[x, y] = row[x];
// This line was to replace "" with " in my output.
// Include or edit it as you wish.
outputGrid[x, y] = outputGrid[x, y].Replace("\"\"", "\"");
}
}
return outputGrid;
}
Related
I am planning to get an array of the averages of each column.
But my app crashes at sum[j] += int.Parse(csvArray[i,j]); due to a FormatException. I have tried using Convert.ToDouble and Double.Parse but it still throws that exception.
The increments in the for loop start at 1 because Row 0 and Column 0 of the CSV array are strings (names and timestamps). For the divisor or total count of the fields that have values per column, I only count the fields that are not BLANK, hence the IF statement. I think I need help at handling the exception.
Below is the my existing for the method of getting the averages.
public void getColumnAverages(string filePath)
{
int col = colCount(filePath);
int row = rowCount(filePath);
string[,] csvArray = csvToArray(filePath);
int[] count = new int[col];
int[] sum = new int[col];
double[] average = new double[col];
for (int i = 1; i < row; i++)
{
for (int j = 1; j < col; j++)
{
if (csvArray[i,j] != " ")
{
sum[j] += int.Parse(csvArray[i,j]);
count[j]++;
}
}
}
for (int i = 0; i < average.Length; i++)
{
average[i] = (sum[i] + 0.0) / count[i];
}
foreach(double d in average)
{
System.Diagnostics.Debug.Write(d);
}
}
}
I have uploaded the CSV file that I use when I test the prototype. It has BLANK values on some columns. Was my existing IF statement unable to handle that case?
There are also entries like this 1.324556-e09due to the number of decimals I think. I guess I have to trim it in the csvToArray(filePath) method or are there other efficient ways? Thanks a million!
So there are a few problems with your code. The main reason for your format exception is that after looking at your CSV file your numbers are surrounded by quotes. Now I can't see from your code exactly how you convert your CSV file to an array but I'm guessing that you don't clear these out - I didn't when I first ran with your CSV and experienced the exact same error.
I then ran into an error because some of the values in your CSV are decimal, so the datatype int can't be used. I'm assuming that you still want the averages of these columns so in my slightly revised verion of your method I change the arrays used to be of type double.
AS #musefan suggested, I have also changed the check for empty places to use the IsNullOrWhiteSpace method.
Finally when you output your results you receive a NaN for the first value in the averages column, this is because when you don't take into account that you never populate the first position of your arrays so as not to process the string values. I'm unsure how you'd best like to correct this behaviour as I'm not sure of the intended purpose - this might be okay - so I've not made any changes to this for the moment, pop a mention in the comments if you want help on how to sort this!
So here is the updated method:
public static void getColumnAverages(string filePath)
{
// Differs from the current implementation, reads a file in as text and
// splits by a defined delim into an array
var filePaths = #"C:\test.csv";
var csvArray = File.ReadLines(filePaths).Select(x => x.Split(',')).ToArray();
// Differs from the current implementation
var col = csvArray[0].Length;
var row = csvArray.Length;
// Update variables to use doubles
double[] count = new double[col];
double[] sum = new double[col];
double[] average = new double[col];
Console.WriteLine("Started");
for (int i = 1; i < row; i++)
{
for (int j = 1; j < col; j++)
{
// Remove the quotes from your array
var current = csvArray[i][j].Replace("\"", "");
// Added the Method IsNullOrWhiteSpace
if (!string.IsNullOrWhiteSpace(current))
{
// Parse as double not int to account for dec. values
sum[j] += double.Parse(current);
count[j]++;
}
}
}
for (int i = 0; i < average.Length; i++)
{
average[i] = (sum[i] + 0.0) / count[i];
}
foreach (double d in average)
{
System.Diagnostics.Debug.Write(d + "\n");
}
}
I have a C# console window program and I am trying to sort "File3" (contains numbers) in ascending and output lines from 3 text files.
So the outcome looks something like this:
===========================================================================
field1.....................field2.....................field3
===========================================================================
[FILE1_LINE1]..............[FILE2_LINE1]..............[FILE3_LINE1]
[FILE1_LINE2]..............[FILE2_LINE2]..............[FILE3_LINE2]
[FILE1_LINE3]..............[FILE2_LINE3]..............[FILE3_LINE3]
and so on...
At the moment, it kinda works I think but it duplicates the first two lines it seems. Could someone give an example of better coding please?
Here is the code that I have atm:
string[] File1 = System.IO.File.ReadAllLines(#"FILE1.txt");
string[] File2 = System.IO.File.ReadAllLines(#"FILE2.txt");
string[] File3 = System.IO.File.ReadAllLines(#"FILE3.txt");
decimal[] File3_1 = new decimal[File3.Length];
for(int i=0; i<File3.Length; i++)
{
File3_1[i] = decimal.Parse(File3[i]);
}
decimal[] File3_2 = new decimal[File3.Length];
for(int i=0; i<File3.Length; i++)
{
File3_2[i] = decimal.Parse(File3[i]);
}
decimal number = 0;
for (double i = 0.00; i < File3_1.Length; i++)
{
for (int sort = 0; sort < File3_1.Length - 1; sort++)
{
if (File3_1[sort] > File3_1[sort + 1])
{
number = File3_1[sort + 1];
File3_1[sort + 1] = File3_1[sort];
File3_1[sort] = number;
}
}
}
if (SortChoice2 == 1)
{
for (int y = 0; y < File3_2.Length; y++)
{
for (int s = 0; s < File3_2.Length; s++)
{
if (File3_1[y] == File3_2[s])
{
Console.WriteLine(File1[s] + File2[s] + File3_1[y]);
}
}
}
}
Just for more info, most of this code was used for another program and worked but in my new program, this doesn't as I've said above - ("it repeats a couple of lines for some reason"). I'm kinda an amateur/ rookie at C# so I only get stuff like this to work with examples.
Thanks in advance :)
Ok, if I understand correctly, what you are trying to do is read the lines from 3 different files, each of them representing a different "field" in a table. You then want to sort this table based on the value of one of the field (in you code, this seems to be the field which values are contained in File3. Well, if I got that right, here's what I suggest you do:
// Read data from files
List<string> inputFileNames = new List<string> {"File1.txt", "File2.txt", "File3.txt"};
decimal[][] fieldValues = new decimal[inputFileNames.Count][];
for (int i = 0; i < inputFileNames.Count; i++)
{
string currentInputfileName = inputFileNames[i];
string[] currentInputFileLines = File.ReadAllLines(currentInputfileName);
fieldValues[i] = new decimal[currentInputFileLines.Length];
for (int j = 0; j < currentInputFileLines.Length; j++)
{
fieldValues[i][j] = decimal.Parse(currentInputFileLines[j]);
}
}
// Create table
DataTable table = new DataTable();
DataColumn field1Column = table.Columns.Add("field1", typeof (decimal));
DataColumn field2Column = table.Columns.Add("field2", typeof (decimal));
DataColumn field3Column = table.Columns.Add("field3", typeof (decimal));
for (int i = 0; i < fieldValues[0].Length; i++)
{
var newTableRow = table.NewRow();
newTableRow[field1Column.ColumnName] = fieldValues[0][i];
newTableRow[field2Column.ColumnName] = fieldValues[1][i];
newTableRow[field3Column.ColumnName] = fieldValues[2][i];
table.Rows.Add(newTableRow);
}
// Sorting
table.DefaultView.Sort = field1Column.ColumnName;
// Output
foreach (DataRow row in table.DefaultView.ToTable().Rows)
{
foreach (var item in row.ItemArray)
{
Console.Write(item + " ");
}
Console.WriteLine();
}
Now, I tried to keep the code above as LINQ free as I could, since you do not seem to be using it in your example, and therefore might not know about it. That being said, while there is a thousand way to do I/O in C#, LINQ would help you a lot in this instance (and in pretty much any other situation really), so I suggest you look it up if you don't know about it already.
Also, the DataTable option I proposed is just to provide a way for you to visualize and organize the data in a more efficient way. That being said, you are in no way obliged to use a DataTable: you could stay with a more direct approach and use more common data structures (such as lists, arrays or even dictionaries if you know what they are) to store the data, depending on your needs. It's just that with a DataTable, you don't, for example, need to do the sorting yourself, or deal with columns indexed only by integers. With time, you'll come to learn about the myriad of useful data structure and native functionalities the C# language offers you and how they can save you doing the work yourself in a lot of cases.
I have a big problem with writing some data to a csv file. I have a lot of measurement values. Every value is described by name, unit, value. So i want to build for every value a column with these three properties.
I want to store it into the csv file like this:
Width Cell Wall Thickness Coarseness Curl-Index etc.
mm mm mg/m % etc.
16,2 3,2 0,000 11,7 etc.
Till now i was coding a header for the names, another for the units and the values (that were previously stored into a string array) i just wrote in one line.
Till now my csv file looks like this:(
Width;Cell Wall Thickness;Coarseness;Curl-Index;etc.
mm;mm;mg/m;%;etc.
16,2;3,2;0,000;11,7;etc.
if it were not many values i wouldn't care about this but there are a lot so when i open the csv file there's the problem that the headers dont fit to the values and units. It's not organized, i cannot match the values to the headers.
I would like everything to be organized in columns. Any help would be strongly appreciated!
That's the code that i have till now:
StreamWriter sw = new StreamWriter("test2.csv");
int RowCount = 3;
int ColumnCount = 4;
string[][] Transfer_2D = new string[RowCount][];
Transfer_2D[0] = new string[3]{"Width", "CWT", "Coarseness", "Curl-Index"};//name of the values
Transfer_2D[1] = new string[3] {"mm", "mm", "mg/m", "%"}; //units
Transfer_2D[2] = new string[3] { TransferData[0], TransferData[1], TransferData[2], TransferData[3] };
for (int i = 0; i < RowCount; i++)
{
for (int j = 0; j < ColumnCount; j++)
{
sw.Write(Transfer_2D[i][j]);//write one row separated by columns
if (j < ColumnCount)
{
sw.Write(";");//use ; as separator between columns
}
}
if (i < RowCount)
{
sw.Write("\n");//use \n to separate between rows
}
}
sw.Close();
}
you can set the string to a fixed length.
example look here: (.NET Format a string with fixed spaces)
int iWantedStringLength = 20;
string sFormatInstruction = "{0,-" + iWantedStringLength.ToString() + "}";
for (int i = 0; i < RowCount; i++)
{
for (int j = 0; j < ColumnCount; j++)
{
sw.Write(String.Format(sFormatInstruction, Transfer_2D[i][j]));//write one row separated by columns
if (j < ColumnCount)
{
sw.Write(";");//use ; as separator between columns
}
}
if (i < RowCount)
{
sw.Write("\n");//use \n to separate between rows
}
}
For CSV work I use http://joshclose.github.io/CsvHelper/, this is a very nice helper class but it would require you to change your working a little bit.
I would advice you create a class to store each entry in and then create a "Mapper" to map it to csv fields. Then you can just pass a collection of objects and your mapping class to the helper and it produces a structured CSV.
There are alot of examples on that page so it should be straight forward for you to work through.
I am new in C# and trying to build a sort of calculator application that will read values from a csv file and then carry on with the manipulation that's needed.
the calculator algorithm is working fine but I need to populate the arrays from a csv file. I have used StreamReader as follows but it puts all values into a single array:
FileStream aFile = new FileStream("d:\test.txt", FileMode.Open, FileAccess.Read);
StreamReader sr = new StreamReader(aFile);
strLine = sr.ReadLine();
while (strLine != null)
{
strArray = strLine.Split(charArray);
for (int x = 1; x <= strArray.GetUpperBound(0); x++)
{
Console.WriteLine(strArray[x].Trim());
I need to read seperate lines into separate arrays and these lines can be dynamic (i.e. csv files can have n number of lines which will require n arrays). The calculator algorithm is below and is working fine. I only need how to use the above code to read csv into arrays and use those arrays in the code below (code below, though an excerpt, is working fine). Any help please.
for (int p = 0; p < n; p++)
{
for (int j = 0; j < t; j++)
{
sum[j] = w[p, j] * a[p, j];
k[p] = k[p] + sum[j];
}
}
double loa=0;
for (int i = 0; i < n; i++)
{
loa1[i] = k[i] * v[i];
loa += loa1[i];
The TextFieldParser class is great for reading CSV files.
TextFieldParser parser = new TextFieldParser("filename.txt");
parser.Delimiters = new[] { "," };
while (!parser.EndOfData)
{
string[] fields = parser.ReadFields();
//process this line of fields, or store it for later processing
}
You can then use int.TryParse or double.TryParse to turn the appropriate strings into numeric values.
Many thanks all for your comments and help -very much appreciated. I was able to read lines separately using:
using (TextFieldParser fieldParser = new TextFieldParser(name)) "d:\\test1.txt"))
{
parts = fieldParser.ReadFields();
and then hold the values from separate lines in an double array using:
double[] array = Array.ConvertAll(parts, double.Parse);
and then I used a counter and EndOfData property to manage line switches that define separate arrays:
counter++;
if (fieldParser.EndOfData)
{
Thanks again. may calculator is now working and is available at http://thaddeus-eze.com
I have a list of numbers on a textbox like so (the numbers used are just examples):
1 1 1
2 2 2
...
So I want to convert that into a 2d array. I know to use .ToArray() or Regex.Split() for 1d lists but am not sure how to use that for 2d. I've also tried to use those functions on a string[] array to make it 2d but there was an error.
Also, the array is supposed to be an int[,] so that the values in the array can be compared. Any help would be appreciated, thanks!
Here you go, if you don't understand any part please ask in the comments:
// assuming the numbers are in perfect 2D format in textBox (only 1 newline separates the lines, only 1 space separates numbers in each line and all lines have the same amount of numbers)
string textWithNumbers = textBox.Text;
// first put all lines into an string array
string[] allLines = textWithNumbers.Split(new string[]{Environment.NewLine}, StringSplitOptions.RemoveEmptyEntries);
// calculate 2D array's dimension lengths, and initialize the 2Darray
int rowCount = allLines.Length;
int columnCount = ((allLines[0].Length + 1) / 2);
int[,] twoDArray = new int[rowCount, columnCount];
// we then iterate through the 2D array
for (int row = 0; row < rowCount; row++)
{
// parse each number from string format to integer format & assign it to the corresponding location in our 2D array
string[] line = allLines[row].Split(' ');
for (int column = 0; column < columnCount; column++)
{
twoDArray[row, column] = int.Parse(line[column]);
}
}
This will give you a nice jagged 2D array that doesn't depend on all the text boxes having the same length. If you need them to all be the same length, it's trivial to check.
string[] data = // text input from all the text boxes
var result = data.Select(x => x.Split(' ')
.Select(y => int.Parse(y)).ToArray())
.ToArray();
Result is not quite an int[,] but an int[int[]], which is practically the same thing.
Of course, you need to deal with input validation or error handling.
Let me first start with the most naive solution, this makes very few assumptions about the data entered by the user. For example, it does not assume that every row has the same number of entries etc. So this could be optimized for those special conditions that you might know hold true or can enforce before running this routine.
// This is the data from the textbox
// hardcoded here for demonstration
string data = "1 1 1" + Environment.NewLine
+ "2 2 2" + Environment.NewLine
+ "12 12 12";
// First we need to determine the size of array dimension
// How many rows and columns do we need
int columnCount;
int rowCount;
// We get the rows by splitting on the new lines
string[] rows = data.Split(new string[]{Environment.NewLine},
StringSplitOptions.RemoveEmptyEntries);
rowCount = rows.Length;
// We iterate through each row to find the max number of items
columnCount = 0;
foreach (string row in rows)
{
int length = row.Split(' ').Length;
if (length > columnCount) columnCount = length;
}
// Allocate our 2D array
int[,] myArray = new int[rowCount, columnCount];
// Populate the array with the data
for (int i = 0; i < rowCount; ++i)
{
// Get each row of data and split the string into the
// separate components
string[] rowData = rows[i].Split(' ');
for (int j = 0; j < rowData.length; ++j)
{
// Convert each component to an integer value and
// enter it into the 2D array
int value;
if (int.TryParse(rowData[j], out value))
{
myArray[i, j] = value;
}
}
}
Given the above where we are considering the possibility of each row not having the same number of elements, you might consider using a sparse array int[][] which coincidentally also yield better performance in .NET.