My csv file contains 8 columns and 300k rows.Here is an example of my csv file
"0195153448";"Classical Mythology";"Mark P. O. Morford";"2002";"Oxford University Press";"http://images.amazon.com/images/P/0195153448.01.THUMBZZZ.jpg";"http://images.amazon.com/images/P/0195153448.01.MZZZZZZZ.jpg";"http://images.amazon.com/images/P/0195153448.01.LZZZZZZZ.jpg"
"0002005018";"Clara Callan";"Richard Bruce Wright";"2001";"HarperFlamingo Canada";"http://images.amazon.com/images/P/0002005018.01.THUMBZZZ.jpg";"http://images.amazon.com/images/P/0002005018.01.MZZZZZZZ.jpg";"http://images.amazon.com/images/P/0002005018.01.LZZZZZZZ.jpg"
Now i have this code for reading
string path = #"C:\Users\SEMRUK\Desktop\exceller\kitaplik.csv";
public DataTable GetDataTabletFromCSVFile(string PathFile)
{
DataTable csvData = new DataTable();
TextFieldParser csvReader = new TextFieldParser(PathFile);
csvReader.SetDelimiters(new string[] {";"});
csvReader.HasFieldsEnclosedInQuotes = true;
string[] colFields = csvReader.ReadFields();
foreach (string column in colFields)
{
DataColumn datecolumn = new DataColumn(column);
datecolumn.AllowDBNull = true;
csvData.Columns.Add(datecolumn);
}
string[] fieldData = csvReader.ReadFields();
while (!csvReader.EndOfData)
{
for (int i = 0; i < fieldData.Length; i++)
{
if (fieldData[i] == "")
{
fieldData[i] = null;
}
MessageBox.Show(fieldData[i]);
}
csvData.Rows.Add(fieldData);
}
return csvData;
}
This way i can read the first 8 rows.But i can't read anything after that.It just reads first 8.How can i tell the code that it should read after tab too.I tried adding \t to the setdelimiters but didn't work.Any suggestions?
static void Main(string[] args)
{
string path = "your_file_path";
string text = System.IO.File.ReadAllText(path);
string[] parsedText= text.Split(';');
foreach (var item in parsedText)
{
//do some
}
}
Related
The data is not displayed correctly in the columns.
The CSV consists of 7 columns. Rows are of different length.
I can not upload a picture.(https://ibb.co/0fnfLW7)
DataTable tblcsv = new DataTable();
tblcsv.Columns.Add("Vorname");
tblcsv.Columns.Add("Nachname");
tblcsv.Columns.Add("RFID");
string csvData = File.ReadAllText(csvPath);
//spliting row after new line
foreach (string csvRow in csvData.Split(';'))
{
if (!string.IsNullOrEmpty(csvRow))
{
//Adding each row into datatable
tblcsv.Rows.Add();
int count = 0;
foreach (string FileRec in csvRow.Split(';'))
{
tblcsv.Rows[tblcsv.Rows.Count - 1][count] = FileRec;
count++;
for(var x=0; x<7; x++)
{
//tblcsv[x][count] = FileRec;
}
count++;
}
}
//Calling Bind Grid Functions
BindgridStaffImport(tblcsv);
}
You split code is using ";", and should it not be "," if a csv?
The code will look much like this:
foreach (string csvRow in csvData.Split(';'))
{
if (!string.IsNullOrEmpty(csvRow))
{
//Adding each row into datatable
DataRow MyNewRow = tblcsv.NewRow();
int count = 0;
foreach (string FileRec in csvRow.Split(','))
{
MyNewRow[count] = FileRec;
count++;
}
tblcsv.Rows.Add(MyNewRow);
}
//Calling Bind Grid Functions
BindgridStaffImport(tblcsv);
So, use the "newRow" method to create a new blank row based on the table. Set the column values, and then add this new row to the rows collection of the table.
And even better is to use the TextFieldParser class.
So,
using Microsoft.VisualBasic.FileIO;
And then you can even assume say the first row has the colum names.
This:
We assume you read the csv file into stirng sFileLocal
// process csv file
TextFieldParser MyParse = new TextFieldParser(sFileLocal);
MyParse.TextFieldType = FieldType.Delimited;
MyParse.SetDelimiters(new string[] { "," });
MyParse.HasFieldsEnclosedInQuotes = true;
// add colums to table.
DataTable rstData = new DataTable();
foreach (string cCol in MyParse.ReadFields())
{
rstData.Columns.Add(cCol);
}
while (!MyParse.EndOfData)
{
string[] OneRow = MyParse.ReadFields();
rstData.Rows.Add(OneRow);
}
// Now display results in a grid
GridView1.DataSource = rstData;
GridView1.DataBind();
Quite much all above is you need.
Thank you very much for the support.
I now have a working variant.
However, the goal is to skip the first line, the header.
Are there any ideas how I can implement this.
List<string> headercolumns = new List<string> { "Vorname", "Nachname", };
DataTable dt = new DataTable();
foreach (string header in headercolumns)
{
dt.Columns.Add(header);
}
List<string> StaffList = new List<string>();
using (StreamReader filereader = new StreamReader(StaffFileUpload.FileContent))
{
string readline = string.Empty;
while ((readline = filereader.ReadLine()) != null)
{
StaffList.Add(readline);
}
if (StaffList != null && StaffList.Count > 0)
{
foreach (string info in StaffList)
{
DataRow row = dt.NewRow();
string[] mitarbeiter = info.Split(';');
for (int i = 0; i < mitarbeiter.Length; i++)
{
row[i] = mitarbeiter[i];
}
dt.Rows.Add(row);
}
}
}
I want to add a new column with checkbox, my data is from a csv file and showed it in a datagridview with this code:
DataTable dtDataSource = new DataTable();
string[] fileContent = File.ReadAllLines(\data.csv);
if (fileContent.Count() > 0)
{
//Create data table columns
dtDataSource.Columns.Add("ID);
dtDataSource.Columns.Add("Data 1");
dtDataSource.Columns.Add("Data 2");
dtDataSource.Columns.Add("Status");
//Add row data dynamically
for (int i = 1; i < fileContent.Count(); i++)
{
string[] rowData = fileContent[i].Split(',');
dtDataSource.Rows.Add(rowData);
}
if (dtDataSource != null)
{
dataGridView1.DataSource = dtDataSource;
}
}
But also I need to validate if checkbox is checked, the column ¨Status¨, their value must be changed by 1 or if is it unchecked the value must be 0 in every row of datagridview.
Example:
ID,Data1,Data2,Status,checkbox
1,aaa,bbb,0,✓
2,ccc,ddd,1,(unchecked)
3,eee,fff,1,(unchecked)
When you click the save button, the csv file should looks like this:
ID,Data1,Data2,Status
1,aaa,bbb,1
2,ccc,ddd,0
3,eee,fff,0
What I should do? Any ideas? CSV file is a little difficult for me.
Thank you!
I resolved this, thanks anyway..
This is the code:
string id;
for (int i = 0; i < dataGridView1.RowCount; i++) {
String path = "\\registros.csv";
List<String> lines = new List<String>();
if (File.Exists(path))
{
using (StreamReader reader = new StreamReader(path))
{
String line;
while ((line = reader.ReadLine()) != null)
{
id = (string)dataGridView1.Rows[i].Cells[2].Value;
if (line.Contains(","))
{
String[] split = line.Split(',');
if (split[1].Equals(id) && (bool)dataGridView1.Rows[i].Cells[0].FormattedValue == true)
{
split[10] = "" + 1;
line = String.Join(",", split);
}
if (split[1].Equals(id) && (bool)dataGridView1.Rows[i].Cells[0].FormattedValue == false)
{
split[10] = "" + 0;
line = String.Join(",", split);
}
}
lines.Add(line);
}
}
using (StreamWriter writer = new StreamWriter(path, false))
{
foreach (String line in lines)
writer.WriteLine(line);
}
}
}
I am trying to convert string to DataTable using the ffollowing method , But It did"nt work
public static DataTable convertStringToDataTable(string data)
{
DataTable dataTable = new DataTable();
bool columnsAdded = false;
foreach (string row in data.Split('\n'))
{
DataRow dataRow = dataTable.NewRow();
foreach (string cell in row.Split(','))
{
string[] keyValue = cell.Split('"');
if (!columnsAdded)
{
DataColumn dataColumn = new DataColumn(keyValue[0]);
dataTable.Columns.Add(dataColumn);
}
dataRow[keyValue[0]] = keyValue[1];
}
columnsAdded = true;
dataTable.Rows.Add(dataRow);
}
return dataTable;
}
Code that contains data string :
StringWriter sw = new StringWriter();
sw.WriteLine("\"NumClient\",\"Raisons Sociale\",\"DateDocument\",\"NumCommandeNAV\",\"Réference\",\"Designation\",\"QteCommandée\",\"QteLivrée\",\"QteAnnulée\",\"Reste à Livrer\",\"Type Disponibilite\",\"DateDisponibilite\"");
var EnTete =
db.AURES_GROS_EnTeteCommande.Where(e => e.NumCommandeNAV != " " && e.NumCommandeNAV != "_")
.OrderBy(x => x.CodeMagasin)
.ThenBy(s => s.NumClient)
.ThenBy(c => c.DateDocument)
.OrderByDescending(x => x.NumCommandeNAV)
.ToList();
foreach (var element in EnTete)
{
string statut = RecuperStatut(element.NumCommandeNAV);
if (statut == "A livrer")
{
Raison = context.Users.First(x => x.No_ == element.NumClient).RaisonSociale;
lignes = db.AURES_GROS_LigneCommande.Where(x => x.NumDocument == element.NumDocument).ToList();
foreach (var elt in lignes)
{
sw.WriteLine(string.Format("\"{0}\",\"{1}\",\"{2}\",\"{3}\",\"{4}\",\"{5}\",\"{6}\",\"{7}\",\"{8}\",\"{9}\",\"{10}\",\"{11}\"",
element.NumClient,
Raison,
element.DateDocument,
element.NumCommandeNAV,
elt.CodeArticle,
elt.Designation,
elt.Quantite,
0,
elt.QteANNULEE,
elt.Quantite,
element.Couleur,
elt.DateDisponibilite
));
}
}
}
DataTable t = convertStringToDataTable(sw.ToString());
Response.ClearContent();
Response.ClearHeaders();
Response.BufferOutput = true;
Response.ContentType = "text/excel";
Response.AddHeader("Content-Disposition", "attachment; filename=Reliquat" + DateTime.Now.ToString("yyyy_MM_dd") + ".csv");
Response.Write(t);
Response.Flush();
Response.Close();
//Response.End();
In this code I want I wnat to export data to Excel file
Any one have solutions
Thanks,
I think you want to create a DataTable from a string. So first split the "rows" and then the "columns". You are adding the DataColumns in the row-loop. You need that only once before the loop. Here is another implementation which handles this and other edge cases you haven't considered yet:
public static DataTable ConvertStringToDataTable(string data)
{
DataTable dataTable = new DataTable();
// extract all lines:
string[] lines = data.Split(new string[] { "\r\n", "\n" }, StringSplitOptions.RemoveEmptyEntries);
string header = lines.FirstOrDefault();
if (header == null)
return dataTable;
// first create the columns:
string[] columns = header.Split(','); // using commas as delimiter is brave ;)
foreach (string col in columns)
dataTable.Columns.Add(col.Trim());
foreach (string line in lines.Skip(1))
{
string[] fields = line.Split(',');
if(fields.Length != dataTable.Columns.Count)
continue; // should not happen
DataRow dataRow = dataTable.Rows.Add();
for (int i = 0; i < fields.Length; i++)
dataRow.SetField(i, fields[i]);
}
return dataTable;
}
You can convert your column foreach to a for loop.
public static DataTable convertStringToDataTable(string data)
{
DataTable dataTable = new DataTable();
bool columnsAdded = false;
foreach (string row in data.Split('\n'))
{
DataRow dataRow = dataTable.NewRow();
string[] cell = row.Split(',');
for (int i = 0; i < cell.Length; i++)
{
string[] keyValue = cell[i].Split('"');
if (!columnsAdded)
{
DataColumn dataColumn = new DataColumn();
dataTable.Columns.Add(dataColumn);
}
dataRow[i] = keyValue[1];
}
columnsAdded = true;
dataTable.Rows.Add(dataRow);
}
return dataTable;
}
However if your split string[] keyValue = cell.Split('"'); is not returning what you are expecting you may need to investigate further.
I am using the below function to import a csv file into a datatable object csvData, however i face an oject reference issue that i'd like to understand more while using Regex.Replace to remove quote marks from the data:
private static DataTable Gettabledata(string cpath)
{
DataTable csvData = new DataTable();
try
{
using (TextFieldParser csvReader = new TextFieldParser(cpath))
{
csvReader.SetDelimiters(new string[] { "," });
csvReader.HasFieldsEnclosedInQuotes = true;
string[] colFields = csvReader.ReadFields();
foreach (string column in colFields)
{
DataColumn datecolumn = new DataColumn(column);
datecolumn.AllowDBNull = true;
csvData.Columns.Add(datecolumn);
}
while (!csvReader.EndOfData)
{
string[] fieldData = csvReader.ReadFields();
string pattern="\""; % remove quotation mark "
string replacement=""; % replace by empty.(eg "a", a)
Regex rgx = new Regex(pattern);
for (int i = 0; i < fieldData.Length; i++)
fieldData[i] = Regex.Replace(fieldData[i],pattern, replacement); %object reference issue
{
if (fieldData[i] == "")
{
fieldData[i] = null;
}
}
csvData.Rows.Add(fieldData);
}
}
}
catch (Exception ex)
{
}
return csvData;
}
The fieldData[i] = Regex.Replace(fieldData[i],pattern, replacement); is outside the {...} block. It should be inside it.
for (int i = 0; i < fieldData.Length; i++)
{
if (fieldData[i] == "")
{
fieldData[i] = null;
}
else
{
fieldData[i] = Regex.Replace(fieldData[i],pattern, replacement);
}
}
Want to create a generic text file parser in c# for any find of text file.Actually i have 4 application all 4 getting input data from txt file format but text files are not homogeneous in nature.i have tried fixedwithdelemition.
private static DataTable FixedWidthDiliminatedTxtRead()
{
string[] fields;
StringBuilder sb = new StringBuilder();
List<StringBuilder> lst = new List<StringBuilder>();
DataTable dtable = new DataTable();
ArrayList aList;
using (TextFieldParser tfp = new TextFieldParser(testOCC))
{
tfp.TextFieldType = FieldType.FixedWidth;
tfp.SetFieldWidths(new int[12] { 2,25,8,12,13,5,6,3,10,11,10,24 });
for (int col = 1; col < 13; ++col)
dtable.Columns.Add("COL" + col);
while (!tfp.EndOfData)
{
fields = tfp.ReadFields();
aList = new ArrayList();
for (int i = 0; i < fields.Length; ++i)
aList.Add(fields[i] as string);
if (dtable.Columns.Count == aList.Count)
dtable.Rows.Add(aList.ToArray());
}
}
return dtable;
}
but i feel its very rigid one and really varies application to application making it configgurable .any better way ..
tfp.SetFieldWidths(new int[12] { 2,25,8,12,13,5,6,3,10,11,10,24 });
File nature :
Its a report kind of file .
position of columns are very similar
row data of file id different .
I get this as a reference
http://www.codeproject.com/Articles/11698/A-Portable-and-Efficient-Generic-Parser-for-Flat-F
any other thoughts ?
If the only thing different is the field widths, you could just try sending the field widths in as a parameter:
private static DataTable FixedWidthDiliminatedTxtRead(int[] fieldWidthArray)
{
string[] fields;
StringBuilder sb = new StringBuilder();
List<StringBuilder> lst = new List<StringBuilder>();
DataTable dtable = new DataTable();
ArrayList aList;
using (TextFieldParser tfp = new TextFieldParser(testOCC))
{
tfp.TextFieldType = FieldType.FixedWidth;
tfp.SetFieldWidths(fieldWidthArray);
for (int col = 1; col < 13; ++col)
dtable.Columns.Add("COL" + col);
while (!tfp.EndOfData)
{
fields = tfp.ReadFields();
aList = new ArrayList();
for (int i = 0; i < fields.Length; ++i)
aList.Add(fields[i] as string);
if (dtable.Columns.Count == aList.Count)
dtable.Rows.Add(aList.ToArray());
}
}
return dtable;
}
If you will have more logic to grab the data, you might want to consider defining an interface or abstract class for a GenericTextParser and create concrete implementations for each other file.
Hey I made one of these last week.
I did not write it with the intentions of other people using it so I appologize in advance if its not documented well but I cleaned it up for you. ALSO I grabbed several segments of code from stack overflow so I am not the original author of several pieces of this.
The places you need to edit are the path and pathout and the seperators of text.
char[] delimiters = new char[]
So it searches for part of a word and then grabs the whole word. I used a c# console application for this.
Here you go:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
namespace UniqueListofStringFinder
{
class Program
{
static void Main(string[] args)
{
string path = #"c:\Your Path\in.txt";
string pathOut = #"c:\Your Path\out.txt";
string data = "!";
Console.WriteLine("Current Path In is set to: " + path);
Console.WriteLine("Current Path Out is set to: " + pathOut);
Console.WriteLine(Environment.NewLine + Environment.NewLine + "Input String to Search For:");
Console.Read();
string input = Console.ReadLine();
// Delete the file if it exists.
if (!File.Exists(path))
{
// Create the file.
using (FileStream fs = File.Create(path))
{
Byte[] info =
new UTF8Encoding(true).GetBytes("This is some text in the file.");
// Add some information to the file.
fs.Write(info, 0, info.Length);
}
}
System.IO.StreamReader file = new System.IO.StreamReader(path);
List<string> Spec = new List<string>();
using (StreamReader sr = File.OpenText(path))
{
while (!file.EndOfStream)
{
string s = file.ReadLine();
if (s.Contains(input))
{
char[] delimiters = new char[] { '\r', '\n', '\t', ')', '(', ',', '=', '"', '\'', '<', '>', '$', ' ', '#', '[', ']' };
string[] parts = s.Split(delimiters,
StringSplitOptions.RemoveEmptyEntries);
foreach (string word in parts)
{
if (word.Contains(input))
{
if( word.IndexOf(input) == 0)
{
Spec.Add(word);
}
}
}
}
}
Spec.Sort();
// Open the stream and read it back.
//while ((s = sr.ReadLine()) != null)
//{
// Console.WriteLine(s);
//}
}
Console.WriteLine();
StringBuilder builder = new StringBuilder();
foreach (string s in Spec) // Loop through all strings
{
builder.Append(s).Append(Environment.NewLine); // Append string to StringBuilder
}
string result = builder.ToString(); // Get string from StringBuilder
Program a = new Program();
data = a.uniqueness(result);
int i = a.writeFile(data,pathOut);
}
public string uniqueness(string rawData )
{
if (rawData == "")
{
return "Empty Data Set";
}
List<string> dataVar = new List<string>();
List<string> holdData = new List<string>();
bool testBool = false;
using (StringReader reader = new StringReader(rawData))
{
string line;
while ((line = reader.ReadLine()) != null)
{
foreach (string s in holdData)
{
if (line == s)
{
testBool = true;
}
}
if (testBool == false)
{
holdData.Add(line);
}
testBool = false;
// Do something with the line
}
}
int i = 0;
string dataOut = "";
foreach (string s in holdData)
{
dataOut += s + "\r\n";
i++;
}
// Write the string to a file.
return dataOut;
}
public int writeFile(string dataOut, string pathOut)
{
try
{
System.IO.StreamWriter file = new System.IO.StreamWriter(pathOut);
file.WriteLine(dataOut);
file.Close();
}
catch (Exception ex)
{
dataOut += ex.ToString();
return 1;
}
return 0;
}
}
}
private static DataTable FixedWidthTxtRead(string filename, int[] fieldWidths)
{
string[] fields;
DataTable dtable = new DataTable();
ArrayList aList;
using (TextFieldParser tfp = new TextFieldParser(filename))
{
tfp.TextFieldType = FieldType.FixedWidth;
tfp.SetFieldWidths(fieldWidths);
for (int col = 1; col <= fieldWidths.length; ++col)
dtable.Columns.Add("COL" + col);
while (!tfp.EndOfData)
{
fields = tfp.ReadFields();
aList = new ArrayList();
for (int i = 0; i < fields.Length; ++i)
aList.Add(fields[i] as string);
if (dtable.Columns.Count == aList.Count) dtable.Rows.Add(aList.ToArray());
}
}
return dtable;
}
Here's what I did:
I built a factory for the type of processor needed (based on file type/format), which abstracted the file reader.
I then built a collection object that contained a set of triggers for each field I was interested in (also contained the property name for which this field is destined). This settings collection is loaded in via an XML configuration file, so all I need to change are the settings, and the base parsing process can react to how the settings are configured. Finally I built a reflection wrapper wherein once a field is parsed, the corresponding property on the model object is set.
As the file flowed through, the triggers for each setting evaluated each lines value. When it found what it was set to find (via pattern matching, or column length values) it fired and event that bubbled up and set a property on the model object. I can show some pseudo code if you're interested. It needs some work for efficiency's sake, but I like the concept.