We Keep Coding

Parsing data from xls and added to a data grid view using c#

I am trying to read a folder that contain a number of xls files (27) and i need to read only 3 specific columns after 21 row e.g. A21,: B21,: ... In a new column a would like to have just the sum of the previous columns. I am thinking to insert a database grid and to insert there the parsing data. My problem is that i have never try to read something from xls. Do you have any ideas. Thanks in advance! (All the data are in the same workshhet in all workbooks)

Load CSV file in Database

I just started learning C# and I’ve made a few simple applications. The application I’m working on now is an application that reads and enters data to a (Access) database. I connected successfully to the database and I can enter data to it. Now the problem.
I have a csv file with data (orders). I want to load these orders to the database. The csv file looks like:
Order 1: 2 cakes,01-01-2013,chocolate,Jan|Order 2: 5 cakes,01-08-2013,vanilla,Piet|
As you can see it is ordered by (same as in my database table): [Order #], [Amount], [Date], [KindOfCake],[Buyer] and the new order comes right after the halfpipe (|).
This is what I've got:
String[] orders1= File.ReadAllText(#"c:\\orders.csv").Split('|');
for (int i = 0; i < orders1.Length; i++)
{
textBox1.AppendText(orders[i] + Environment.NewLine);
}
But this will load all the text into one textBox. I'm thinking about loading the data in the csv (seperated with a ',') to different textBoxes and then loading the values of the textBoxes to a database. But that is not the best way I'm guessing.
What is the best way for me to load all data in the CSV to my database?
Thank you very much for the tips.

Just like you're splitting on the pipe character, you can then split each string on the comma.
String[] orders = File.ReadAllText(#"c:\\orders.csv").Split('|');
foreach (string order in orders) {
String[] orderFields = order.Split(',');
// Now you have your fields, put them in the DB. No need to put them
// into text boxes
}
But, what happens if a comma appears in a field? This wouldn't work for that case.

I would load a DataTable with the information from your CSV. Then create a connection to your database and insert that DataTable into the database. This link appears to do what you are attempting, with some good code samples: http://www.codeproject.com/Articles/11435/Importing-CSV-Data-and-saving-it-in-database or http://www.codeproject.com/Articles/9258/A-Fast-CSV-Reader.

Handling empty fields in csv during bulk import to database

I'm importing geocode data to our database from a csv file.
I've used the following library
A fast csv reader to read the csv and then using SqlBulkCopy
Here's an example of the data I'm importing
"AB10","1BH","L",0,0,,,20
"AB10","1BR","L",39320,80570,57.14214,-2.11400,21
It works ok on good data but on the top line it will throw an exception because the database is set up to not accept null values.
Is there a way to tell bulkcopy to ignore bad data? I've tried to get the csv reader to ignore bad lines by using the in built properties of the library like so but they don't appear to work.
csv.SkipEmptyLines = true;
csv.MissingFieldAction = MissingFieldAction.ParseError;
csv.DefaultParseErrorAction = ParseErrorAction.AdvanceToNextLine;
I guess another option would be to pre-parse the csv and remove all the offending rows. Perhaps there's a better csv library out there for .net?

If you could post your csv reader code then we could help more. But looking at the code on your linked page, you could do something like this:
while (csv.ReadNextRecord())
{
for (int i = 0; i < fieldCount; i++)
Console.Write(string.Format("{0} = {1};",
headers[i],
csv[i] ?? "null"));
Console.WriteLine();
}
See where I have added that null-coalescing operator? This should change your output from:
"AB10","1BH","L",0,0,,,20
to
"AB10","1BH","L",0,0,null,null,20

I used the Microsoft Text Driver to import CSV information for a project. It worked pretty well. I defined a Schema.ini file to specify the column headers, data types, number of rows to scan (MaxScanRows=0, will scan the whole file).
I haven't tried this but when you use Microsoft Text Driver you issue a select query to pull the data out of the csv file, I'm wondering if you could add criteria to filter out the null records.
How to populate IDataReader from .csv for use with SqlBulkCopy.WriteToServer(IDataReader)
http://msdn.microsoft.com/en-us/library/windows/desktop/ms709353(v=vs.85).aspx
http://www.connectionstrings.com/textfile
Hope this helps.

To deal with the null entries I ended up parsing the csv into a DataTable object 1000 entries at a time and then imported them as I went.

Import data from multiple CSV files to an Excel sheet

I need to import data from 50 similar csv files to a single excel sheet.
Is there any way to get only selected columns from each file and put them together in one sheet.
Structure of my csv files: A few columns exactly same in all the files. (I want them to be in excel) then one column with same column name but different data which I want to place next to each other with different names in the excel sheet. I do not not want all other remaining columns from csv files.
In short,
read all csv files,
get all the columns which are common to all csv & put in excel sheet.
Now, take one column from each file which has the same header but different data and put one after the other in excel sheet with named from the csv file name.
Leave remaining rest of the columns.
Write excel sheet to a excel file.
Initially I thought it can be easily done, but considering my programming skill in learning stage it is way difficult for me. Please help.

Microsoft Text Driver allows you to read CSV data in a DataSet, making data manipulation easy.
This Stack Overflow question is a good starting point.

Fastest way could be using FileHelpers to read CSV into a DataTable :
http://filehelpers.sourceforge.net/FileHelpers.CommonEngine.CsvToDataTable_overload_4.html
and then with EPPlus export that DataTable in excel, use method DataTableToExcelXlsx from this snippet:
https://stackoverflow.com/a/9569827/351383
With EPPlus you don't have to have Excel installed on machine that is executing this code, and you can use it on server (ASP.NET)

With a very simple coding, I was able to read the files. Now, the only thing we need to do is to make this code a bit fancy to loop thorough all the CSV files in the folder given and collect data. Once we read the data, it can be filtered and put to an excel as we want.
Of course, excel can import CSV itself, but it is not that practical to do this every time. And again we can add the code to application to use in flexibility, exactly what I am trying to do.
public static System.Data.DataTable GetDataTable(string strFileName)
{
System.Data.OleDb.OleDbConnection dbConnect = new System.Data.OleDb.OleDbConnection("Provider=Microsoft.Jet.OleDb.4.0; Data Source = " + System.IO.Path.GetDirectoryName(strFileName) + ";Extended Properties = \"Text;HDR=YES;FMT=TabDelimited\"");
dbConnect.Open();
string strQuery = "SELECT * FROM [" + System.IO.Path.GetFileName(strFileName) + "]";
System.Data.OleDb.OleDbDataAdapter adapter = new System.Data.OleDb.OleDbDataAdapter(strQuery, dbConnect);
System.Data.DataSet dSet = new System.Data.DataSet("CSV File");
adapter.Fill(dSet);
dbConnect.Close();
return dSet.dbTables[0];
}

How to eliminate trailing empty row/columns

I am using Microsoft Interop to convert excel files into csv files. I use sheet.SaveAs function.
My initial excel sheet has data from A1 to AZ columns for 100 rows.
I need in the CSV just the data from A1 to AP and only for 50 rows.
Using the Range function, I delete the row51-100, I clear the contents for the same rows, still when I save as CSV, I find rows 51-100 as below: (just commas). I do not want to see these commas in CSV.
,,,,,,,,,,,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,,,,,,,,,
The same for column AQ-AZ as well. I do not want these data in CSV. I delete, clear contents using Range function, yet these AQ-AZ columns appears in CSV files as “,,,,,,,,,,,,,,,,,,,,,” .
Is there a way to save XLS as CSV with only Range that I want to see in the CSV file. Is there a way to control the range that goes into CSV file?
In short, I want to see in CSV file just the data for column A1 to AP for 50 rows. No empty trailing “,”s. Is there a way?

The issue you are describing seems like a "Last Cell" issue. The last cell is the original end of your data, even after you delete rows/columns.
Here is what Microsoft has to say about it: How to reset the last cell in Excel
I seem to remember a programmatic way of doing this, but for the life of me, I cannot recall how.
Having looked at that info, maybe you could rethink how you can do this.
Perhaps you could just read the data you need and write it out yourself.
i.e. For each row in range, get the row as a value which will be an array of object,
convert to array of string, string.join with the delimiter as a comma and append
to a .csv file.

Clearing the contents as suggested in another answer did not work for me, what did work was copying the populated columns in a new worksheet and overwriting the old CSV.

Simply select the trailing empty columns in Excel, right click and select: clear contents. Then save.