We Keep Coding

Import txt data into existing SQL Server table in .net using LINQ

The txt file is of a specific form, it uses ';' as delimiter and has a specific number of columns. I also have a table that I created code-first with Entity Framework, which has the same number of columns.
So far I was able to import that kind of txt files to tables using "raw" SQL queries like BULK INSERT. But I am trying to learn how to do this from a web app using C# (or LINQ if needed).
I came across this solution from another question, but it seems that it creates a table named tbl, what I would like to do instead is to insert the data into an existing one.
public DataTable ConvertToDataTable (string filePath, int numberOfColumns)
{
DataTable tbl = new DataTable();
for(int col =0; col < numberOfColumns; col++)
tbl.Columns.Add(new DataColumn("Column" + (col+1).ToString()));
string[] lines = System.IO.File.ReadAllLines(filePath);
foreach(string line in lines)
{
var cols = line.Split(':');
DataRow dr = tbl.NewRow();
for(int cIndex=0; cIndex < 3; cIndex++)
{
dr[cIndex] = cols[cIndex];
}
tbl.Rows.Add(dr);
}
return tbl;
}

First of all, my advise would be not to read the CSV file yourself. Use a NUGET CSV file serializer like CSVHelper
With CSVHelper you directly convert the lines into your destination type:
using (TextReader txtReader = new StreamReader(sourceFileName)
{
csvReader = new CsvReader(txtReader)
IEnumerable<MyClass> result = csvReader.GetRecords<MyClass>()
// TODO: put result into database
}
One of the constructors of CsvReader takes a configuration object in which you can define your delimiter (":"); header rows; Comment lines; what to do with empty lines etc.
If you decide not to use CsvHelper you will need to convert your lines into MyClass objects:
IEnumerable<MyClass> ConvertTxtFile(string fileName)
{
// TODO: checks to see if fileName is proper file
IEnumerable<string> lines = System.IO.File.ReadAllLines(fileName);
foreach(string line in lines)
{
yield return StringToMyClass(line);
}
}
MyClass StringToMyClass(string line)
{
// TODO: code to convert your line into a MyClass.
}
As you don't ask how to convert a line into a MyClass, I leave this to you.
After a while, you have a sequence of MyClass objects. Your question is how to add them to your database using Entity Framework and Linq
Well, that will be the easy part (once you've learned how to use entity framework).
Supposing your DbContext has a DbSet<MyClass>, representing a table of MyClass objects
IEnumerable<MyClass> readItems = ConvertTxtFile(fileName);
using (var dbContext = new MyDbContext())
{
dbContext.MyClasses.AddRange(readItems.ToList());
dbContext.SaveChanges();
}

Determine if input file is usable by program

I have a C# program that looks through directories for .txt files and loads each into a DataTable.
static IEnumerable<string> ReadAsLines(string fileName)
{
using (StreamReader reader = new StreamReader(fileName))
while (!reader.EndOfStream)
yield return reader.ReadLine();
}
public DataTable GetTxtData()
{
IEnumerable<string> reader = ReadAsLines(this.File);
DataTable txtData = new DataTable();
string[] headers = reader.First().Split('\t');
foreach (string columnName in headers)
txtData.Columns.Add(columnName);
IEnumerable<string> records = reader.Skip(1);
foreach (string rec in records)
txtData.Rows.Add(rec.Split('\t'));
return txtData;
}
This works great for regular tab-delimited files. However, the catch is that not every .txt file in the folders I need to use contains tab-delimited data. Some .txt files are actually SQL queries, notes, etc. that have been saved as plain text files, and I have no way of determining that beforehand. Trying to use the above code on such files clearly won't lead to the expected result.
So my question is this: How can I tell whether a .txt file actually contains tab-delimited data before I try to read it into a DataTable using the above code?
Just searching the file for any tab character won't work because, for example, a SQL query saved as plain text might have tabs for code formatting.
Any guidance here at all would be much appreciated!

If each line contains the same number of elements, then simply read each line, and verify that you get the correct number of fields in each record. If not error out.
if (headers.Count() != CORRECTNUMBER)
{
// ERROR
}
foreach (string rec in records)
{
string[] recordData = rec.Split('\t');
if (recordData.Count() != headers.Count())
{
// ERROR
}
txtData.Rows.Add(recordData);
}

To do this you need a set of "signature" logic providers which can check a given sample of the file for "signature" content. This is similar to how virus scanners work.
Consider you would create a set of classes where the ISignature was implemented by set of classes;
class TSVFile : ISignature
{
enumFileType ISignature.Evaluate(IEnumerable<byte> inputStream);
}
class SQLFile : ISignature
{
enumFileType ISignature.Evaluate(IEnumerable<byte> inputStream);
}
each one would read an appropriate number of bytes in and return the known file type, if it can be evaluated. Each file parser would need its own logic to determine how many bytes to read and on what basis to make its evaluation.

Print an array/list to excel in c#

I am able to save a single value into excel but I need help to save a full list/array into an excel sheet.
Code I have so far:
var MovieNames = session.Query<Movie>()
.ToArray();
List<string> MovieList = new List<string>();
foreach (var movie in MovieNames)
{
MovieList.Add(movie.MovieName);
}
//If I want to print a single value or a string,
//I can use the following to print/save to excel
// How can I do this if I want to print that entire
//list thats generated in "MovieList"
return File(new System.Text.UTF8Encoding().GetBytes(MovieList), "text/csv", "demo.csv");

You could use FileHelpers to serialize some strongly typed object into CSV. Just promise me to never roll your own CSV parser.

If you mean you want to create a .csv file with all movie names in one column so you can open it in Excel then simply loop over it:
byte[] content;
using (var ms = new MemoryStream())
{
using (var writer = new StreamWriter(ms))
{
foreach (var movieName in MovieList)
writer.WriteLine(movieName);
}
content = ms.ToArray();
}
return File(content, "text/csv", "demo.csv");
Edit
You can add more columns and get fancier with your output but then you run into the problem that you have check for special characters which need escaping (like , and "). If you want to do more than just a simple output then I suggest you follow #Darins suggestion and use the FileHelpers utilities. If you can't or don't want to use them then this article has an implementation of a csv writer.

Convert Dataset to Textfile tab delimited file

I have a DataSet. I would like to convert dataset column as header and row data as data into a tab delimited text file.
Is there any technique I can do in my end or I have to do the looping manually?
Sincerely Thanks,
- Sel

private static string GetTextFromDataTable(DataTable dataTable)
{
var stringBuilder = new StringBuilder();
stringBuilder.AppendLine(string.Join("\t", dataTable.Columns.Cast<DataColumn>().Select(arg => arg.ColumnName)));
foreach (DataRow dataRow in dataTable.Rows)
stringBuilder.AppendLine(string.Join("\t", dataRow.ItemArray.Select(arg => arg.ToString())));
return stringBuilder.ToString();
}
Usage:
var text = GetTextFromDataTable(dataSet.Tables[0]);
File.WriteAllText(filePath, text);

Exporting to XML is built right in, but exporting to CSV, you can use the following code - from http://social.msdn.microsoft.com/Forums/en-US/vbgeneral/thread/d2071fd4-8c7d-4d0e-94c3-9586df754df8/
this only writes the data, not the columns, you'll need to loop the column headers first..
Edit: Updated to include column names... I have not run this, and this is an edit from the link above, so it may or may not work, but the concept is here
StringBuilder str = new StringBuilder();
// get the column headers
foreach (var c in NorthwindDataSet.Customers.Columns) {
str.Append("\"" + c.ColumnName.ToString() + "\"\t");
}
str.Append("\r\n");
// write the data here
foreach (DataRow dr in this.NorthwindDataSet.Customers) {
foreach (var field in dr.ItemArray) {
str.Append("\"" + field.ToString() + "\"\t");
}
str.Append("\r\n");
}
try {
My.Computer.FileSystem.WriteAllText("C:\\temp\\testcsv.csv", str.ToString(), false);
} catch (Exception ex) {
MessageBox.Show("Write Error");
}

Note you will need to be using Linq for this solution to work. Add the following using statement to your code:
using System.Linq;

How can i find out the Last row in a datagridview and write it to text file

I would like to find the last row of the datagridview and i would like to write that particular row data to a text file can any one help me

I will give you a sample code, it may help.
List<string> lstContents = new List<string>();
foreach (DataGridViewCell cell in mydataGrid.Rows[mydataGrid.RowCount - 1].Cells)
{
lstContents .Add((string)cell.Value);
}
string myData= string.Join(",", lstContents.ToArray());
Now using streamwriter you can write this string to your file. Also you can use any separator. I have used "," comma here.
///This will append data to your text file...
using (StreamWriter sw = new StreamWriter(FilePath, true))
{
sw.WriteLine(myData);
}