We Keep Coding

Is it possible to convert a DataTable to IEnumerable<T> where the T can not be defined at compile time and is not known beforehand?

So I'll explain my situation first.
I have a WPF View for my customer that is populated based on SQL strings that the customer defines. They can change these and add/remove these at any point and the structure of the result set is not in my control.
My expected output for this is
Populating the DataGrid at runtime without prior knowledge of the structure so using AutoGenerateColumns and providing dataTable.DefaultView as the ItemsSource for the DataGrid. This is bound to my DataGrid.
GetItemsSource = dataTable.DefaultView;
Export this DataGrid to a CSV for the customer to check whenever they want.
Now I already have a Generic List function to Save to CSV but since the structure is not known I can't change my dataTable to a list to use this.
My current solution is Save To CSV function that uses a dataTable instead of a List.
Is there some other type of data structure I could use instead of dataTable that would make using my generic function possible or do I have just have an extra Save To CSV function just for this scenario?
UPDATE
My generic list function
public static void SaveToCsv<T>(List<T> data, string filePath) where T : class
{
CreateDirectoryIfNotExists(filePath);
List<string> lines = new();
StringBuilder line = new();
if (data == null || data.Count == 0)
{
throw new ArgumentNullException("data", "You must populate the data parameter with at least one value.");
}
var cols = data[0].GetType().GetProperties();
foreach (var col in cols)
{
line.Append(col.Name);
line.Append(",");
}
lines.Add(line.ToString().Substring(0, line.Length - 1));
foreach (var row in data)
{
line = new StringBuilder();
foreach (var col in cols)
{
line.Append(col.GetValue(row));
line.Append(",");
}
lines.Add(line.ToString().Substring(0, line.Length - 1));
}
System.IO.File.WriteAllLines(filePath, lines);
}
My current Data Table function
public static void SaveToCsv(DataTable data, string filePath)
{
CreateDirectoryIfNotExists(filePath);
List<string> lines = new();
StringBuilder line = new();
if(data == null)
{
throw new ArgumentNullException("data", "The DataTable has no values to Save to CSV.");
}
IEnumerable<string> columnNames = data.Columns.Cast<DataColumn>().Select(column => column.ColumnName);
line.AppendLine(string.Join(",", columnNames));
lines.Add(line.ToString().Substring(0, line.Length - 3));
int prevlinelength = line.Length - 1;
foreach (DataRow row in data.Rows)
{
IEnumerable<string> fields = row.ItemArray.Select(field => field.ToString());
line.AppendLine(string.Join(",", fields));
lines.Add(line.ToString().Substring(prevlinelength + 1, line.Length - 3 - prevlinelength));
prevlinelength = line.Length - 1;
}
File.WriteAllLines(filePath, lines);
}

Is it possible to convert a DataTable to IEnumerable where the T can not be defined at compile time and is not known beforehand?
you can create generic objects at runtime, but it is not simple, so I would avoid it if possible.
Is there some other type of data structure I could use instead of dataTable that would make using my generic function possible or do I have just have an extra Save To CSV function just for this scenario?
You could simply convert the Rows property on your datatable and convert it to a List<DataRow> and give to your function. But it would probably not do what you want.
What you need is a some way to convert a DataRow into an object of a class with properties for each column, and while it is possible to create classes from a database model, it will be a lot of work to do so at runtime. I would guess far more than your current solution.
To conclude, keep your current solution if it works. Messing around with reflection and runtime code generation will just make things more complicated.

C# List Sort vs Inserting

I have a list of objects which I sort multiple times throughout code and when the user interacts with the program. I was wondering if it would be better to insert new items into the list rather than add to the end of the list and resort the entire list.
The code below is for importing browser bookmarks - Here I add a bunch of bookmarks to the List (this._MyLinks) which are Link objects and then sort the final List - Which I think is probably best in this given scenario....
public void ImportBookmarks(string importFile)
{
using (var file = File.OpenRead(importFile))
{
var reader = new NetscapeBookmarksReader();
var bookmarks = reader.Read(file);
foreach (var b in bookmarks.AllLinks)
{
bool duplicate = this._MyLinks.Any(link => link._URL == b.Url);
if(duplicate)
{
continue;
}
Link bookmark = new Link();
bookmark._URL = b.Url;
bookmark._SiteName = b.Title;
bookmark.BrowserPath = "";
bookmark.BrowserName = "";
if (bookmark.AddToConfig(true))
{
this._MyLinks.Add(bookmark);
}
}
}
this._MyLinks = this._MyLinks.OrderBy(o => o._SiteName).ToList();
}
Now a user also has the option to add their own links (one at a time). Whenever the user adds a link the ENTIRE list is sorted again using
this._MyLinks = this._MyLinks.OrderBy(o => o._SiteName).ToList();
Is it better from a preformance standpoint (or just generally) to just insert the item directly into it's specified location? If so would you have suggestions on how I can go about doing that?
Thanks!

Since you want a sorted set of data you should be using a more appropriate data structure, specifically a sorted data structure, rather than using an unsorted data structure that you re-sort every time, or that forces you to inefficiently add items to the middle of a list.
SortedSet is specifically designed to maintain a sorted set of data efficiently.

Determine if input file is usable by program

I have a C# program that looks through directories for .txt files and loads each into a DataTable.
static IEnumerable<string> ReadAsLines(string fileName)
{
using (StreamReader reader = new StreamReader(fileName))
while (!reader.EndOfStream)
yield return reader.ReadLine();
}
public DataTable GetTxtData()
{
IEnumerable<string> reader = ReadAsLines(this.File);
DataTable txtData = new DataTable();
string[] headers = reader.First().Split('\t');
foreach (string columnName in headers)
txtData.Columns.Add(columnName);
IEnumerable<string> records = reader.Skip(1);
foreach (string rec in records)
txtData.Rows.Add(rec.Split('\t'));
return txtData;
}
This works great for regular tab-delimited files. However, the catch is that not every .txt file in the folders I need to use contains tab-delimited data. Some .txt files are actually SQL queries, notes, etc. that have been saved as plain text files, and I have no way of determining that beforehand. Trying to use the above code on such files clearly won't lead to the expected result.
So my question is this: How can I tell whether a .txt file actually contains tab-delimited data before I try to read it into a DataTable using the above code?
Just searching the file for any tab character won't work because, for example, a SQL query saved as plain text might have tabs for code formatting.
Any guidance here at all would be much appreciated!

If each line contains the same number of elements, then simply read each line, and verify that you get the correct number of fields in each record. If not error out.
if (headers.Count() != CORRECTNUMBER)
{
// ERROR
}
foreach (string rec in records)
{
string[] recordData = rec.Split('\t');
if (recordData.Count() != headers.Count())
{
// ERROR
}
txtData.Rows.Add(recordData);
}

To do this you need a set of "signature" logic providers which can check a given sample of the file for "signature" content. This is similar to how virus scanners work.
Consider you would create a set of classes where the ISignature was implemented by set of classes;
class TSVFile : ISignature
{
enumFileType ISignature.Evaluate(IEnumerable<byte> inputStream);
}
class SQLFile : ISignature
{
enumFileType ISignature.Evaluate(IEnumerable<byte> inputStream);
}
each one would read an appropriate number of bytes in and return the known file type, if it can be evaluated. Each file parser would need its own logic to determine how many bytes to read and on what basis to make its evaluation.

how to represent a CSV File as a data structure in a C# program

I have a csv file I am going to read from disk. I do not know up front how many columns or the names of the columns.
Any thoughts on how I should represent the fields. Ideally I want to say something like,
string Val = DataStructure.GetValue(i,ColumnName).
where i is the ith Row.
Oh just as an aside I will be parsing using the TextFieldParser class
http://msdn.microsoft.com/en-us/library/cakac7e6(v=vs.90).aspx

That sounds as if you would need a DataTable which has a Rows and Columns property.
So you can say:
string Val = table.Rows[i].Field<string>(ColumnName);
A DataTable is a table of in-memory data. It can be used strongly typed (as suggested with the Field method) but actually it stores it's data as objects internally.
You could use this parser to convert the csv to a DataTable.
Edit: I've only just seen that you want to use the TextFieldParser. Here's a possible simple approach to convert a csv to a DataTable:
var table = new DataTable();
using (var parser = new TextFieldParser(File.OpenRead(path)))
{
parser.Delimiters = new[]{","};
parser.HasFieldsEnclosedInQuotes = true;
// load DataColumns from first line
String[] headers = parser.ReadFields();
foreach(var h in headers)
table.Columns.Add(h);
// load all other lines as data '
String[] fields;
while ((fields = parser.ReadFields()) != null)
{
table.Rows.Add().ItemArray = fields;
}
}

If the column names are in the first row read that and store in a Dictionary<string, int> that maps the column name to the column index.
You could then store the remaining rows in a simple structure like List<string[]>.
To get a column for a row you'd do csv[rowIndex][nameToIndex[ColumnName]];
nameToIndex[ColumnName] gets the column index from the name, csv[rowIndex] gets the row (string array) we want.
This could of course be wrapped in a class.

Use the csv parser if you want, but a text parser is something very easy to do by yourself if you need customization.
For you need, i would use one (or more) Dictionnary. At least one to have the PropertyString --> column index. And maybe the reverse one column index--> PropertyString if needed.
When i parse a file for csv, i usually put the result in a list while parsing, and then in an array once complete for speed reasons (List.ToArray()).

Print an array/list to excel in c#

I am able to save a single value into excel but I need help to save a full list/array into an excel sheet.
Code I have so far:
var MovieNames = session.Query<Movie>()
.ToArray();
List<string> MovieList = new List<string>();
foreach (var movie in MovieNames)
{
MovieList.Add(movie.MovieName);
}
//If I want to print a single value or a string,
//I can use the following to print/save to excel
// How can I do this if I want to print that entire
//list thats generated in "MovieList"
return File(new System.Text.UTF8Encoding().GetBytes(MovieList), "text/csv", "demo.csv");

You could use FileHelpers to serialize some strongly typed object into CSV. Just promise me to never roll your own CSV parser.

If you mean you want to create a .csv file with all movie names in one column so you can open it in Excel then simply loop over it:
byte[] content;
using (var ms = new MemoryStream())
{
using (var writer = new StreamWriter(ms))
{
foreach (var movieName in MovieList)
writer.WriteLine(movieName);
}
content = ms.ToArray();
}
return File(content, "text/csv", "demo.csv");
Edit
You can add more columns and get fancier with your output but then you run into the problem that you have check for special characters which need escaping (like , and "). If you want to do more than just a simple output then I suggest you follow #Darins suggestion and use the FileHelpers utilities. If you can't or don't want to use them then this article has an implementation of a csv writer.