We Keep Coding

Is it possible to convert a DataTable to IEnumerable<T> where the T can not be defined at compile time and is not known beforehand?

So I'll explain my situation first.
I have a WPF View for my customer that is populated based on SQL strings that the customer defines. They can change these and add/remove these at any point and the structure of the result set is not in my control.
My expected output for this is
Populating the DataGrid at runtime without prior knowledge of the structure so using AutoGenerateColumns and providing dataTable.DefaultView as the ItemsSource for the DataGrid. This is bound to my DataGrid.
GetItemsSource = dataTable.DefaultView;
Export this DataGrid to a CSV for the customer to check whenever they want.
Now I already have a Generic List function to Save to CSV but since the structure is not known I can't change my dataTable to a list to use this.
My current solution is Save To CSV function that uses a dataTable instead of a List.
Is there some other type of data structure I could use instead of dataTable that would make using my generic function possible or do I have just have an extra Save To CSV function just for this scenario?
UPDATE
My generic list function
public static void SaveToCsv<T>(List<T> data, string filePath) where T : class
{
CreateDirectoryIfNotExists(filePath);
List<string> lines = new();
StringBuilder line = new();
if (data == null || data.Count == 0)
{
throw new ArgumentNullException("data", "You must populate the data parameter with at least one value.");
}
var cols = data[0].GetType().GetProperties();
foreach (var col in cols)
{
line.Append(col.Name);
line.Append(",");
}
lines.Add(line.ToString().Substring(0, line.Length - 1));
foreach (var row in data)
{
line = new StringBuilder();
foreach (var col in cols)
{
line.Append(col.GetValue(row));
line.Append(",");
}
lines.Add(line.ToString().Substring(0, line.Length - 1));
}
System.IO.File.WriteAllLines(filePath, lines);
}
My current Data Table function
public static void SaveToCsv(DataTable data, string filePath)
{
CreateDirectoryIfNotExists(filePath);
List<string> lines = new();
StringBuilder line = new();
if(data == null)
{
throw new ArgumentNullException("data", "The DataTable has no values to Save to CSV.");
}
IEnumerable<string> columnNames = data.Columns.Cast<DataColumn>().Select(column => column.ColumnName);
line.AppendLine(string.Join(",", columnNames));
lines.Add(line.ToString().Substring(0, line.Length - 3));
int prevlinelength = line.Length - 1;
foreach (DataRow row in data.Rows)
{
IEnumerable<string> fields = row.ItemArray.Select(field => field.ToString());
line.AppendLine(string.Join(",", fields));
lines.Add(line.ToString().Substring(prevlinelength + 1, line.Length - 3 - prevlinelength));
prevlinelength = line.Length - 1;
}
File.WriteAllLines(filePath, lines);
}

Is it possible to convert a DataTable to IEnumerable where the T can not be defined at compile time and is not known beforehand?
you can create generic objects at runtime, but it is not simple, so I would avoid it if possible.
Is there some other type of data structure I could use instead of dataTable that would make using my generic function possible or do I have just have an extra Save To CSV function just for this scenario?
You could simply convert the Rows property on your datatable and convert it to a List<DataRow> and give to your function. But it would probably not do what you want.
What you need is a some way to convert a DataRow into an object of a class with properties for each column, and while it is possible to create classes from a database model, it will be a lot of work to do so at runtime. I would guess far more than your current solution.
To conclude, keep your current solution if it works. Messing around with reflection and runtime code generation will just make things more complicated.

How to get Column Names using Linq from DataTable

I'm trying to use LINQ on DataTable that's getting it's data from sql. So I have a data table with it's usual columns and rows and it appears exactly like a sql select statement. Now I need to get certain rows and columns (including column names) from this data.
I converted the datatable to something LINQ can use using AsEnumerable but I'm not sure what exactly it does. Does it convert the data into an array of objects where each row becomes an object?
I'm used to working with Javascript and it's newer arrow functions so i'd like to use Linq with lambda to keep it consistent.
I'm trying to get rows and column names where first column has a value equal to 2018
DataTable myTable = getData(); // populates the datatable and I've verified the data
var linqTable = myTable.AsEnumerable().Select( x => x[0] = 2018);
I need to get the rows and column names. e.g like an object or array of objects.However, the code above doesn't return the data or column names but just two rows with 2018 in it.
My goal is to eventually serialize this data as json and send it to web page.

To Get the column names:
myTable.Columns.Cast<DataColumn>().Select(dc =>dc.ColumnName).ToList();

The problem is Select() is projecting the objects into a new form. You are seeing 2018 because of '=' instead of '=='. You need to use Where()
var linqTable = myTable.AsEnumerable().Where( x => x.Field<int>(0) == 2018);
You will still end up with a list of DataRows though. The DataTable object isn't really what you should be using because it already provides a nice way to filter its rows:
myTable.Rows.Find(2018);
If you are trying to convert it to a list of objects you should use the Select() method something like:
var linqTable = myTable.AsEnumerable().Where(x => x.Field<int>(0) == 2018)
.Select(x => new
{
year = x[0],
p1 = x[1],
p2 = x[2] // etc...
});

You can create the following function:
public static DataTable CreateDataTableFromAnyCollection<T>(IEnumerable<T> list)
{
Type type = typeof(T);
var properties = type.GetProperties();
DataTable dataTable = new DataTable();
foreach (PropertyInfo info in properties)
{
dataTable.Columns.Add(new DataColumn(info.Name, Nullable.GetUnderlyingType(info.PropertyType) ?? info.PropertyType));
}
foreach (T entity in list)
{
object[] values = new object[properties.Length];
for (int i = 0; i < properties.Length; i++)
{
values[i] = properties[i].GetValue(entity,null);
}
dataTable.Rows.Add(values);
}
return dataTable;
}
and pass any type of object your LINQ query returning.
DataTable dt = CreateDataTableFromAnyCollection(query);
I hope this will help you.
Creating a DataTable From a Query (LINQ to DataSet)

Import txt data into existing SQL Server table in .net using LINQ

The txt file is of a specific form, it uses ';' as delimiter and has a specific number of columns. I also have a table that I created code-first with Entity Framework, which has the same number of columns.
So far I was able to import that kind of txt files to tables using "raw" SQL queries like BULK INSERT. But I am trying to learn how to do this from a web app using C# (or LINQ if needed).
I came across this solution from another question, but it seems that it creates a table named tbl, what I would like to do instead is to insert the data into an existing one.
public DataTable ConvertToDataTable (string filePath, int numberOfColumns)
{
DataTable tbl = new DataTable();
for(int col =0; col < numberOfColumns; col++)
tbl.Columns.Add(new DataColumn("Column" + (col+1).ToString()));
string[] lines = System.IO.File.ReadAllLines(filePath);
foreach(string line in lines)
{
var cols = line.Split(':');
DataRow dr = tbl.NewRow();
for(int cIndex=0; cIndex < 3; cIndex++)
{
dr[cIndex] = cols[cIndex];
}
tbl.Rows.Add(dr);
}
return tbl;
}

First of all, my advise would be not to read the CSV file yourself. Use a NUGET CSV file serializer like CSVHelper
With CSVHelper you directly convert the lines into your destination type:
using (TextReader txtReader = new StreamReader(sourceFileName)
{
csvReader = new CsvReader(txtReader)
IEnumerable<MyClass> result = csvReader.GetRecords<MyClass>()
// TODO: put result into database
}
One of the constructors of CsvReader takes a configuration object in which you can define your delimiter (":"); header rows; Comment lines; what to do with empty lines etc.
If you decide not to use CsvHelper you will need to convert your lines into MyClass objects:
IEnumerable<MyClass> ConvertTxtFile(string fileName)
{
// TODO: checks to see if fileName is proper file
IEnumerable<string> lines = System.IO.File.ReadAllLines(fileName);
foreach(string line in lines)
{
yield return StringToMyClass(line);
}
}
MyClass StringToMyClass(string line)
{
// TODO: code to convert your line into a MyClass.
}
As you don't ask how to convert a line into a MyClass, I leave this to you.
After a while, you have a sequence of MyClass objects. Your question is how to add them to your database using Entity Framework and Linq
Well, that will be the easy part (once you've learned how to use entity framework).
Supposing your DbContext has a DbSet<MyClass>, representing a table of MyClass objects
IEnumerable<MyClass> readItems = ConvertTxtFile(fileName);
using (var dbContext = new MyDbContext())
{
dbContext.MyClasses.AddRange(readItems.ToList());
dbContext.SaveChanges();
}

Targeting a specific column in a DataRow

I'm trying to perform the C# equivalent of Select * where [columnname] = [value]. I began with a foreach loop to iterate through the table row by row, however I had forgotten that one cannot access a column via row.column["<colname>"].
How do I achieve this objective? Most of the examples I have seen target one specific row with the intention of casting it's value to a string, however my task is to move all entries with a value of DateTime == < DateTime.Today to an archived table.
Can I continue with the following code? Or am I approaching this in the wrong manner?
void archiveDates()
{
foreach (DataRow row in workingupdates.storageTable.Rows)
{
//target DateTime column here
}
}

You can use the Field extension method that is strongly typed and also supports nullable types. You have an overload for the index, name or the DataColumn(among others):
foreach (DataRow row in workingupdates.storageTable.Rows)
{
DateTime dt = row.Field<DateTime>("columnname");
}
If you instead want to find all rows where the date column has a specific value you can use Linq-To-DataTable:
var matchingDataRows = workingupdates.storageTable.AsEnumerable()
.Where(row => row.Field<DateTime>("columnname") == dateTimeVariable);
Now you can simply enumerate this query:
foreach (DataRow row in matchingDataRows)
{
// ...
}
Or create a collection like
a DataRow[] with matchingDataRows.ToArray() or
a List<DataRow> with matchingDataRows.ToList()
a new DataTable with matchingDataRows.CopyToDataTable()
Note that you have to add System.Linq; to the top of the file.

Best practice when converting DataColumn values to an array of strings?

Best practice when converting DataColumn values to an array of strings?
[Edit]
All values for certain DataColumn for all DataTable rows to be converted to an array of string?

If I understood your goal you want to specify a particular column and return all its values as a string array.
Try these approaches out:
int columnIndex = 2; // desired column index
// for loop approach
string[] results = new string[dt.Rows.Count];
for (int index = 0; index < dt.Rows.Count; index++)
{
results[index] = dt.Rows[index][columnIndex].ToString();
}
// LINQ
var result = dt.Rows.Cast<DataRow>()
.Select(row => row[columnIndex].ToString())
.ToArray();
You could replace columnIndex with columnName instead, for example:
string columnName = "OrderId";"
EDIT: you've asked for a string array specifically but in case you're flexible about the requirements I would prefer a List<string> to avoid the need to determine the array length prior to the for loop in the first example and simply add items to it. It's also a good opportunity to use a foreach loop instead.
I would then rewrite the code as follows:
List<string> list = new List<string>();
foreach (DataRow row in dt.Rows)
{
list.Add(row[columnIndex].ToString());
}

DataRow.ItemArray Property -
http://msdn.microsoft.com/en-us/library/system.data.datarow.itemarray.aspx
Also, which version are you using? You should check out the DataTableExtensions class -
http://msdn.microsoft.com/en-us/library/system.data.datatableextensions.aspx
And the DataRowExtensions class -
http://msdn.microsoft.com/en-us/library/system.data.datarowextensions.aspx

I know this question is old, but I found it in my Google search trying to do something similar. I wanted to create a list from all the values contained in a specific row of my datatable. In my code example below, I added a SQL datasource to my project in Visual Studio using the GUI wizards and I dropped the needed table adapter into the designer.
'Create a private DataTable
Private authTable As New qmgmtDataSet.AuthoritiesDataTable
'Fill the private table using the table adapter
Me.AuthoritiesTableAdapter1.Fill(Me.authTable)
'Make the list of values
Dim authNames As List(Of String) = New List(Of String)(From value As qmgmtDataSet.AuthoritiesRow In Me.authTable.Rows Select names.authName)