We Keep Coding

Is it possible to convert a DataTable to IEnumerable<T> where the T can not be defined at compile time and is not known beforehand?

So I'll explain my situation first.
I have a WPF View for my customer that is populated based on SQL strings that the customer defines. They can change these and add/remove these at any point and the structure of the result set is not in my control.
My expected output for this is
Populating the DataGrid at runtime without prior knowledge of the structure so using AutoGenerateColumns and providing dataTable.DefaultView as the ItemsSource for the DataGrid. This is bound to my DataGrid.
GetItemsSource = dataTable.DefaultView;
Export this DataGrid to a CSV for the customer to check whenever they want.
Now I already have a Generic List function to Save to CSV but since the structure is not known I can't change my dataTable to a list to use this.
My current solution is Save To CSV function that uses a dataTable instead of a List.
Is there some other type of data structure I could use instead of dataTable that would make using my generic function possible or do I have just have an extra Save To CSV function just for this scenario?
UPDATE
My generic list function
public static void SaveToCsv<T>(List<T> data, string filePath) where T : class
{
CreateDirectoryIfNotExists(filePath);
List<string> lines = new();
StringBuilder line = new();
if (data == null || data.Count == 0)
{
throw new ArgumentNullException("data", "You must populate the data parameter with at least one value.");
}
var cols = data[0].GetType().GetProperties();
foreach (var col in cols)
{
line.Append(col.Name);
line.Append(",");
}
lines.Add(line.ToString().Substring(0, line.Length - 1));
foreach (var row in data)
{
line = new StringBuilder();
foreach (var col in cols)
{
line.Append(col.GetValue(row));
line.Append(",");
}
lines.Add(line.ToString().Substring(0, line.Length - 1));
}
System.IO.File.WriteAllLines(filePath, lines);
}
My current Data Table function
public static void SaveToCsv(DataTable data, string filePath)
{
CreateDirectoryIfNotExists(filePath);
List<string> lines = new();
StringBuilder line = new();
if(data == null)
{
throw new ArgumentNullException("data", "The DataTable has no values to Save to CSV.");
}
IEnumerable<string> columnNames = data.Columns.Cast<DataColumn>().Select(column => column.ColumnName);
line.AppendLine(string.Join(",", columnNames));
lines.Add(line.ToString().Substring(0, line.Length - 3));
int prevlinelength = line.Length - 1;
foreach (DataRow row in data.Rows)
{
IEnumerable<string> fields = row.ItemArray.Select(field => field.ToString());
line.AppendLine(string.Join(",", fields));
lines.Add(line.ToString().Substring(prevlinelength + 1, line.Length - 3 - prevlinelength));
prevlinelength = line.Length - 1;
}
File.WriteAllLines(filePath, lines);
}

Is it possible to convert a DataTable to IEnumerable where the T can not be defined at compile time and is not known beforehand?
you can create generic objects at runtime, but it is not simple, so I would avoid it if possible.
Is there some other type of data structure I could use instead of dataTable that would make using my generic function possible or do I have just have an extra Save To CSV function just for this scenario?
You could simply convert the Rows property on your datatable and convert it to a List<DataRow> and give to your function. But it would probably not do what you want.
What you need is a some way to convert a DataRow into an object of a class with properties for each column, and while it is possible to create classes from a database model, it will be a lot of work to do so at runtime. I would guess far more than your current solution.
To conclude, keep your current solution if it works. Messing around with reflection and runtime code generation will just make things more complicated.

How pass only some string values from one string list to enother

I have array string list that is actually a csv file that every filed of this list is a file row .
In this csv file (List) i have more then 20 columns and i need to create a new list that will contain only specific columns from the original list .
how can i do that ?
this is the list:
List<string[]> parsedData = new List<string[]>();
parsedData = ParseResultCSV();

Every cell in a CSV file is separated with a ';'.
So something like this:
var listOfParsedValues = new List<List<string>>();
foreach(var row in parsedData){
var cells = row.Split(';');
// if you for example want to save values at cell 4 and 7:
var valuesOfThisRow;
valuesOfThisRow.Add(cells[4]);
valuesOfThisRow.Add(cells[7]);
listOfParsedValues.Add(valuesOfThisRow);
}
Not optimal, but it works :)

Fastest way to fill DataTable manually in C#

I want to know if someone of you know a faster way to fill a DataTable manually then I do.
Here is what I got, I have a List with about 1.7b entries.
I want to fill this entries as fast as possible into DataTable with one column.
An entry in my list looks like this here {"A2C","DDF","ER","SQ","8G"}
My code need about 7-8 seconds
for (int i = 0; i <= lists.Count; i++)
{
table_list.Rows.Add();
}
for (int a = 0; a < list.Count; a++)
{
table_list.Rows[a][0] = list[a][0] + list[a][1] +
list[a][2] + list[a][3] + list[a][4];
}
As I didn't find any similar question on the board (just questions about how to fill datatable by sql and fill method), I decided to post my question.
Any input is highly appreciated!

i add this DataTable into an sql server database (i do this by SqlBulkCopy)
This is a mistake; the DataTable is pure overhead here. What you should expose is an IDataReader over that data. This API is a bit tricky, but FastMember makes it easier. For example, it sounds like you have 1 column; so consider:
class Foo {
public string ColumnName {get;set;}
}
Now write an iterator block method that converts this from the original list per item:
IEnumerable<Foo> Convert(List<TheOldType> list) {
foreach(var row in list) {
yield return new Foo { ColumnName = /* TODO */ };
}
}
and now create an IDataReader via FastMember on top of that lazy sequence:
List<TheOldType> list
var data = Convert(list);
using(var bcp = new SqlBulkCopy(connection))
using(var reader = ObjectReader.Create(data, "ColumnName"))
{
bcp.DestinationTableName = "SomeTable";
bcp.WriteToServer(reader);
}
This works much better than populating a DataTable - in particular, it avoids populating a huge DataTable. Emphasis: the above is spooling - not buffered.

Why do you create an empty row first, then loop the table again to fill them?
I would use a simple foreach:
var table_list = new DataTable();
table_list.Columns.Add();
foreach(string[] fields in lists)
{
DataRow newRow = table_list.Rows.Add();
newRow.SetField(0, string.Join("", fields));
}
Why do you put all into one field?

Why not use the LoadDataRow method of the DataTable.
// turnoff notifications
table_list.BeginLoadData();
// load each row into the table
foreach(string[] fields in lists)
table_list.LoadDataRow(new object[] { string.Join("", fields) }, false);
// turn notifications back on
table_list.EndLoadData();
Also see: DataTable.LoadDataRow Method http://msdn.microsoft.com/en-us/library/kcy03ww2(v=vs.110).aspx

How to filter data in one List from data in another?

I have two identically sized Lists in my application. One contains date information, and the other contains the water conductivity data for that date. I am using the two Lists to plot information on a graph. I am now attempting to add a slider which allows the user to filter the data by a certain number of days. Here is my code so far:
// Filter the date data (this works!)
var filteredDates = from n in parsedDateList[0]
where n >= beginDate.Date
select n;
//Filter the y-axis data (this does not work!)
var filteredCond = waterConductivityList[1].Where(x => parsedDateList[0].Any(y=> y.Date > beginDate));
Could someone fill me in on what I am doing wrong? The y-axis filter simply returns the full list of information rather than filtering.

So here is what I did:
Dictionary<DateTime, int> conductivityData = new Dictionary<DateTime, int>();
// Get the y-values
i=0;
foreach (var entry in waterConductivityData[1])
{
condData[i] = Convert.ToInt32(entry);
i++;
}
// Add the dates ("entry" from datelist) and the y-value (condData) to the dictionary
i=0;
foreach (var entry in parsedDateList[0])
{
conductivityData.Add(entry, condData[i]);
i++;
}
//Add the data to the plot series using "key" for dates and "value" for y-data
foreach (var entry in conductivityData)
{
filteredDateStrings[0].Add(entry.Key.ToString("M/d/yy hh:mm tt"));
filteredCondData[0].Add(entry.Value);
}
// Update plot data
i = 0;
foreach (var entry in filteredCondData[0])
{
waterSourceTwoChart.Series["conductivity"].Points.AddXY(filteredDateStrings[0].ElementAt(i), filteredCondData[0].ElementAt(i));
i++;
}
Thanks everyone for the help!

Nested foreach put into a string array in C#

So I have an array coming in with hotels information and one piece I need out of it is the location of each hotel so then I can send that into a different method. So I have my first foreach setup, but now I am wondering how to collect all of the data on the locations into a string array so I can send that out after all of the hotels are read. Can someone help, thanks.
// API call to get the info
ContentAPI.BasicHotelMedia[] rawData = DataHelper.NewContentAPI().GetBasicHotelMedia(ProductCode, ProductYear, SiteBrand);
//setting up the datatable
DataTable dtHotels = InitHotelTable();
//set my variables
foreach (ContentAPI.BasicHotelMedia item in rawData)
{
DataRow dr = dtHotels.NewRow();
dr["HotelCode"] = item.BasicHotelCode;
dr["HotelDescription"] = item.BasicDescription;
dr["WiFi"] = item.HasWifi;
// This is the variable that i need set in the string array so i can send into another method
dr["SellingLocation"] = item.BasicSellingLocation;
// Add other raw data
// Get other info about the hotel
GetHotelMedia(item.BasicHotelCode, ProductYear, item.HasWifi, ref dr);
dtHotels.Rows.Add(dr.ItemArray);
}

I'd suggest using a List instead of initializing a string[]. They're just easier to work with.
Llike this:
var locations = new List<string>();
foreach (ContentAPI.BasicHotelMedia item in rawData)
{
...
locations.Add(item.BasicSellingLocation);
}
OtherMethod(locations.ToArray());