Currently, I'm using:
DataTable dt = CreateDataTableInSomeWay();
List<DataRow> list = new List<DataRow>();
foreach (DataRow dr in dt.Rows)
{
list.Add(dr);
}
Is there a better/magic way?
If you're using .NET 3.5, you can use DataTableExtensions.AsEnumerable (an extension method) and then if you really need a List<DataRow> instead of just IEnumerable<DataRow> you can call Enumerable.ToList:
IEnumerable<DataRow> sequence = dt.AsEnumerable();
or
using System.Linq;
...
List<DataRow> list = dt.AsEnumerable().ToList();
List<Employee> emp = new List<Employee>();
//Maintaining DataTable on ViewState
//For Demo only
DataTable dt = ViewState["CurrentEmp"] as DataTable;
//read data from DataTable
//using lamdaexpression
emp = (from DataRow row in dt.Rows
select new Employee
{
_FirstName = row["FirstName"].ToString(),
_LastName = row["Last_Name"].ToString()
}).ToList();
With C# 3.0 and System.Data.DataSetExtensions.dll,
List<DataRow> rows = table.Rows.Cast<DataRow>().ToList();
You could use
List<DataRow> list = new List<DataRow>(dt.Select());
dt.Select() will return all rows in your table, as an array of datarows, and the List constructor accepts that array of objects as an argument to initially fill your list with.
If you just want a list of values from the "ID" int field returned, you could use...
List<int> ids = (from row in dt.AsEnumerable() select Convert.ToInt32(row["ID"])).ToList();
You can create a extension function as :
public static List<T> ToListof<T>(this DataTable dt)
{
const BindingFlags flags = BindingFlags.Public | BindingFlags.Instance;
var columnNames = dt.Columns.Cast<DataColumn>()
.Select(c => c.ColumnName)
.ToList();
var objectProperties = typeof(T).GetProperties(flags);
var targetList = dt.AsEnumerable().Select(dataRow =>
{
var instanceOfT = Activator.CreateInstance<T>();
foreach (var properties in objectProperties.Where(properties => columnNames.Contains(properties.Name) && dataRow[properties.Name] != DBNull.Value))
{
properties.SetValue(instanceOfT, dataRow[properties.Name], null);
}
return instanceOfT;
}).ToList();
return targetList;
}
var output = yourDataInstance.ToListof<targetModelType>();
I have added some modification to the code from this answer (https://stackoverflow.com/a/24588210/4489664) because for nullable Types it will return exception
public static List<T> DataTableToList<T>(this DataTable table) where T: new()
{
List<T> list = new List<T>();
var typeProperties = typeof(T).GetProperties().Select(propertyInfo => new
{
PropertyInfo = propertyInfo,
Type = Nullable.GetUnderlyingType(propertyInfo.PropertyType) ?? propertyInfo.PropertyType
}).ToList();
foreach (var row in table.Rows.Cast<DataRow>())
{
T obj = new T();
foreach (var typeProperty in typeProperties)
{
object value = row[typeProperty.PropertyInfo.Name];
object safeValue = value == null || DBNull.Value.Equals(value)
? null
: Convert.ChangeType(value, typeProperty.Type);
typeProperty.PropertyInfo.SetValue(obj, safeValue, null);
}
list.Add(obj);
}
return list;
}
using System.Data;
var myEnumerable = myDataTable.AsEnumerable();
List<MyClass> myClassList =
(from item in myEnumerable
select new MyClass{
MyClassProperty1 = item.Field<string>("DataTableColumnName1"),
MyClassProperty2 = item.Field<string>("DataTableColumnName2")
}).ToList();
Again, using 3.5 you may do it like:
dt.Select().ToList()
BRGDS
// this is better suited for expensive object creation/initialization
IEnumerable<Employee> ParseEmployeeTable(DataTable dtEmployees)
{
var employees = new ConcurrentBag<Employee>();
Parallel.ForEach(dtEmployees.AsEnumerable(), (dr) =>
{
employees.Add(new Employee()
{
_FirstName = dr["FirstName"].ToString(),
_LastName = dr["Last_Name"].ToString()
});
});
return employees;
}
A more 'magic' way, and doesn't need .NET 3.5.
If, for example, DBDatatable was returning a single column of Guids (uniqueidentifier in SQL) then you could use:
Dim gList As New List(Of Guid)
gList.AddRange(DirectCast(DBDataTable.Select(), IEnumerable(Of Guid)))
DataTable dt; // datatable should contains datacolumns with Id,Name
List<Employee> employeeList=new List<Employee>(); // Employee should contain EmployeeId, EmployeeName as properties
foreach (DataRow dr in dt.Rows)
{
employeeList.Add(new Employee{EmployeeId=dr.Id,EmplooyeeName=dr.Name});
}
The Easiest way of Converting the DataTable into the Generic list of class
using Newtonsoft.Json;
var json = JsonConvert.SerializeObject(dataTable);
var model = JsonConvert.DeserializeObject<List<ClassName>>(json);
DataTable.Select() doesnt give the Rows in the order they were present in the datatable.
If order is important I feel iterating over the datarow collection and forming a List is the right way to go or you could also use overload of DataTable.Select(string filterexpression, string sort).
But this overload may not handle all the ordering (like order by case ...) that SQL provides.
/* This is a generic method that will convert any type of DataTable to a List
*
*
* Example : List< Student > studentDetails = new List< Student >();
* studentDetails = ConvertDataTable< Student >(dt);
*
* Warning : In this case the DataTable column's name and class property name
* should be the same otherwise this function will not work properly
*/
The following are the two functions in which if we pass a
DataTable
and a user defined class.
It will then return the List of that class with the DataTable data.
public static List<T> ConvertDataTable<T>(DataTable dt)
{
List<T> data = new List<T>();
foreach (DataRow row in dt.Rows)
{
T item = GetItem<T>(row);
data.Add(item);
}
return data;
}
private static T GetItem<T>(DataRow dr)
{
Type temp = typeof(T);
T obj = Activator.CreateInstance<T>();
foreach (DataColumn column in dr.Table.Columns)
{
foreach (PropertyInfo pro in temp.GetProperties())
{
//in case you have a enum/GUID datatype in your model
//We will check field's dataType, and convert the value in it.
if (pro.Name == column.ColumnName){
try
{
var convertedValue = GetValueByDataType(pro.PropertyType, dr[column.ColumnName]);
pro.SetValue(obj, convertedValue, null);
}
catch (Exception e)
{
//ex handle code
throw;
}
//pro.SetValue(obj, dr[column.ColumnName], null);
}
else
continue;
}
}
return obj;
}
This method will check the datatype of field, and convert dataTable value in to that datatype.
private static object GetValueByDataType(Type propertyType, object o)
{
if (o.ToString() == "null")
{
return null;
}
if (propertyType == (typeof(Guid)) || propertyType == typeof(Guid?))
{
return Guid.Parse(o.ToString());
}
else if (propertyType == typeof(int) || propertyType.IsEnum)
{
return Convert.ToInt32(o);
}
else if (propertyType == typeof(decimal) )
{
return Convert.ToDecimal(o);
}
else if (propertyType == typeof(long))
{
return Convert.ToInt64(o);
}
else if (propertyType == typeof(bool) || propertyType == typeof(bool?))
{
return Convert.ToBoolean(o);
}
else if (propertyType == typeof(DateTime) || propertyType == typeof(DateTime?))
{
return Convert.ToDateTime(o);
}
return o.ToString();
}
To call the preceding method, use the following syntax:
List< Student > studentDetails = new List< Student >();
studentDetails = ConvertDataTable< Student >(dt);
Change the Student class name and dt value based on your requirements. In this case the DataTable column's name and class property name should be the same otherwise this function will not work properly.
lPerson = dt.AsEnumerable().Select(s => new Person()
{
Name = s.Field<string>("Name"),
SurName = s.Field<string>("SurName"),
Age = s.Field<int>("Age"),
InsertDate = s.Field<DateTime>("InsertDate")
}).ToList();
Link to working DotNetFiddle Example
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Data.DataSetExtensions;
public static void Main()
{
DataTable dt = new DataTable();
dt.Columns.Add("Name", typeof(string));
dt.Columns.Add("SurName", typeof(string));
dt.Columns.Add("Age", typeof(int));
dt.Columns.Add("InsertDate", typeof(DateTime));
var row1= dt.NewRow();
row1["Name"] = "Adam";
row1["SurName"] = "Adam";
row1["Age"] = 20;
row1["InsertDate"] = new DateTime(2020, 1, 1);
dt.Rows.Add(row1);
var row2 = dt.NewRow();
row2["Name"] = "John";
row2["SurName"] = "Smith";
row2["Age"] = 25;
row2["InsertDate"] = new DateTime(2020, 3, 12);
dt.Rows.Add(row2);
var row3 = dt.NewRow();
row3["Name"] = "Jack";
row3["SurName"] = "Strong";
row3["Age"] = 32;
row3["InsertDate"] = new DateTime(2020, 5, 20);
dt.Rows.Add(row3);
List<Person> lPerson = new List<Person>();
lPerson = dt.AsEnumerable().Select(s => new Person()
{
Name = s.Field<string>("Name"),
SurName = s.Field<string>("SurName"),
Age = s.Field<int>("Age"),
InsertDate = s.Field<DateTime>("InsertDate")
}).ToList();
foreach(Person pers in lPerson)
{
Console.WriteLine("{0} {1} {2} {3}", pers.Name, pers.SurName, pers.Age, pers.InsertDate);
}
}
public class Person
{
public string Name { get; set; }
public string SurName { get; set; }
public int Age { get; set; }
public DateTime InsertDate { get; set; }
}
}
Use System.Data namespace then you will get .AsEnumerable().
This worked for me:
Need at least .Net Framework 3.5,
Code below displays DataRow turned to Generic.IEnumerable, comboBox1 has been used for a better illustration.
using System.Linq;
DataTable dt = new DataTable();
dt = myClass.myMethod();
List<object> list = (from row in dt.AsEnumerable() select (row["name"])).ToList();
comboBox1.DataSource = list;
Output
public class ModelUser
{
#region Model
private string _username;
private string _userpassword;
private string _useremail;
private int _userid;
/// <summary>
///
/// </summary>
public int userid
{
set { _userid = value; }
get { return _userid; }
}
/// <summary>
///
/// </summary>
public string username
{
set { _username = value; }
get { return _username; }
}
/// <summary>
///
/// </summary>
public string useremail
{
set { _useremail = value; }
get { return _useremail; }
}
/// <summary>
///
/// </summary>
public string userpassword
{
set { _userpassword = value; }
get { return _userpassword; }
}
#endregion Model
}
public List<ModelUser> DataTableToList(DataTable dt)
{
List<ModelUser> modelList = new List<ModelUser>();
int rowsCount = dt.Rows.Count;
if (rowsCount > 0)
{
ModelUser model;
for (int n = 0; n < rowsCount; n++)
{
model = new ModelUser();
model.userid = (int)dt.Rows[n]["userid"];
model.username = dt.Rows[n]["username"].ToString();
model.useremail = dt.Rows[n]["useremail"].ToString();
model.userpassword = dt.Rows[n]["userpassword"].ToString();
modelList.Add(model);
}
}
return modelList;
}
static DataTable GetTable()
{
// Here we create a DataTable with four columns.
DataTable table = new DataTable();
table.Columns.Add("userid", typeof(int));
table.Columns.Add("username", typeof(string));
table.Columns.Add("useremail", typeof(string));
table.Columns.Add("userpassword", typeof(string));
// Here we add five DataRows.
table.Rows.Add(25, "Jame", "Jame#hotmail.com", DateTime.Now.ToString());
table.Rows.Add(50, "luci", "luci#hotmail.com", DateTime.Now.ToString());
table.Rows.Add(10, "Andrey", "Andrey#hotmail.com", DateTime.Now.ToString());
table.Rows.Add(21, "Michael", "Michael#hotmail.com", DateTime.Now.ToString());
table.Rows.Add(100, "Steven", "Steven#hotmail.com", DateTime.Now.ToString());
return table;
}
protected void Page_Load(object sender, EventArgs e)
{
List<ModelUser> userList = new List<ModelUser>();
DataTable dt = GetTable();
userList = DataTableToList(dt);
gv.DataSource = userList;
gv.DataBind();
}[enter image description here][1]
</asp:GridView>
</div>
We can use a Generic Method for converting DataTable to List instead of manually converting a DataTable to List.
Note: DataTable's ColumnName and Type's PropertyName should be same.
Call the below Method:
long result = Utilities.ConvertTo<Student>(dt ,out listStudent);
// Generic Method
public class Utilities
{
public static long ConvertTo<T>(DataTable table, out List<T> entity)
{
long returnCode = -1;
entity = null;
if (table == null)
{
return -1;
}
try
{
entity = ConvertTo<T>(table.Rows);
returnCode = 0;
}
catch (Exception ex)
{
returnCode = 1000;
}
return returnCode;
}
static List<T> ConvertTo<T>(DataRowCollection rows)
{
List<T> list = null;
if (rows != null)
{
list = new List<T>();
foreach (DataRow row in rows)
{
T item = CreateItem<T>(row);
list.Add(item);
}
}
return list;
}
static T CreateItem<T>(DataRow row)
{
string str = string.Empty;
string strObj = string.Empty;
T obj = default(T);
if (row != null)
{
obj = Activator.CreateInstance<T>();
strObj = obj.ToString();
NameValueCollection objDictionary = new NameValueCollection();
foreach (DataColumn column in row.Table.Columns)
{
PropertyInfo prop = obj.GetType().GetProperty(column.ColumnName);
if (prop != null)
{
str = column.ColumnName;
try
{
objDictionary.Add(str, row[str].ToString());
object value = row[column.ColumnName];
Type vType = obj.GetType();
if (value == DBNull.Value)
{
if (vType == typeof(int) || vType == typeof(Int16)
|| vType == typeof(Int32)
|| vType == typeof(Int64)
|| vType == typeof(decimal)
|| vType == typeof(float)
|| vType == typeof(double))
{
value = 0;
}
else if (vType == typeof(bool))
{
value = false;
}
else if (vType == typeof(DateTime))
{
value = DateTime.MaxValue;
}
else
{
value = null;
}
prop.SetValue(obj, value, null);
}
else
{
prop.SetValue(obj, value, null);
}
}
catch(Exception ex)
{
}
}
}
PropertyInfo ActionProp = obj.GetType().GetProperty("ActionTemplateValue");
if (ActionProp != null)
{
object ActionValue = objDictionary;
ActionProp.SetValue(obj, ActionValue, null);
}
}
return obj;
}
}
You can use a generic method like that for datatable to generic list
public static List<T> DataTableToList<T>(this DataTable table) where T : class, new()
{
try
{
List<T> list = new List<T>();
foreach (var row in table.AsEnumerable())
{
T obj = new T();
foreach (var prop in obj.GetType().GetProperties())
{
try
{
PropertyInfo propertyInfo = obj.GetType().GetProperty(prop.Name);
if (propertyInfo.PropertyType.IsEnum)
{
propertyInfo.SetValue(obj, Enum.Parse(propertyInfo.PropertyType, row[prop.Name].ToString()));
}
else
{
propertyInfo.SetValue(obj, Convert.ChangeType(row[prop.Name], propertyInfo.PropertyType), null);
}
}
catch
{
continue;
}
}
list.Add(obj);
}
return list;
}
catch
{
return null;
}
}
Converting DataTable to Generic Dictionary
public static Dictionary<object,IList<dynamic>> DataTable2Dictionary(DataTable dt)
{
Dictionary<object, IList<dynamic>> dict = new Dictionary<dynamic, IList<dynamic>>();
foreach(DataColumn column in dt.Columns)
{
IList<dynamic> ts = dt.AsEnumerable()
.Select(r => r.Field<dynamic>(column.ToString()))
.ToList();
dict.Add(column, ts);
}
return dict;
}
Use Extension :
public static class Extensions
{
#region Convert Datatable To List
public static IList<T> ToList<T>(this DataTable table) where T : new()
{
IList<PropertyInfo> properties = typeof(T).GetProperties().ToList();
IList<T> result = new List<T>();
foreach (var row in table.Rows)
{
var item = CreateItemFromRow<T>((DataRow)row, properties);
result.Add(item);
}
return result;
}
private static T CreateItemFromRow<T>(DataRow row, IList<PropertyInfo> properties) where T : new()
{
T item = new T();
foreach (var property in properties)
{
property.SetValue(item, row[property.Name], null);
}
return item;
}
#endregion
}
To assign the DataTable rows to the generic List of class
List<Candidate> temp = new List<Candidate>();//List that holds the Candidate Class,
//Note:The Candidate class contains RollNo,Name and Department
//tb is DataTable
temp = (from DataRow dr in tb.Rows
select new Candidate()
{
RollNO = Convert.ToInt32(dr["RollNO"]),
Name = dr["Name"].ToString(),
Department = dr["Department"].ToString(),
}).ToList();
you can use following two Generic functions
private static List<T> ConvertDataTable<T>(DataTable dt)
{
List<T> data = new List<T>();
foreach (DataRow row in dt.Rows)
{
T item = GetItem<T>(row);
data.Add(item);
}
return data;
}
private static T GetItem<T>(DataRow dr)
{
Type temp = typeof(T);
T obj = Activator.CreateInstance<T>();
foreach (DataColumn column in dr.Table.Columns)
{
foreach (PropertyInfo pro in temp.GetProperties())
{
if (pro.Name == column.ColumnName)
pro.SetValue(obj, dr[column.ColumnName].ToString(), null);
else
continue;
}
}
return obj;
}
and use it as following
List<StudentScanExamsDTO> studentDetails = ConvertDataTable<StudentScanExamsDTO>(dt);
If anyone want's to create custom function to convert datatable to list
class Program
{
static void Main(string[] args)
{
DataTable table = GetDataTable();
var sw = new Stopwatch();
sw.Start();
LinqMethod(table);
sw.Stop();
Console.WriteLine("Elapsed time for Linq Method={0}", sw.ElapsedMilliseconds);
sw.Reset();
sw.Start();
ForEachMethod(table);
sw.Stop();
Console.WriteLine("Elapsed time for Foreach method={0}", sw.ElapsedMilliseconds);
Console.ReadKey();
}
private static DataTable GetDataTable()
{
var table = new DataTable();
table.Columns.Add("ID", typeof(double));
table.Columns.Add("CategoryName", typeof(string));
table.Columns.Add("Active", typeof(double));
var rand = new Random();
for (int i = 0; i < 100000; i++)
{
table.Rows.Add(i, "name" + i, rand.Next(0, 2));
}
return table;
}
private static void LinqMethod(DataTable table)
{
var list = table.AsEnumerable()
.Skip(1)
.Select(dr =>
new Category
{
Id = Convert.ToInt32(dr.Field<double>("ID")),
CategoryName = dr.Field<string>("CategoryName"),
IsActive =
dr.Field<double>("Active") == 1 ? true : false
}).ToList();
}
private static void ForEachMethod(DataTable table)
{
var categoryList = new List<Category>(table.Rows.Count);
foreach (DataRow row in table.Rows)
{
var values = row.ItemArray;
var category = new Category()
{
Id = Convert.ToInt32(values[0]),
CategoryName = Convert.ToString(values[1]),
IsActive = (double)values[2] == 1 ? true : false
};
categoryList.Add(category);
}
}
private class Category
{
public int Id { get; set; }
public string CategoryName { get; set; }
public bool IsActive { get; set; }
}
}
If we execute above code, Foreach method finishes in 56ms while linq one takes 101ms ( for 1000 records).
So Foreach method is better to use.
Source:Ways to Convert Datatable to List in C# (with performance test example)
try this using Newtonsoft Json:
var json = JsonConvert.SerializeObject(dataTable);
var YourConvertedDataType = JsonConvert.DeserializeObject<YourDataType>(json);
To get List of values instead of ItemArray, do this:
List<string> s = dt.AsEnumerable().Select(x => x[0].ToString()).ToList();
The above assumes that you want list of string values from column 0.
Related
Have put together the code below to read a particular set of CSV files. It works but is very much a work in progress. There is one section of the code (populating datatable row - see snip below) that is taking as long to run as the SqlBulkCopy operation. Asking for advice/recommendations on how to improve performance.
In the code (below) processing a ~15M row file in 50K batches took just under 11.5 minutes. Breaking down the sections. SqlBulkCopy took ~236Kms (4 min) the reader only needed 105Kms (~1.5min), and the section populating the datatable took ~200Kms (3.33 min).
csvTableTimer.Start();
// Process row and populate datatable
DataRow dr = dt.NewRow();
foreach (DataColumn dc in dt.Columns)
{
if (row.GetType().GetProperty(dc.ToString()).GetValue(row) != null)
{
dr[dc.ToString()] = row.GetType().GetProperty(dc.ToString()).GetValue(row);
}
}
dt.Rows.Add(dr);
csvTableTimer.Stop();
The CSV files are very large (10+GB) and do not have headers. I'm using the Class to build the datatable structure and like to continue with that approach when populating the datatable rows as I'll need to expand this to work with multiple CSV types.
The datatable reflects the column names from the class which line up with the SQL DB table. Had wanted to use GetField (converted, not raw) walking each column in the datatable row[column.ColumnName] = csv.GetField( column.DataType, column.ColumnName ); but kept getting an error about there not being headers. Found an open issue relating to HasHeaderRecord = false that matches up with what I was trying to do so that added to my desire to seek advice from those who are more skilled at this. Appreciate the help!
Expanding on the code block;
var rconfig = new CsvHelper.Configuration.CsvConfiguration(CultureInfo.InvariantCulture)
{
BufferSize = 1024,
Delimiter = ",",
AllowComments = true,
HasHeaderRecord = false,
HeaderValidated = null,
IgnoreBlankLines = true,
MissingFieldFound = null,
Comment = '#',
Escape = '"',
TrimOptions = TrimOptions.Trim,
BadDataFound = x =>
{
isBadRecord = true;
ErrRecords.Add(x.RawRecord);
++badCount;
}
};
var loadFType = #"B";
// Create datatable using class as definition.
PropertyDescriptorCollection props1 = TypeDescriptor.GetProperties(loaderFileType);
DataTable dt = new DataTable();
dt = UtilExtensions.CreateDataTable(props1);
using (var reader = new StreamReader(rFile))
{
reader.ReadLine();
using (var csv = new CsvReader(reader, rconfig))
{
switch (loadFType)
{
case "ALL":
csv.Context.RegisterClassMap<CSVLoader.AMap>();
var allRecords = new List<CSVLoader.A>();
break;
case "BAL":
csv.Context.RegisterClassMap<CSVLoader.BMap>();
var balRecords = new List<CSVLoader.B>();
break;
case "CIF":
csv.Context.RegisterClassMap<CSVLoader.CMap>();
var cifRecords = new List<CSVLoader.C>();
break;
}
dt.BeginLoadData();
while (csv.Read())
{
csvReadTimer.Start();
var row = csv.GetRecord(loaderFileType);
csvReadTimer.Stop();
runningCount++;
if (!isBadRecord)
{
csvTableTimer.Start();
// Process row and populate datatable
DataRow dr = dt.NewRow();
foreach (DataColumn dc in dt.Columns)
{
if (row.GetType().GetProperty(dc.ToString()).GetValue(row) != null)
{
dr[dc.ToString()] = row.GetType().GetProperty(dc.ToString()).GetValue(row);
}
}
dt.Rows.Add(dr);
csvTableTimer.Stop();
++goodCount;
if (batchCount >= dtbatchSize || runningCount >= fileRecCount)
{
try
{
// Write from the source to the destination.
bcpLoadTimer.Start();
bulkCopy.WriteToServer(dt);
bcpLoadTimer.Stop();
bcpLoadBatchCount++;
}
catch (Exception ex)
{
}
dt.Clear();
batchCount = 0;
}
batchCount++;
}
isBadRecord = false;
}
dt.EndLoadData();
reader.Close();
dt.Clear();
transaction.Commit();
// B
public class B
{
[Index(0)]
public string A { get; set; }
[Index(1)]
public string BString { get; set; }
[Index(2)]
public int? C { get; set; }
[Index(3)]
public string D { get; set; }
[Index(4)]
public string E { get; set; }
[Index(5)]
public DateTime? F { get; set; }
[Index(6)]
public decimal? G { get; set; }
[Index(7)]
public decimal? H { get; set; }
[Index(8)]
public decimal? I { get; set; }
[Index(9)]
public decimal? J { get; set; }
[Index(10)]
public int? K { get; set; }
[Index(11)]
public string L { get; set; }
[Index(12)]
public DateTime? M { get; set; }
}
// B
public sealed class BMap : ClassMap<B>
{
public BMap()
{
// AutoMap(CultureInfo.InvariantCulture);
Map(m => m.A).Index(0);
Map(m => m.BString).Index(1);
Map(m => m.C).Index(2);
Map(m => m.D).Index(3);
Map(m => m.E).Index(4);
Map(m => m.F).Index(5).TypeConverterOption.Format("yyyyMMdd");
Map(m => m.G).Index(6);
Map(m => m.H).Index(7);
Map(m => m.I).Index(8);
Map(m => m.J).Index(9);
Map(m => m.K).Index(10);
Map(m => m.L).Index(11);
Map(m => m.M).Index(12).TypeConverterOption.Format("yyyy-MM-dd-hh.mm.ss.ffffff");
}
}
Your question doesn't really include a minimal reproducible example, so I simplified your code to create the following FileLoader class that times how long it takes to populate the DataTable from instances of some class TClass (here B) that had been read from a CSV row using CsvReader:
public class FileLoader
{
public System.Diagnostics.Stopwatch csvTableTimer { get; } = new();
public long Load<TClass, TClassMap>(string rFile, int dtbatchSize) where TClassMap : ClassMap<TClass>, new()
{
bool isBadRecord = false;
long badCount = 0;
long runningCount = 0;
long goodCount = 0;
long batchCount = 0;
var rconfig = CreateCsvConfiguration(
x =>
{
isBadRecord = true;
//ErrRecords.Add(x.RawRecord);
++badCount;
});
// Create datatable using class as definition.
var dt = UtilExtensions.CreateDataTable(typeof(TClass));
using (var reader = new StreamReader(rFile))
{
//reader.ReadLine(); FIXED - THIS SKIPPED THE FIRST LINE AND CAUSED A RECORD TO BE OMITTED.
using (var csv = new CsvReader(reader, rconfig))
{
csv.Context.RegisterClassMap<TClassMap>();
dt.BeginLoadData();
while (csv.Read())
{
isBadRecord = false;
//csvReadTimer.Start();
var record = csv.GetRecord<TClass>();
//csvReadTimer.Stop();
runningCount++;
if (!isBadRecord)
{
csvTableTimer.Start();
// Process row and populate datatable
DataRow dr = dt.NewRow();
foreach (DataColumn dc in dt.Columns)
{
if (record.GetType().GetProperty(dc.ToString()).GetValue(record) != null)
{
dr[dc.ToString()] = record.GetType().GetProperty(dc.ToString()).GetValue(record);
}
}
dt.Rows.Add(dr);
csvTableTimer.Stop();
goodCount++;
if (++batchCount >= dtbatchSize)
{
// Flush the data table
FlushTable(dt);
batchCount = 0;
}
}
}
dt.EndLoadData();
FlushTable(dt);
Commit();
}
}
return goodCount;
}
protected virtual void FlushTable(DataTable dt) => dt.Clear(); // Replace with SqlBulkCopy
protected virtual void Commit() {} // Replace with transaction.Commit();
public static CsvConfiguration CreateCsvConfiguration(BadDataFound badDataFound) =>
new CsvHelper.Configuration.CsvConfiguration(CultureInfo.InvariantCulture)
{
BufferSize = 1024,
Delimiter = ",",
AllowComments = true,
HasHeaderRecord = false,
HeaderValidated = null,
IgnoreBlankLines = true,
MissingFieldFound = null,
Comment = '#',
Escape = '"',
TrimOptions = TrimOptions.Trim,
BadDataFound = badDataFound,
};
}
public static partial class UtilExtensions
{
static IEnumerable<PropertyInfo> GetSerializableProperties(this Type type) =>
type.GetProperties().Where(p => p.GetIndexParameters().Length == 0 && p.CanRead && p.CanWrite && p.GetGetMethod() != null && p.GetSetMethod() != null);
public static DataTable CreateDataTable(Type type)
{
var dt = new DataTable();
foreach (var p in type.GetSerializableProperties())
dt.Columns.Add(p.Name, Nullable.GetUnderlyingType(p.PropertyType) ?? p.PropertyType);
return dt;
}
}
Then, if I use the file loader and call loader.Load<B, BMap>(rFile, 1000) to read a CSV file with 5555 rows 20 times, it takes roughly 1049 ms on dotnetfiddle. See demo #1 here.
One problem you are encountering is that reflection in c# can be very slow. You are calling record.GetType().GetProperty(dc.ToString()).GetValue(record) twice, and if I simply reduce the number of calls by 1, the time is reduced to around 706 ms:
foreach (DataColumn dc in dt.Columns)
{
var value = record.GetType().GetProperty(dc.ToString()).GetValue(record);
if (value != null)
{
dr[dc.ToString()] = value;
}
}
Demo #2 here.
However, we can do better by manufacturing a delegate in runtime. First, add the following utility methods that make use of the System.Linq.Expressions namespace:
public static partial class UtilExtensions
{
public static Func<TSource, object> CreatePropertyGetter<TSource>(PropertyInfo propertyInfo)
{
var parameter = Expression.Parameter(typeof(TSource), "obj");
var property = Expression.Property(parameter, propertyInfo);
var convert = Expression.Convert(property, typeof(object));
var lambda = Expression.Lambda(typeof(Func<TSource, object>), convert, parameter);
return (Func<TSource, object>)lambda.Compile();
}
public static ReadOnlyDictionary<string, Func<TSource, object>> PropertyGetters<TSource>() => PropertyExpressionsCache<TSource>.PropertyGetters;
static ReadOnlyDictionary<string, Func<TSource, object>> CreatePropertyGetters<TSource>() =>
typeof(TSource)
.GetSerializableProperties()
.ToDictionary(p => p.Name,
p => CreatePropertyGetter<TSource>(p))
.ToReadOnly();
static class PropertyExpressionsCache<TSource>
{
public static ReadOnlyDictionary<string, Func<TSource, object>> PropertyGetters { get; } = UtilExtensions.CreatePropertyGetters<TSource>();
}
public static ReadOnlyDictionary<TKey, TValue> ToReadOnly<TKey, TValue>(this IDictionary<TKey, TValue> dictionary) =>
new ReadOnlyDictionary<TKey, TValue>(dictionary ?? throw new ArgumentNullException());
}
And modify Load<TClass, TClassMap>() as follows:
public long Load<TClass, TClassMap>(string rFile, int dtbatchSize) where TClassMap : ClassMap<TClass>, new()
{
bool isBadRecord = false;
long badCount = 0;
long runningCount = 0;
long goodCount = 0;
long batchCount = 0;
var rconfig = CreateCsvConfiguration(
x =>
{
isBadRecord = true;
//ErrRecords.Add(x.RawRecord);
++badCount;
});
var loaderFileType = typeof(TClass);
// Create datatable using class as definition.
var dt = UtilExtensions.CreateDataTable(loaderFileType);
var properties = UtilExtensions.PropertyGetters<TClass>();
using (var reader = new StreamReader(rFile))
{
//reader.ReadLine(); FIXED - THIS SKIPPED THE FIRST LINE AND CAUSED A RECORD TO BE OMITTED.
using (var csv = new CsvReader(reader, rconfig))
{
csv.Context.RegisterClassMap<TClassMap>();
dt.BeginLoadData();
while (csv.Read())
{
isBadRecord = false;
//csvReadTimer.Start();
var record = csv.GetRecord<TClass>();
//csvReadTimer.Stop();
runningCount++;
if (!isBadRecord)
{
csvTableTimer.Start();
// Process row and populate datatable
DataRow dr = dt.NewRow();
foreach (var p in properties)
{
var value = p.Value(record);
if (value != null)
dr[p.Key] = value;
}
dt.Rows.Add(dr);
csvTableTimer.Stop();
goodCount++;
if (++batchCount >= dtbatchSize)
{
// Flush the data table
FlushTable(dt);
batchCount = 0;
}
}
}
dt.EndLoadData();
FlushTable(dt);
}
}
return goodCount;
}
The time will be further reduced, to roughly 404 ms. Demo fiddle #3 here.
I also tried using Delegate.CreateDelegate() instead of Expression:
public static partial class UtilExtensions
{
static Func<TSource, object> CreateTypedPropertyGetter<TSource, TValue>(PropertyInfo propertyInfo)
{
var typedFunc = (Func<TSource, TValue>)Delegate.CreateDelegate(typeof(Func<TSource, TValue>), propertyInfo.GetGetMethod());
return i => (object)typedFunc(i);
}
public static Func<TSource, object> CreatePropertyGetter<TSource>(PropertyInfo propertyInfo)
{
var typedCreator = typeof(UtilExtensions).GetMethod(nameof(CreateTypedPropertyGetter), BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic);
var concreteTypedCreator = typedCreator = typedCreator.MakeGenericMethod(typeof(TSource), propertyInfo.PropertyType);
return (Func<TSource, object>)concreteTypedCreator.Invoke(null, new object [] { propertyInfo });
}
public static ReadOnlyDictionary<string, Func<TSource, object>> PropertyGetters<TSource>() => PropertyExpressionsCache<TSource>.PropertyGetters;
static ReadOnlyDictionary<string, Func<TSource, object>> CreatePropertyGetters<TSource>() =>
typeof(TSource)
.GetSerializableProperties()
.ToDictionary(p => p.Name,
p => CreatePropertyGetter<TSource>(p))
.ToReadOnly();
static class PropertyExpressionsCache<TSource>
{
public static ReadOnlyDictionary<string, Func<TSource, object>> PropertyGetters { get; } = UtilExtensions.CreatePropertyGetters<TSource>();
}
public static ReadOnlyDictionary<TKey, TValue> ToReadOnly<TKey, TValue>(this IDictionary<TKey, TValue> dictionary) =>
new ReadOnlyDictionary<TKey, TValue>(dictionary ?? throw new ArgumentNullException());
}
And got roughly the same time, of 410 ms. Demo fiddle #4 here.
Notes:
The code in your question skips the first line of the CSV file, by calling reader.ReadLine();.
In my test harness this caused an incorrect number of records to be read, so I removed this line.
Rather than having a non-generic method that has a switch on the record type, I extracted a generic method that takes the record type and class map type as generic parameters. This makes delegate creation a little easier as it is no longer necessary to do runtime casting to the record type.
How can I Convert a List<Person> personList to DBDataReader ?
In the code that I have given below, I am trying to bulk insert personList. I have around 500k records, and the method WriteToServer expects a DBDataReader and I have a List<Person>. How can I convert List<Person> to DBDataReader
using (SqlBulkCopy bc= new SqlBulkCopy(constr)) {
bc.DestinationTableName = "MyPersonTable";
try
{
bc.WriteToServer(personList);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}\
\
**Person Model**
public int personId {get;set;} // Primarykey
public string personName {get;set;}
public int personAge {get;set;}
public DateTime personCreatedDate {get;set;}
you can use this one and after that with SqlBulkCopy insert all data to database:
IList<Name> list = new List<Name>();
list.Add(new Name{ Forename="Bert", Surname="Fred"});
list.Add(new Name { Forename = "John", Surname = "Smith" });
DataTable table = new DataTable();
table.Columns.Add("Forename");
table.Columns.Add("Surname");
foreach (Name item in list)
{
var row = table.NewRow();
row["Forename"] = item.Forename;
row["Surname"] = item.Surname;
table.Rows.Add(row);
}
Here is a generic way in creating a table.
There is another way of doing this too by creating a custom class that inherit from IDataReader if you are intressted in it feel free to tell.
Anyway have a look below, this is one way.
public static DataTable MakeTable(this List<object> o)
{
var data = new DataTable();
var props = o.FirstOrDefault()?.GetType().GetProperties();
if (props == null)
return data;
forEach(var p in props){
DataColumn c = new DataColumn();
c.DataType = p.PropertyType;
c.ColumnName = p.Name;
// c.AutoIncrement = true; // if this is a primaryKey
data.Columns.Add(c);
}
forEach(var item in o){
var row = data.NewRow();
forEach(var p in props){
row[p.Name] = p.GetValue(item);
}
data.Rows.Add(row);
}
return data;
}
// now all you need is to call MakeTable
using (SqlBulkCopy bc= new SqlBulkCopy(constr)) {
bc.DestinationTableName = "MyPersonTable";
try
{
bc.WriteToServer(personList.MakeTable());
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
Here is example of Generic of Bulk Insert
public async Task CreateBulkAsync(IEnumerable<T> entities, CancellationToken cancellationToken = default)
{
var options = SqlBulkCopyOptions.KeepIdentity | SqlBulkCopyOptions.CheckConstraints |
SqlBulkCopyOptions.KeepNulls;
var props = _context.Model.FindEntityType(typeof(T)).GetProperties();
// Берем connection и !не закрываем! это connection контекста, поэтому он сам этим управляет
if (_context.Database.GetDbConnection() is SqlConnection connection)
{
if (connection.State == ConnectionState.Closed)
{
await connection.OpenAsync(cancellationToken);
}
using var sqlBulkCopy = new SqlBulkCopy(connection, options, null);
{
sqlBulkCopy.DestinationTableName = $"dbo.{typeof(T).Name}";
foreach (var property in props)
{
sqlBulkCopy.ColumnMappings.Add(property.Name, property.Name);
}
var dt = entities.EntityToDataTable();
await sqlBulkCopy.WriteToServerAsync(dt, cancellationToken);
}
}
else
{
throw new ArgumentNullException(nameof(connection));
}
}
Here EntityToDataTable function code:
public static DataTable EntityToDataTable<T>(this IEnumerable<T> source)
{
var dataTable = new DataTable(typeof(T).Name);
var props = typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (var prop in props)
{
dataTable.Columns.Add(prop.Name, prop.PropertyType.Name.Contains("Nullable") ? typeof(string) : prop.PropertyType);
}
foreach (T item in source)
{
var values = new object[props.Length];
for (var i = 0; i < props.Length; i++)
{
values[i] = props[i].GetValue(item, null);
}
dataTable.Rows.Add(values);
}
return dataTable;
}
I am trying to create a extension method to export the class as data-table, in this method I want to give facility to user to export the property with different name in datatable, suppose property name in class is "LoginName" but user want to export it as "Login" in data-table, also user can specify multiple properties to rename.
for example following is the class
public class UserInfo
{
public int UserID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string LoginName { get; set; }
public int CompanyID { get; set; }
}
to export this class as data-table user will use extension method like this
UserInfo us = UserRep.GetUser("userID","Pass");
DataTable userDetails = null;
//Follwing is a pseudo code it could be different in possible manner
userDetails = us.ExportAsDataTable(u=> new {{u.LoginName,"Login"}, {u.CompanyID ,"Company"}});
//Or
userDetails = us.ExportAsDataTable(u=> new { Login = u.LoginName, Company = u.CompanyID});
following ExportAsDataTable metod I have created to do the functionality but unable to give correct expression to take the user input.
public static DataTable ExportAsDataTable<TSource, TProperty>(this TSource instance, Expression<Func<TSource, KeyValuePair<TProperty, string>>> renamePropertyMap)
{
DataTable dataTable = new DataTable();
//Doing export stuff here
return dataTable;
}
//Or
public static DataTable ConvertToDataTable<T>(this T instance, Expression<Func<T, object>> renamePropertyMap) where T : EntityBase
{
//Using this method I am able to get the new name of column from expression like this but not getting the original property name
string columnName = (renamePropertyMap.Body as NewExpression).Members[0].Name
/*note :- result in columnName is "Login" which is fine,
but I need to get orignal property name as well, that is
"LoginName", I am unable to get it from expression.*/
DataTable dataTable = new DataTable();
//Doing export stuff here
return dataTable;
}
I would do something like this instead:
public class FluentBuilder<T>
{
private readonly T _input;
private readonly Dictionary<string, string> _mappings = new Dictionary<string, string>();
public FluentBuilder(T input)
{
_input = input;
}
public FluentBuilder<T> Map(Expression<Func<T, object>> selector, string name)
{
MemberExpression member = selector.Body as MemberExpression;
if (member == null)
throw new ArgumentException(string.Format("Expression '{0}' refers to a method, not a property.", selector));
var propInfo = member.Member as PropertyInfo;
if (propInfo == null)
throw new ArgumentException(string.Format("Expression '{0}' refers to a field, not a property.", selector));
_mappings.Add(propInfo.Name, name);
return this;
}
private string GetName(PropertyInfo prop)
{
string map;
if (_mappings.TryGetValue(prop.Name, out map))
return map;
return prop.Name;
}
public DataTable ToDataTable(string tableName = null)
{
var result = new DataTable(tableName);
foreach (var prop in _input.GetType().GetProperties())
{
result.Columns.Add(GetName(prop));
}
var values = _input.GetType().GetProperties().Select(x => x.GetMethod.Invoke(_input, new object[0])).ToArray();
result.Rows.Add(values);
return result;
}
}
public static class FluentBuilderExtensions
{
public static FluentBuilder<T> SetupWith<T>(this T input)
{
return new FluentBuilder<T>(input);
}
}
class Program
{
public class UserInfo
{
public string MailAddress { get; set; }
public string Username { get; set; }
public string Password { get; set; }
}
static void Main(string[] args)
{
var userInfo = new UserInfo()
{
MailAddress = "foo#bar.com",
Username = "foouser",
Password = "barpassword"
};
var dt = userInfo.SetupWith()
.Map(x => x.MailAddress, "address")
.Map(x => x.Username, "user")
.ToDataTable();
}
}
MoreLINQ already provides ToDatatable() to convert an IEnumerable result to a DataTable. The method is available as part of the full NuGet package or as a source package that you can add to your project.
To generate a DataTable from a subset of UserInfo properties, use a Select before calling ToDataTable(), eg :
var table = myUserInfos.Select(us=>new {Login=us.LoginName, Company=us.CompanyID})
.ToDataTable();
If you only have one item and want to convert it to a single-row DataTable:
That's a very strange request. Why do you want to do that instead of eg binding the single object to the UI controls?
You can wrap it in an array eg:
var table = new[]{theUser}.Select(us=>new {Login=us.LoginName, Company=us.CompanyID})
.ToDataTable();
I used DataAnnotations and Reflection.
You need to add reference of missing libraries from following. I tested this code and it is working.
using System;
using System.Data;
using System.Linq;
using System.ComponentModel.DataAnnotations;
using System.Reflection;
class Program
{
static void Main(string[] args)
{
var user = new UserInfo();
var name = GetAttributeFrom<DisplayAttribute>(user, "LoginName").Name;
var dt = objToDataTable(user);
Console.ReadLine();
}
public static DataTable objToDataTable(UserInfo obj)
{
DataTable dt = new DataTable();
UserInfo objU = new UserInfo();
foreach (PropertyInfo info in typeof(UserInfo).GetProperties())
{
dt.Columns.Add(GetAttributeFrom<DisplayAttribute>(objU, info.Name).Name);
}
dt.AcceptChanges();
return dt;
}
public static T GetAttributeFrom<T>(object instance, string propertyName) where T : Attribute
{
var attrType = typeof(T);
var property = instance.GetType().GetProperty(propertyName);
return (T)property.GetCustomAttributes(attrType, false).First();
}
}
public class UserInfo
{
[Display(Name = "ID")]
public int UserID { get; set; }
[Display(Name = "FName")]
public string FirstName { get; set; }
[Display(Name = "LName")]
public string LastName { get; set; }
[Display(Name = "Login")]
public string LoginName { get; set; }
[Display(Name = "Company")]
public int CompanyID { get; set; }
}
This is how I build the extension method to convert the class as datatable
public static class EntityExtensions
{
private static readonly string expressionCannotBeNullMessage = "The expression cannot be null.";
private static readonly string invalidExpressionMessage = "Invalid expression.";
public static DataTable ConvertToDataTable<T>(this T instance, Expression<Func<T, object>> proprtiesToSkip = null, Expression<Func<T, object>> proprtiesToRename = null) where T : EntityBase
{
string columnName = "";
string orgPropName;
int counter = 0;
Dictionary<string, string> renameProperties = null;
MemberInfo newName = null;
NewExpression expression = null;
List<string> skipProps = null;
try
{
if (proprtiesToSkip != null )
{
if (proprtiesToSkip.Body is NewExpression)
{
skipProps = new List<string>();
expression = (proprtiesToSkip.Body as NewExpression);
foreach (var cExpression in expression.Arguments)
{
skipProps.Add(GetMemberName(cExpression));
}
}
else
{
throw new ArgumentException("Invalid expression supplied in proprtiesToSkip while converting class to datatable");
}
}
if (proprtiesToRename != null)
{
if (proprtiesToRename.Body is NewExpression)
{
renameProperties = new Dictionary<string, string>();
expression = (proprtiesToRename.Body as NewExpression);
foreach (var cExpression in expression.Arguments)
{
newName = expression.Members[counter];
orgPropName = GetMemberName(cExpression);
renameProperties.Add(orgPropName, newName.Name);
counter++;
}
}
else
{
throw new ArgumentException("Invalid expression supplied in proprtiesToRename while converting class to datatable");
}
}
var properties = instance.GetType().GetProperties().Where(o =>
{
return (skipProps != null && !skipProps.Contains(o.Name, StringComparer.OrdinalIgnoreCase) &&
(o.PropertyType != typeof(System.Data.DataTable) && o.PropertyType != typeof(System.Data.DataSet)));
}).ToArray();
DataTable dataTable = new DataTable();
foreach (PropertyInfo info in properties)
{
columnName = "";
if (renameProperties != null && renameProperties.ContainsKey(info.Name))
{
columnName = renameProperties[info.Name];
}
if (string.IsNullOrEmpty(columnName))
{
columnName = info.Name;
}
dataTable.Columns.Add(new DataColumn(columnName, Nullable.GetUnderlyingType(info.PropertyType) ?? info.PropertyType));
}
object[] values = new object[properties.Length];
for (int i = 0; i < properties.Length; i++)
{
values[i] = properties[i].GetValue(instance);
}
dataTable.Rows.Add(values);
return dataTable;
}
finally
{
renameProperties = null;
newName = null;
expression = null;
skipProps = null;
}
}
private static string GetMemberName(Expression expression)
{
if (expression == null)
{
throw new ArgumentException(expressionCannotBeNullMessage);
}
if (expression is MemberExpression)
{
// Reference type property or field
var memberExpression = (MemberExpression)expression;
return memberExpression.Member.Name;
}
throw new ArgumentException(invalidExpressionMessage);
}
}
Calling it like this
class Program
{
static void Main(string[] args)
{
CancelReason ad = new CancelReason();
ad.BusinessLineID = 1;
ad.CancelReasonCodeID = 2;
ad.CancelRefund = "C";
ad.CompanyID = 0;
ad.DBOperation = 1;
ad.Description = "test";
ad.IsActive = "Y";
ad.ReasonCode = "TestCode";
ad.UpdateStamp = new byte[] { 1, 2, 3, 4 };
DataTable dt = ad.ConvertToDataTable(s => new { s.DBOperation, s.BusinessLineID, s.LoggedInUser }, r => new { Refund = r.CancelRefund, Company = r.CompanyID });
}
}
I want to fetch my result and convert it to Dataset. My code is :
var res =
from i in dbconnect.tblManageDates
where i.internationalCode == _international
select new
{
i.companyId,
i.endDate,
i.startDate,
i.dateId
};
_result = (DataSet)res;
When I fetch it to Dataset I get this error:
Unable to cast object of type
'System.Data.Linq.DataQuery1[<>f__AnonymousType04[System.String,System.String,System.String,System.Int32]]'
to type 'System.Data.DataSet'.
Try to use such extension:
public static class Extesions
{
public static DataSet ToDataSet<T>(this IList<T> list)
{
Type elementType = typeof(T);
DataSet ds = new DataSet();
DataTable t = new DataTable();
ds.Tables.Add(t);
//add a column to table for each public property on T
foreach (var propInfo in elementType.GetProperties())
{
Type ColType = Nullable.GetUnderlyingType(propInfo.PropertyType) ?? propInfo.PropertyType;
t.Columns.Add(propInfo.Name, ColType);
}
//go through each property on T and add each value to the table
foreach (T item in list)
{
DataRow row = t.NewRow();
foreach (var propInfo in elementType.GetProperties())
{
row[propInfo.Name] = propInfo.GetValue(item, null) ?? DBNull.Value;
}
t.Rows.Add(row);
}
return ds;
}
}
Usage:
public class Entity
{
public int CompanyId { get; set; }
public DateTime EndDate { get; set; }
public DateTime StartDate { get; set; }
public int DateId { get; set; }
}
// ...
List<Entity> res = dbconnect.tblManageDates.Where(i => i.internationalCode == _international).Select(i => new Entity
{
CompanyId = i.companyId,
EndDate = i.endDate,
StartDate = i.startDate,
DateId = i.dateId
}).ToList();
DataSet dataSet = res.ToDataSet();
I'm getting values from another data table as input to list. Now i need to save those list values into another DataTable.
List:
List<DataRow> list = slectedFieldsTable.AsEnumerable().ToList();
foreach (DataRow dr in slectedFieldsTable.Rows)
{
list.Add(dr);
}
New Data table :
DataRow newRow = tempTable.NewRow();
newRow["Field Name"] = fieldLabel;
newRow["Field Type"] = fieldType;
for(int gg =0 ; gg<list.Count; gg++)
{
tempTable.Rows.Add(????);
}
I'm stuck here in adding up rows in to new data table.
public static DataTable ToDataTable<T>(List<T> items)
{
DataTable dataTable = new DataTable(typeof(T).Name);
//Get all the properties
PropertyInfo[] Props = typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (PropertyInfo prop in Props)
{
//Setting column names as Property names
dataTable.Columns.Add(prop.Name);
}
foreach (T item in items)
{
var values = new object[Props.Length];
for (int i = 0; i < Props.Length; i++)
{
//inserting property values to datatable rows
values[i] = Props[i].GetValue(item, null);
}
dataTable.Rows.Add(values);
}
//put a breakpoint here and check datatable
return dataTable;
}
Variable declare:
DataTable tempTable = new DataTable();
DataTable slectedFieldsTable = new DataTable();
DataRow newRow;
List<object> list = new List<object>();
Add Column in DataTable:
slectedFieldsTable = new DataTable();
slectedFieldsTable.Columns.Add("Field Name");
slectedFieldsTable.Columns.Add("Field Type");
Add Value in DataTable:
slectedFieldsTable.Rows.Add("1", "AAA");
slectedFieldsTable.Rows.Add("2", "BBB");
slectedFieldsTable.Rows.Add("3", "CCC");
Convert DataTable to List:
foreach (DataRow dr in slectedFieldsTable.Rows)
{
list.Add(dr);
}
Add Column in another DataTable:
tempTable.Columns.Add("Field Name", typeof(string));
tempTable.Columns.Add("Field Type", typeof(string));
Convert List to dataTable:
foreach(DataRow dr in list)
{
newRow = tempTable.NewRow();
newRow["Field Name"] = dr.ItemArray[0].ToString();
newRow["Field Type"] = dr.ItemArray[1].ToString();
tempTable.Rows.Add(newRow);
tempTable.AcceptChanges();
}
use CopyToDataTable() method. CopyToDataTable
IEnumerable<DataRow> query = TempselectedFieldsTable.AsEnumerable().ToList();
// Create a table from the query.
DataTable boundTable = query.CopyToDataTable<DataRow>();
The answer providing the ToDataTable is a very nice start but it is missing some key elements. Namely, it ignores that List item properties may:
...be marked ReadOnly
...use the DisplayName attribute
...have a DefaultValue which the DataColumn should know about
...be Nullable
...be marked BrowsableAttribute(false)
The following is an extension method to return a DataTable and either accounts for the above or provides the means for your code to apply them. It also uses an Interface to get the values from the class object rather than Reflection.
public static DataTable ToDataTable<T>(this IList<T> lst, bool includeAll = true)
{
DataTable dt = new DataTable();
DataColumn dc;
PropertyDescriptor pd;
bool Browsable;
PropertyDescriptorCollection propCol = TypeDescriptor.GetProperties(typeof(T));
for (int n = 0; n < propCol.Count; n++)
{
pd = propCol[n];
Type propT = pd.PropertyType;
dc = new DataColumn(pd.Name);
// if Nullable, get underlying type
// the first test may not be needed
if (propT.IsGenericType && Nullable.GetUnderlyingType(propT) != null )
{
propT = Nullable.GetUnderlyingType(propT);
dc.DataType = propT;
dc.AllowDBNull = true;
}
else
{
dc.DataType = propT;
dc.AllowDBNull = false;
}
// is it readonly?
if (pd.Attributes[typeof(ReadOnlyAttribute)] != null)
{
dc.ReadOnly = ((ReadOnlyAttribute)pd.
Attributes[typeof(ReadOnlyAttribute)]).IsReadOnly;
}
// DefaultValue ...
if (pd.Attributes[typeof(DefaultValueAttribute)] != null)
{
dc.DefaultValue = ((DefaultValueAttribute)pd.
Attributes[typeof(DefaultValueAttribute)]).Value;
}
// caption / display name
dc.ExtendedProperties.Add("DisplayName", dc.Caption);
if (pd.Attributes[typeof(DisplayNameAttribute)] != null)
{
// these are usually present but blank
string theName = ((DisplayNameAttribute)pd.
Attributes[typeof(DisplayNameAttribute)]).DisplayName;
dc.Caption = string.IsNullOrEmpty(theName) ? dc.Caption : theName;
// DGV doesnt use Caption...save for later
dc.ExtendedProperties["DisplayName"] = dc.Caption;
}
Browsable = true;
dc.ExtendedProperties.Add("Browsable", Browsable);
var foo = pd.Attributes[typeof(BrowsableAttribute)];
if (pd.Attributes[typeof(BrowsableAttribute)] != null)
{
Browsable = ((BrowsableAttribute)pd.Attributes[typeof(BrowsableAttribute)]).Browsable;
// no such thing as a NonBrowsable DataColumn
dc.ExtendedProperties["Browsable"] = Browsable;
}
// ToDo: add support for custom attributes
if (includeAll || Browsable)
{
dt.Columns.Add(dc);
}
}
// the lst could be empty such as creating a typed table
if (lst.Count == 0) return dt;
if (lst[0] is IDataValuesProvider)
{
IDataValuesProvider dvp;
// copy the data - let the class do the work
foreach (T item in lst)
{
dvp = (IDataValuesProvider)item;
dt.Rows.Add(dvp.GetDataValues(includeAll).ToArray());
}
}
else
{
List<object> values;
foreach (T item in lst)
{
values = new List<object>();
// only Browsable columns added
for (int n = 0; n < dt.Columns.Count; n++)
{
values.Add(propCol[dt.Columns[n].ColumnName].GetValue(item));
}
dt.Rows.Add(values.ToArray());
}
}
return dt;
}
The method allows you to specify whether columns for non Browsable properties should be added to the DataTable. Rather than hiding the columns later, you can omit them entirely if you want.
An interface proves the means to get the data values from collection members in order (as an alternative to a reflection loop):
public interface IDataValuesProvider
{
IEnumerable<object> GetDataValues(bool includeAll);
}
... on the class:
public class StockItem : IDataValuesProvider
{
public int Id { get; set; }
public string ItemName {get; set;}
[Browsable(false), DisplayName("Ignore")]
public string propA {get; set;}
[ReadOnly(true)]
public string Zone { get; set; }
public string Size {get; set;}
[DisplayName("Nullable")]
public int? Foo { get; set; }
public int OnHand {get; set;}
public string ProdCode {get; set;}
[Browsable(false)]
public string propB { get; set; }
public DateTime ItemDate {get; set;}
// IDataValuesProvider implementation
public IEnumerable<object> GetDataValues(bool IncludeAll)
{
List<object> values = new List<object>();
values.AddRange(new object[] {Id, ItemName });
if (IncludeAll) values.Add(propA);
values.AddRange(new object[] { Zone, Size, Foo, OnHand, ProdCode });
if (IncludeAll) values.Add(propB);
values.Add(ItemDate);
return values;
}
}
Add the data values in the same order as they are listed in your class; be sure to update it when you add properties. The reflection version is still there so you can do it either way.
Finally, there are a few common Attributes which do not have a related DataColumn property. The method stores these for you as ExtendedProperties allowing you to easily apply them to the DGV:
var dtX = someData.ToDataTable();
dgvB.SuspendLayout();
dgvB.DataSource = dtX;
// process extended props
foreach (DataColumn dc in dtX.Columns)
{
// no need to test, the code adds them everytime
//if (dc.ExtendedProperties.ContainsKey("DisplayName"))
//{
dgvB.Columns[dc.ColumnName].HeaderText = dc.ExtendedProperties["DisplayName"].ToString();
//}
//if (dc.ExtendedProperties.ContainsKey("Browsable"))
//{
dgvB.Columns[dc.ColumnName].Visible = (bool)dc.ExtendedProperties["Browsable"];
//}
}
dgvB.ResumeLayout();
Results using a list of the class shown above:
Both OnHand and Foo show the DisplayName and both PropA and PropB are hidden. Most importantly, columns created for ReadOnly and Nullable properties act accordingly.
Try this:
foreach (DataRow dr in list)
{
tempTable.Rows.Add(dr);
}