We Keep Coding

How to split datatable row in to multiple rows based on character split in .net

I have a datatable as shown below
Dosage Drug Patient
-----------------------------------
25 Indocin David
50 Enebrel,Crozine Sam
10 Hydralazine Christoff
21 Combivent Janet
100 Dilantin Melanie
which should be converted in to as shown below based on comma split
Dosage Drug Patient
------------------------------
25 Indocin David
50 Enebrel Sam
50 Crozine Sam
10 Hydralazine Christoff
21 Combivent Janet
100 Dilantin Melanie
I did following code which is not giving the expected result. Can somebody suggest an answer?
private static void ProcessDatatable(DataTable dt)
{
DataTable dtnew = new DataTable();
IEnumerable<string[]> allRowValues = dt.AsEnumerable()
.Select(r => r.Field<string>(1).Split(','));
dtnew = allRowValues.ToDataTable();
}
Extension method:
public static DataTable ToDataTable<T>(this IEnumerable<T> collection, string tableName)
{
DataTable tbl = ToDataTable(collection);
tbl.TableName = tableName;
return tbl;
}
public static DataTable ToDataTable<T>(this IEnumerable<T> collection)
{
DataTable dt = new DataTable();
Type t = typeof(T);
PropertyInfo[] pia = t.GetProperties();
// Create the columns in the DataTable
foreach (PropertyInfo pi in pia)
{
dt.Columns.Add(pi.Name, pi.PropertyType);
}
// Populate the table
foreach (T item in collection)
{
DataRow dr = dt.NewRow();
dr.BeginEdit();
foreach (PropertyInfo pi in pia)
{
dr[pi.Name] = pi.GetValue(item, null);
}
dr.EndEdit();
dt.Rows.Add(dr);
}
return dt;
}

What you think you want is this:
IEnumerable<object[]> allRowValues = dataTable.AsEnumerable()
.SelectMany(dataRow =>
dataRow.Field<string>(1).Split(',').Select(drug => new[] { dataRow[0], drug, dataRow[2] }));
But what you really want is this:
IEnumerable<Record> allRowValues = dataTable.AsEnumerable()
.Select(dataRow => new Record(dataRow))
.SelectMany(record => record.SplitDrugs());
// ...
public class Record
{
public int Dosage { get; }
public string Drug { get; }
public string Patient { get; }
public Record(int dosage, string drug, string patient)
{
Dosage = dosage;
Drug = drug;
Patient = patient;
}
public Record(DataRow dataRow)
: this((int)dataRow["Dosage"], (string)dataRow["Drug"], (string)dataRow["Patient"])
{
}
public IEnumerable<Record> SplitDrugs()
{
return Drug.Split(',').Select(drug => new Record(Dosage, drug, Patient));
}
}
Short explanation: with a fancy LINQ you are trying to solve just too many problems of the world liek extracting information from data table, processing rows one by one, applying bussiness logic and merging the result to new data table. That is a good way how to write error-prone, unreadable, untestable, unstable and unmaintanable code.
Incomplete list of people who will eventually thank you for chosing the second option:
your future self
your teammates
your code reviewer
unit test writer
end users
your teacher (if assignment)
SO community
While I'm at it I'll save you some time debugging your transformation of allRowValues (which in your case is of type IEnumerable<string[]>) back to DataTable. If you think it will contain 3 columns, then you're wrong. Instead, it will contain columns like Length, LongLength, Rank, ... Have a look at properties Array class to figure out why.
Edit
OP has refined the original intent in a comment under another answer.
... , but i just posted a prototype of datatable,infact actually 180 columns are there.DO i need to add all 180 columns manually in newRow.ItemArray, when there is a split of comma seperated values???Any easier way?
Yes, there is an easier way. Involving generics you can expand the usage beyond this limited use case:
// extension method
public static DataTable ExpandColumn<T>(this DataTable dataTable, string columnName,
Func<T, IEnumerable<T>> expandField)
{
var clonedDataTable = dataTable.Clone();
var columnIndex = dataTable.Columns.IndexOf(columnName);
var column = dataTable.Columns[columnIndex];
foreach (var dataRow in dataTable.AsEnumerable())
{
var valueToExpand = dataRow.Field<T>(column);
var expandedValues = expandField(valueToExpand);
var originalValues = dataRow.ItemArray;
foreach (var value in expandedValues)
{
originalValues[columnIndex] = value;
clonedDataTable.Rows.Add(originalValues);
}
}
return clonedDataTable;
}
// usage
var dataTableNew = dataTable.ExpandColumn<string>("Drug", drug => drug.Split(','));
The above extension method clones DataTable instance by copying original rows and expands values in specified column by applying expandField function for each value.
I'd still like you to learn lesson from what I wrove above the edit and think twice about your design.

I am only barely comfortable with C# so I had to do this the old fashioned way but it works.
public partial class Form1 : Form
{
private DataTable dt;
private DataTable dtExpanded;
public Form1()
{
InitializeComponent();
LoadTable();
LoadExpandedTable();
}
//Dosage Drug Patient
private void LoadTable()
{
dt = new DataTable();
using (SqlConnection cn = new SqlConnection("Your connection string"))
{
using (SqlCommand cmd = new SqlCommand("Select * From DrugDoses", cn))
{
cn.Open();
dt.Load(cmd.ExecuteReader());
}
}
dataGridView1.DataSource = dt;
}
private void LoadExpandedTable()
{
dtExpanded = new DataTable();
dtExpanded.Columns.Add("Dose");
dtExpanded.Columns.Add("Drug");
dtExpanded.Columns.Add("Patient");
foreach (DataRow r in dt.Rows)
{
string s = (string)r["Drug"];
if(s.Contains(","))
{
string[] splitName = s.Split(',');
foreach (string drug in splitName)
{
DataRow newRow = dtExpanded.NewRow();
newRow.ItemArray = new Object[] { r["Dosage"], drug , r["Patient"]};
dtExpanded.Rows.Add(newRow);
}
}
else
{
dtExpanded.Rows.Add(r.ItemArray);
}
}
dataGridView2.DataSource = dtExpanded;
}
}

OleDbDataReader skips first record

I have the following code :
OleDbDataReader xlsReader =
new OleDbCommand("Select * from [" +spreadSheetName + "]", xlsFileConnection).
ExecuteReader();
In the spreadSheetName parameter i keep my file name.
The connection string for xlsFileConnection was set as
"Provider=Microsoft.Jet.OLEDB.4.0;
Data Source='<directory path>';
Extended Properties='text; HDR=No; FMT=Delimited'"
When i start to execute while (xlsReader.Read()) i take a row #2 but not #1 from data source.
The first suggestion was that HDR parameter has invalid value but it seems it's ok.

There are better and easier ways to reading xlsx files, if I were you I would grab closedXML from nuget and this code to read your excel file into a data table
public void ProcessExcel(string fileName)
{
_dt = ImportSheet(fileName);
dgContacts.ItemsSource = _dt.DefaultView;
}
public static DataTable ImportSheet(string fileName)
{
var datatable = new DataTable();
var workbook = new XLWorkbook(fileName);
var xlWorksheet = workbook.Worksheet(1);
var range = xlWorksheet.Range(xlWorksheet.FirstCellUsed(), xlWorksheet.LastCellUsed());
var col = range.ColumnCount();
var row = range.RowCount();
datatable.Clear();
for (var i = 1; i <= col; i++)
{
var column = xlWorksheet.Cell(1, i);
datatable.Columns.Add(column.Value.ToString());
}
var firstHeadRow = 0;
foreach (var item in range.Rows())
{
if (firstHeadRow != 0)
{
var array = new object[col];
for (var y = 1; y <= col; y++)
{
array[y - 1] = item.Cell(y).Value;
}
datatable.Rows.Add(array);
}
firstHeadRow++;
}
return datatable;
}
The grab the data out of your datatable as you need.
This is live and working code, so you just need to copy and paste

Getting exception on retriving data for the second time using LINQ in c#

Sir I have filled my dataset with linq as
public void FillDataSet(DataSet ds1,int Id)
{
try
{
var y = from ins in cstmrDC.customers_rd(Id) select ins;
var z = from ins in cstmrDC.customersCntcts_rd(Id) select ins;
DataTable dtCst = new DataTable("dtCstmr");
dtCst.Columns.Add("cst_Id");
dtCst.Columns.Add("cst_Name");
dtCst.Columns.Add("cst_SName");
dtCst.Columns.Add("cst_AdLn1");
DataTable dtDtls = new DataTable("dtDtails");
dtDtls.Columns.Add("cst_SrlNo");
dtDtls.Columns.Add("cst_CntName");
dtDtls.Columns.Add("cst_cntDsgn");
foreach (var dtbl in y)
{
DataRow dr;
dr = dtCst.NewRow();
dr[0] = dtbl.cust_Id;
dr[1] = dtbl.cust_Name;
dr[2] = dtbl.cust_Sname;
dr[3] = dtbl.cust_Adrsln1;
dtCst.Rows.Add(dr);
}
foreach (var dtbl in z)
{
DataRow drDtls;
drDtls = dtDtls.NewRow();
drDtls[0] = dtbl.cust_Slno;
drDtls[1] = dtbl.cust_Cntctnm;
drDtls[2] = dtbl.cust_Cntctdesig;
dtDtls.Rows.Add(drDtls);
}
ds1.Tables.Add(dtCst);
ds1.Tables.Add(dtDtls);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
and the Id is passing from another class as
intId = int.Parse(txtSearch.Text);
cstCls.FillDataSet(ds1w, intId);
from that dataset iam fillimg my textbox controllers and giving theDataSource to the dataGridView as
dgvCustInfo.DataSource = ds1w.Tables["dtDtails"];
In this way if I searched 1st time with integer id 1055 meanse the exact result is comming from database. At the same time If I gave the another integer Id as 1066 meance Iam getting exception as DataTable named 'dtCstmr' already belongs to this DataSet .. Soo how can I solve the problem.

You can check if a table is already contained in a DataSet with Contains:
if(!ds1.Tables.Contains(dtCst.TableName))
ds1.Tables.Add(dtCst);
if(!ds1.Tables.Contains(dtDtls.TableName))
ds1.Tables.Add(dtDtls);
However, as Raphael has mentioned this would not refresh the table in the DataSet. So an easy way would be to remove the old table and add the new:
if(ds1.Tables.Contains(dtCst.TableName))
ds1.Tables.Remove(dtCst.TableName);
if(ds1.Tables.Contains(dtDtls.TableName))
ds1.Tables.Remove(dtDtls.TableName);
ds1.Tables.Add(dtCst);
ds1.Tables.Add(dtDtls);

It's quite a bad idea to create and populate in the same method.
Your code is really confusing.
Create another method :
public void CreateTables(DataSet ds1) {
var dtCst = new DataTable("dtCstmr");
dtCst.Columns.Add("cst_Id");
dtCst.Columns.Add("cst_Name");
dtCst.Columns.Add("cst_SName");
dtCst.Columns.Add("cst_AdLn1");
var dtDtls = new DataTable("dtDtails");
dtDtls.Columns.Add("cst_SrlNo");
dtDtls.Columns.Add("cst_CntName");
dtDtls.Columns.Add("cst_cntDsgn");
ds1.Tables.Add(dtCst);
ds1.Tables.Add(dtDtls);
}
public void FillDataSet(DataSet ds1,int Id)
{
try
{
var y = from ins in cstmrDC.customers_rd(Id) select ins;
var z = from ins in cstmrDC.customersCntcts_rd(Id) select ins;
var dtCst = ds1.Tables["dtCstmr"];
var dtDtls = ds1.Tables["dtDtails"];
dtCst.Clear();
dtDtls.Clear();
foreach (var dtbl in y)
{
DataRow dr;
dr = dtCst.NewRow();
dr[0] = dtbl.cust_Id;
dr[1] = dtbl.cust_Name;
dr[2] = dtbl.cust_Sname;
dr[3] = dtbl.cust_Adrsln1;
dtCst.Rows.Add(dr);
}
foreach (var dtbl in z)
{
DataRow drDtls;
drDtls = dtDtls.NewRow();
drDtls[0] = dtbl.cust_Slno;
drDtls[1] = dtbl.cust_Cntctnm;
drDtls[2] = dtbl.cust_Cntctdesig;
dtDtls.Rows.Add(drDtls);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
Method CreateTables should be called only one time, and FillDataSet as many times as needed.

How to convert DataTable to class Object?

I have already developed an application which returns DataTable everywhere.
Now my client wants to convert (use some part using service stack), so I need to return DTO (objects) in my application.
I don't want to change my existing stored procedures or even not want to use LINQ as much as possible (I am not too much aware with LINQ).
For small functionality, I can use Linq no issue.
My question is: how can I change my DataTable to objects of that class?
The sample code is below:
string s = DateTime.Now.ToString();
DataTable dt = new DataTable();
dt.Columns.Add("id");
dt.Columns.Add("name");
for (int i = 0; i < 5000000; i++)
{
DataRow dr = dt.NewRow();
dr["id"] = i.ToString();
dr["name"] = "name" + i.ToString();
dt.Rows.Add(dr);
dt.AcceptChanges();
}
List<Class1> clslist = new List<Class1>();
for (int i = 0; i < dt.Rows.Count; i++)
{
Class1 cls = new Class1();
cls.id = dt.Rows[i]["id"].ToString();
cls.name = dt.Rows[i]["name"].ToString();
clslist.Add(cls);
}
Response.Write(s);
Response.Write("<br>");
Response.Write(DateTime.Now.ToString());
I know, the above method is time-consuming, and I am trying to find an alternate solution.
Is there any alternative way (I guess, LINQ to DataTable) by which it directly converts the rows of tables to List<Class1>?
So that I can return objects in my service stack and go ahead.

Initialize DataTable:
DataTable dt = new DataTable();
dt.Columns.Add("id", typeof(String));
dt.Columns.Add("name", typeof(String));
for (int i = 0; i < 5; i++)
{
string index = i.ToString();
dt.Rows.Add(new object[] { index, "name" + index });
}
Query itself:
IList<Class1> items = dt.AsEnumerable().Select(row =>
new Class1
{
id = row.Field<string>("id"),
name = row.Field<string>("name")
}).ToList();

Amit, I have used one way to achieve this with less coding and more efficient way.
but it uses Linq.
I posted it here because maybe the answer helps other SO.
Below DAL code converts datatable object to List of YourViewModel and it's easy to understand.
public static class DAL
{
public static string connectionString = ConfigurationManager.ConnectionStrings["YourWebConfigConnection"].ConnectionString;
// function that creates a list of an object from the given data table
public static List<T> CreateListFromTable<T>(DataTable tbl) where T : new()
{
// define return list
List<T> lst = new List<T>();
// go through each row
foreach (DataRow r in tbl.Rows)
{
// add to the list
lst.Add(CreateItemFromRow<T>(r));
}
// return the list
return lst;
}
// function that creates an object from the given data row
public static T CreateItemFromRow<T>(DataRow row) where T : new()
{
// create a new object
T item = new T();
// set the item
SetItemFromRow(item, row);
// return
return item;
}
public static void SetItemFromRow<T>(T item, DataRow row) where T : new()
{
// go through each column
foreach (DataColumn c in row.Table.Columns)
{
// find the property for the column
PropertyInfo p = item.GetType().GetProperty(c.ColumnName);
// if exists, set the value
if (p != null && row[c] != DBNull.Value)
{
p.SetValue(item, row[c], null);
}
}
}
//call stored procedure to get data.
public static DataSet GetRecordWithExtendedTimeOut(string SPName, params SqlParameter[] SqlPrms)
{
DataSet ds = new DataSet();
SqlCommand cmd = new SqlCommand();
SqlDataAdapter da = new SqlDataAdapter();
SqlConnection con = new SqlConnection(connectionString);
try
{
cmd = new SqlCommand(SPName, con);
cmd.Parameters.AddRange(SqlPrms);
cmd.CommandTimeout = 240;
cmd.CommandType = CommandType.StoredProcedure;
da.SelectCommand = cmd;
da.Fill(ds);
}
catch (Exception ex)
{
return ex;
}
return ds;
}
}
Now, The way to pass and call method is below.
DataSet ds = DAL.GetRecordWithExtendedTimeOut("ProcedureName");
List<YourViewModel> model = new List<YourViewModel>();
if (ds != null)
{
//Pass datatable from dataset to our DAL Method.
model = DAL.CreateListFromTable<YourViewModel>(ds.Tables[0]);
}
Till the date, for many of my applications, I found this as the best structure to get data.

Was looking at this and realized: it's from one type of object to another; basicaclly we're trying to do proper reflection.
There are proper ways to construct the relationship between different fields but give the class definition is done, it can be easily done by Newtonsoft.Json
Process: DataSet/DataTable (Serialize) ==> Json (Deserialize) ==> Target Object List
In this example as the OP, simply do:
string serializeddt = JsonConvert.SerializeObject(dt, Formatting.Indented);
Now the DataTable is serialized into a plain string.
Then do this:
List<Class1> clslist = JsonConvert.DeserializeObject<List<Class1>>(serialized, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });
Now you should have the List with all the DataTable rows as individual objects.

It is Vb.Net version:
Public Class Test
Public Property id As Integer
Public Property name As String
Public Property address As String
Public Property createdDate As Date
End Class
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim x As Date = Now
Debug.WriteLine("Begin: " & DateDiff(DateInterval.Second, x, Now) & "-" & Now)
Dim dt As New DataTable
dt.Columns.Add("id")
dt.Columns.Add("name")
dt.Columns.Add("address")
dt.Columns.Add("createdDate")
For i As Integer = 0 To 100000
dt.Rows.Add(i, "name - " & i, "address - " & i, DateAdd(DateInterval.Second, i, Now))
Next
Debug.WriteLine("Datatable created: " & DateDiff(DateInterval.Second, x, Now) & "-" & Now)
Dim items As IList(Of Test) = dt.AsEnumerable().[Select](Function(row) New _
Test With {
.id = row.Field(Of String)("id"),
.name = row.Field(Of String)("name"),
.address = row.Field(Of String)("address"),
.createdDate = row.Field(Of String)("createdDate")
}).ToList()
Debug.WriteLine("List created: " & DateDiff(DateInterval.Second, x, Now) & "-" & Now)
Debug.WriteLine("Complated")
End Sub

Is it very expensive to do this by json convert? But at least you have a 2 line solution and its generic. It does not matter eather if your datatable contains more or less fields than the object class:
Dim sSql = $"SELECT '{jobID}' AS ConfigNo, 'MainSettings' AS ParamName, VarNm AS ParamFieldName, 1 AS ParamSetId, Val1 AS ParamValue FROM StrSVar WHERE NmSp = '{sAppName} Params {jobID}'"
Dim dtParameters As DataTable = DBLib.GetDatabaseData(sSql)
Dim paramListObject As New List(Of ParameterListModel)()
If (Not dtParameters Is Nothing And dtParameters.Rows.Count > 0) Then
Dim json = Newtonsoft.Json.JsonConvert.SerializeObject(dtParameters).ToString()
paramListObject = Newtonsoft.Json.JsonConvert.DeserializeObject(Of List(Of ParameterListModel))(json)
End If

Can you get the column names from a SqlDataReader?

After connecting to the database, can I get the name of all the columns that were returned in my SqlDataReader?

var reader = cmd.ExecuteReader();
var columns = new List<string>();
for(int i=0;i<reader.FieldCount;i++)
{
columns.Add(reader.GetName(i));
}
or
var columns = Enumerable.Range(0, reader.FieldCount).Select(reader.GetName).ToList();

There is a GetName function on the SqlDataReader which accepts the column index and returns the name of the column.
Conversely, there is a GetOrdinal which takes in a column name and returns the column index.

You can get the column names from a DataReader.
Here is the important part:
for (int col = 0; col < SqlReader.FieldCount; col++)
{
Console.Write(SqlReader.GetName(col).ToString()); // Gets the column name
Console.Write(SqlReader.GetFieldType(col).ToString()); // Gets the column type
Console.Write(SqlReader.GetDataTypeName(col).ToString()); // Gets the column database type
}

Already mentioned. Just a LINQ answer:
var columns = reader.GetSchemaTable().Rows
.Cast<DataRow>()
.Select(r => (string)r["ColumnName"])
.ToList();
//Or
var columns = Enumerable.Range(0, reader.FieldCount)
.Select(reader.GetName)
.ToList();
The second one is cleaner and much faster. Even if you cache GetSchemaTable in the first approach, the querying is going to be very slow.

If you want the column names only, you can do:
List<string> columns = new List<string>();
using (SqlDataReader reader = cmd.ExecuteReader(CommandBehavior.SchemaOnly))
{
DataTable dt = reader.GetSchemaTable();
foreach (DataRow row in dt.Rows)
{
columns.Add(row.Field<String>("ColumnName"));
}
}
But if you only need one row, I like my AdoHelper addition. This addition is great if you have a single line query and you don't want to deal with data table in you code. It's returning a case insensitive dictionary of column names and values.
public static Dictionary<string, string> ExecuteCaseInsensitiveDictionary(string query, string connectionString, Dictionary<string, string> queryParams = null)
{
Dictionary<string, string> CaseInsensitiveDictionary = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
try
{
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
using (SqlCommand cmd = new SqlCommand())
{
cmd.Connection = conn;
cmd.CommandType = CommandType.Text;
cmd.CommandText = query;
// Add the parameters for the SelectCommand.
if (queryParams != null)
foreach (var param in queryParams)
cmd.Parameters.AddWithValue(param.Key, param.Value);
using (SqlDataReader reader = cmd.ExecuteReader())
{
DataTable dt = new DataTable();
dt.Load(reader);
foreach (DataRow row in dt.Rows)
{
foreach (DataColumn column in dt.Columns)
{
CaseInsensitiveDictionary.Add(column.ColumnName, row[column].ToString());
}
}
}
}
conn.Close();
}
}
catch (Exception ex)
{
throw ex;
}
return CaseInsensitiveDictionary;
}

Use an extension method:
public static List<string> ColumnList(this IDataReader dataReader)
{
var columns = new List<string>();
for (int i = 0; i < dataReader.FieldCount; i++)
{
columns.Add(dataReader.GetName(i));
}
return columns;
}

For me, I would write an extension method like this:
public static string[] GetFieldNames(this SqlDataReader reader)
{
return Enumerable.Range(0, reader.FieldCount).Select(x => reader.GetName(x)).ToArray();
}

I use the GetSchemaTable method, which is exposed via the IDataReader interface.

You sure can.
protected void GetColumNames_DataReader()
{
System.Data.SqlClient.SqlConnection SqlCon = new System.Data.SqlClient.SqlConnection("server=localhost;database=northwind;trusted_connection=true");
System.Data.SqlClient.SqlCommand SqlCmd = new System.Data.SqlClient.SqlCommand("SELECT * FROM Products", SqlCon);
SqlCon.Open();
System.Data.SqlClient.SqlDataReader SqlReader = SqlCmd.ExecuteReader();
System.Int32 _columncount = SqlReader.FieldCount;
System.Web.HttpContext.Current.Response.Write("SqlDataReader Columns");
System.Web.HttpContext.Current.Response.Write(" ");
for ( System.Int32 iCol = 0; iCol < _columncount; iCol ++ )
{
System.Web.HttpContext.Current.Response.Write("Column " + iCol.ToString() + ": ");
System.Web.HttpContext.Current.Response.Write(SqlReader.GetName( iCol ).ToString());
System.Web.HttpContext.Current.Response.Write(" ");
}
}
This is originally from: http://www.dotnetjunkies.ddj.com/Article/B82A22D1-8437-4C7A-B6AA-C6C9BE9DB8A6.dcik

It is easier to achieve it in SQL
var columnsList = dbContext.Database.SqlQuery<string>("SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'SCHEMA_OF_YOUE_TABLE' AND TABLE_NAME = 'YOUR_TABLE_NAME'").ToList();