I have a DataTable which has 10 columns, and after I add some rows in it, I want to add it to the database using SqlBulkCopy.
My problem is that in the DataTable I have strings and some fields in the database are decimal and date respectively. Is there a way to convert them to the desired data types?
When programmatically adding columns to your DataTable, you can specify type:-
DataTable dt = new DataTable();
dt.Columns.Add("FieldName", typeof(System.Decimal));
These will be honored when you perform the bulk copy.
EDIT
In response to user comment.
When populating the DataRow with information, an explicit conversion is only required if the source data doesn't match the target field.
If the types are the same, you can just assign it.
e.g.
// With a decimal field, decimal target
Decimal myNum = 123;
// Works fine - assuming the Price datacolumn has been set up as a System.Decimal
row["Price"] = myNum;
Sounds like you may be creating the columns in the DataTable in code (as opposed to Filling via a DataAdapter which would set the column data types appropriately).
If that's the case, then can you not just create the DataColumns with the correct data types to start with, instead of all strings...and then ensure when you add rows to it, you convert the data into the appropriate type?
i.e. get to the point where your DataTable has the correct types and data in correctly, so SqlBulkCopy can then blast it into the DB with data already in the correct form.
was having the same problem where in I was suppose to copy data from an excel file which had strings and I wanted them to be stored in the database as double. I have pasted the code related to the question below :-
DataTable dt = new DataTable();
//Read Data from First Sheet
connExcel.Open();
cmdExcel.CommandText = "SELECT * From [" + SheetName + "] ";
oda.SelectCommand = cmdExcel;
oda.Fill(dt);
connExcel.Close();
Convert.ToDouble(dt.Columns["Product"]);
//where product(product temperature) is the a column whose datatype in the database is float and in the excel file is string.
Related
I have developed a web application to import Excel data into a SQL Server table using bulk copy column mapping. I used a select * from sheet query to copy the sheet data and insert into data table before writing it to table.
There are 20 columns in the sheet. The last column contains data of type datetime, string, integers and special characters. Its a mixed data type. When the application tried to copy and insert data into data table means it shows
Not a legal ole auth date
I have checked the last column values. Date value is correct only but it shows not a legal ole auth date issue.
Instead of using select * from sheet, is there any alternative approach like insert data into data table from Excel by row wise using for loop ?
Code is
OleDbConnection excelConnection1 = new OleDbConnection(excelConnectionString);
string query = string.Format("Select * from [{0}]", excelSheets[0]);
using (OleDbDataAdapter dataAdapter = new OleDbDataAdapter(query, excelConnection1))
{
dataAdapter.Fill(dt); **// issue raise when it try to fill the datatable**
}
I am passing in a TempTable to my SQLCLR code with the following schema
-- Schema for temp table
CREATE TABLE ##temp_table_configurationXml_local (
[OrchConfigXML] [xml]
)
SQLCLR Code:
DataTable dt = new DataTable(tableToUse); // tableToUse is the temp table from SQL
dt.Columns.Add("OrchConfigXML", typeof(SqlXml));
DataRow dr = dt.NewRow();
dr["orchConfigXML"] = xmlToUse; // This is type of SqlXML and has valid XML in it
dataTableToUse.Rows.Add(dr);
// Write Data
// auto-disposable bulk copy operation
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(conn))
{
bulkCopy.DestinationTableName = destTable;
bulkCopy.ColumnMappings.Add(new
SqlBulkCopyColumnMapping("OrchConfigXML", "OrchConfigXML"));
// Bulk copy operation will drain memory so it is better to setup a batchsize.
bulkCopy.BatchSize = 50000;
bulkCopy.WriteToServer(dt);
conn.Close(); //Close the SQL Connection.
}
While executing the SQL Script and debugging the SQLCLR code I get a error in C#/SQLCLR.
The given ColumnMapping does not match up with any column in the source or destination.
How to Map SQLXML to SQL XML datatype.
Questions:
How many columns are in the destTable? If just one, then you don't need SqlBulkCopyColumnMapping in the first place.
(answer: "There is only one column in destTable" )
What are you ultimately trying to do, and why are you using SqlBulkCopy?
How many rows will be in ##temp_table_configurationXml_local? If just one, then passing value in via SqlXml input parameter might be better.
Is xmlToUse being added, as another row, to whatever is in ##temp_table_configurationXml_local? This seems like an odd and/or convoluted setup.
Things to try / consider:
I can't remember, but using column names for the mapping might be case-sensitive, and you have "OrchConfigXML" when you defined the column, but "orchConfigXML" in the column mapping.
For the column definition in the DataTable, you might want to try using SqlDbType.Xml instead of SqlXml. SqlXml is used for input / output parameters, return types, and result set fields returned by SQLCLR objects.
I have a field declared as datetime in the CREATE TABLE query, I call this field BirthDay, if I have the following query and fill in a DataTable like this:
SQLiteDataAdapter da = new SQLiteDataAdapter("SELECT BirthDay FROM myTable", myConnection);
DataTable dt = new DataTable();
da.Fill(dt);
myDataGridView.DataSource = dt;
The column BirthDay of myDataGridView has type of DateTime and I can check this via myDataGridView.Columns["BirthDay"].ValueType.
However if I change the SELECT query a little like this:
SQLiteDataAdapter da = new SQLiteDataAdapter("SELECT CASE WHEN 1 THEN BirthDay END AS BirthDay2 FROM myTable", myConnection);
I expected the BirthDay2 column is also type of DateTime after filling in my DataTable and bind to myDataGridView. However the type is System.Object?
I've tried with CAST(... as DATETIME) but the returned type is System.Int64 (I understand that DATETIME in SQLite in fact corresponds to Int64 in .NET), I wonder how I could make it return System.DateTime as the original column BirthDay.
I need the final type (can be checked via DataGridViewColumn.ValueType) is DateTime so that I can use DataGridViewColumn.Format to format my datetime string as what I want.
Our team at work have been researching this issue for a long time. Our final conclusion is that any expression wrapping source field makes source data type unavailable. For ADO.NET providers using of expressions leads to System.Object final type, for ODBC provider it leads to System.String type. So it completely depends on data provider you use. Why does it behave in such way is a question to data provider developers (I haven't found the answer).
By the way, .NET Framework provides a variety of tools to test whether given value is of type expected. There are only 5 basic data types in SQLite 3 (NULL, INTEGER, REAL, TEXT, BLOB). So in .NET code you can pass your value through the test for 3 most common types (INTEGER, REAL, TEXT).
How do I read data from a DataSet in WPF? I have a train schedule table with just 2 columns and I want to be able to read the departure times and calculate when the next train is leaving. For example, the time now is 12:29 and my application should tell me that next train will depart at 12:33.
I already googled left and right. I'm on .NET 3.5.
DataSet resembles database. DataTable resembles database table, and DataRow resembles a record in a table. If you want to add filtering or sorting options, you then do so with a DataView object, and convert it back to a separate DataTable object.
If you're using database to store your data, then you first load a database table to a DataSet object in memory. You can load multiple database tables to one DataSet, and select specific table to read from the DataSet through DataTable object. Subsequently, you read a specific row of data from your DataTable through DataRow. Following codes demonstrate the steps:
SqlCeDataAdapter da = new SqlCeDataAdapter();
DataSet ds = new DataSet();
DataTable dt = new DataTable();
da.SelectCommand = new SqlCommand(#"SELECT * FROM FooTable", connString);
da.Fill(ds, "FooTable");
dt = ds.Tables["FooTable"];
foreach (DataRow dr in dt.Rows)
{
MessageBox.Show(dr["Column1"].ToString());
}
To read a specific cell in a row:
int rowNum // row number
string columnName = "DepartureTime"; // database table column name
dt.Rows[rowNum][columnName].ToString();
If ds is the DataSet, you can access the CustomerID column of the first row in the first table with something like:
DataRow dr = ds.Tables[0].Rows[0];
Console.WriteLine(dr["CustomerID"]);
If this is from a SQL Server datebase you could issue this kind of query...
Select Top 1 DepartureTime From TrainSchedule where DepartureTime >
GetUTCDate()
Order By DepartureTime ASC
GetDate() could also be used, not sure how dates are being stored.
I am not sure how the data is being stored and/or read.
TL;DR: - grab the datatable from the dataset and read from the rows property.
DataSet ds = new DataSet();
DataTable dt = new DataTable();
DataColumn col = new DataColumn("Id", typeof(int));
dt.Columns.Add(col);
dt.Rows.Add(new object[] { 1 });
ds.Tables.Add(dt);
var row = ds.Tables[0].Rows[0];
//access the ID column.
var id = (int) row.ItemArray[0];
A DataSet is a copy of data accessed from a database, but doesn't even require a database to use at all. It is preferred, though.
Note that if you are creating a new application, consider using an ORM, such as the Entity Framework or NHibernate, since DataSets are no longer preferred; however, they are still supported and as far as I can tell, are not going away any time soon.
If you are reading from standard dataset, then #KMC's answer is what you're looking for. The proper way to do this, though, is to create a Strongly-Typed DataSet and use that so you can take advantage of Intellisense. Assuming you are not using the Entity Framework, proceed.
If you don't already have a dedicated space for your data access layer, such as a project or an App_Data folder, I suggest you create one now. Otherwise, proceed as follows under your data project folder:
Add > Add New Item > DataSet. The file created will have an .xsd extension.
You'll then need to create a DataTable. Create a DataTable (click on the file, then right click on the design window - the file has an .xsd extension - and click Add > DataTable). Create some columns (Right click on the datatable you just created > Add > Column). Finally, you'll need a table adapter to access the data. You'll need to setup a connection to your database to access data referenced in the dataset.
After you are done, after successfully referencing the DataSet in your project (using statement), you can access the DataSet with intellisense. This makes it so much easier than untyped datasets.
When possible, use Strongly-Typed DataSets instead of untyped ones. Although it is more work to create, it ends up saving you lots of time later with intellisense. You could do something like:
MyStronglyTypedDataSet trainDataSet = new MyStronglyTypedDataSet();
DataAdapterForThisDataSet dataAdapter = new DataAdapterForThisDataSet();
//code to fill the dataset
//omitted - you'll have to either use the wizard to create data fill/retrieval
//methods or you'll use your own custom classes to fill the dataset.
if(trainDataSet.NextTrainDepartureTime > CurrentTime){
trainDataSet.QueueNextTrain = true; //assumes QueueNextTrain is in your Strongly-Typed dataset
}
else
//do some other work
The above example assumes that your Strongly-Typed DataSet has a column of type DateTime named NextTrainDepartureTime. Hope that helps!
How do you convert a DataColumn of GUIDs set as type string to a DataColumn of GUIDs where the type is Guid? A column in the source data has the wrong type and I cannot change it there.
Once the datatable has been populated with data, you can't change the column type. There's a few approaches you can take:
If you're using a DataAdapter to fill the table, you can use DataAdapter.FillSchema() to get the structure of the table first. Then you can modify the column type, and then fill it with data.
Take the initial table and clone it, change the column type on the new table, and then use DataTable.ImportRow() to import each row from the old table.
Create a new column, copy the values from the old column to the new column, and delete the old column.
You can add to womp's answer the following option:
After you get the table full of the data you can add a new DataColumn to the Columns of the data table this way:
DataColumn dc = new DataColumn("GUIDColumnName",typeof(Guid),"StringColumnName");
DataTableObj.Columns.Add(dc);
This will not have a performance hit if your data volume is huge.