I have excel sheet (named as $Sheet1) and sql table(named as Users) with same Formate which is given below
ID | UserName | FirstName | LastName | DateOfBirth |
1 | robert | robert | poinan | 1984 |
2 | joy | joy | rob | 1990 |
I Have read the whole excel sheet data in DataSet (Named as 'ds') now I want to insert the whole DataSet (which is 'ds') in sql table (which is 'Users')
I am using for loop (can also use foreach loop) to insert 'ds' (DataSet) rows one by one into Users (Sql table) table
sqlConn.Open();
for (int i = 0; i < ds.Tables[0].Rows.Count; i++)
{
SqlCommand cmd = new SqlCommand("INSERT INTO [Users] ([ID],[UserName],[FirstName],[LastName],[DateOfBirth]) VALUES(#ID,#UserName,#FirstName,#LastName,#DateOfBirth))", sqlConn);
cmd.Parameters.AddWithValue("#ID", ds.Tables[0].Rows[i][0].ToString());
cmd.Parameters.AddWithValue("#UserName", ds.Tables[0].Rows[i][1].ToString());
cmd.Parameters.AddWithValue("#FirstName", ds.Tables[0].Rows[i][2].ToString());
cmd.Parameters.AddWithValue("#LastName", ds.Tables[0].Rows[i][3].ToString());
cmd.Parameters.AddWithValue("#DateOfBirth", ds.Tables[0].Rows[i][4].ToString());
cmd.ExecuteNonQuery();
}
}
sqlConn.Close();
in this code I am facing a lot of problems one of these is, if there is any error in inserting the row the program stop but the rows that are inserted before are exists in the sql database next time when I try to run this program the data rows are duplicated.
I have millions of records. if I want to check the data row in sql table it takes a lot of time to execute the whole process.
My quetion is. Is there any way to insert the whole 'DataTable (which is in DataSet)' into Users Table at once
Something Like This
INSERT INTO [Users](ID, FirstName,LastName,DateOfBirth)
SELECT ID, FirstName,LastName,DateOfBirth FROM ds.Tables[0]
;
This is the approach given in MSDN(https://msdn.microsoft.com/en-us/library/ex21zs8x(v=vs.110).aspx). Make a DT and pass the DT as parameter to BulkCopy.
DataTable dt = new DataTable();
dt.Columns.Add("FieldName", typeof(System.Decimal));
Related
I wrote this code:
SqlConnection Conn = new SqlConnection("ConnectionString");
SqlCommand cmd = new SqlCommand("select * from tablename", conn);
SqlDataReader dr = cmd.ExecuteReader();
datatable dt = new datatable();
dt.load(dr);
But I get an exception as shown below when I load data into the datatable because I have an xml column with a big size (102 MB).
Exception of type 'System.OutOfMemoryException' was thrown.
I'm very grateful if someone can give me the solution for this exception.
Solutions to your problem:
Either normalize your database such that rather than raw XML, appropriate relational model is stored. That is how SQL works in its essentials. Hopefully, that will solve your problem.
However, no matter how well your database is normalized, there are limits beyond which data simply does not fit available memory. If it is your case, you need to abandon tough select * way and reimplement it in fetch-next-batch size, i.e. repeatedly fetch batches of fixed predefined size, process them, mark as processed somewhere and go on.
Here is a conceptual example.
T-SQL shreds XML data type column into a rectangular format.
The c# side of the equation will not have any problem.
SQL
-- DDL and sample data population, start
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, product VARCHAR(20), xmldata XML);
INSERT INTO #tbl (product, xmldata) VALUES
('vase', N'<root>
<r>red</r>
<r>blue</r>
<r>yellow</r>
</root>'),
('phone', N'<root>
<r>black</r>
<r>white</r>
</root>');
-- DDL and sample data population, start
SELECT id, product
, c.value('(./text())[1]', 'VARCHAR(20)') AS color
FROM #tbl CROSS APPLY xmldata.nodes('/root/r') AS t(c);
Output
+----+---------+--------+
| id | product | color |
+----+---------+--------+
| 1 | vase | red |
| 1 | vase | blue |
| 1 | vase | yellow |
| 2 | phone | black |
| 2 | phone | white |
+----+---------+--------+
We got in-house software and I'm trying to connect to it via OdbcDataAdapter. It's a in-house database as well.
I have managed to connect to Db via excel but I'm having problems in C.
I'm not sure how to phrase correctly the table names. I have managed in excel as trial and error.
String from Excel:
queryString = "SELECT * FROM ADI.\"kzn-57 | 600 | Survey Disabled | Realtime\"";
ADI = databasename
Table = kz - 50 | 600 | Data Disabled | Realtime
I'm getting errors as below:
Unknown Table Name.
string connectionString = "dsn=int_db";
DataSet dataSet = new DataSet();
OdbcConnection connection =
new OdbcConnection(connectionString);
string queryString = "SELECT * FROM ADI.\"kz-50 | 0600 | Data Disabled | Realtime\"";
OdbcDataAdapter adapter =
new OdbcDataAdapter(queryString, connection);
connection.Open(); // Connection established ok
adapter.Fill(dataSet); // unknown tables error
Console.WriteLine(dataSet.GetXml());
My scenario is that I have an Excel Spreadsheet that I want to upload to an SQL Database and UPDATE the information based upon the primary key value (which is locked and hidden within the Excel Spreadsheet). I have the below code that can be used to insert new entries into the database, but im not sure how to adapt it to UPDATE:
string path = string.Concat((Server.MapPath("~/temp/" + FileUpload1.FileName)));
FileUpload1.PostedFile.SaveAs(path);
OleDbConnection OleDbcon =
new OleDbConnection(#"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + path +
";Extended Properties=\"Excel 8.0;HDR=Yes;IMEX=1\";");
OleDbCommand cmd = new OleDbCommand("select * from [Sheet1$]",
OleDbcon);
OleDbDataAdapter objAdapter1 = new OleDbDataAdapter(cmd);
OleDbcon.Open();
DbDataReader dr = cmd.ExecuteReader();
string con_str =
#"Data Source=****************mydatasource""""""""""";
SqlBulkCopy bulkInsert = new SqlBulkCopy(con_str);
bulkInsert.DestinationTableName = "TableName";
bulkInsert.WriteToServer(dr);
OleDbcon.Close();
Example Data:
ID StartDate EndDate OrderNumber
1 01/02/2015 NULL 100
2 02/02/2015 NULL 100
3 03/02/2015 NULL 101
4 04/02/2015 NULL 102
5 05/02/2015 NULL 103
When the data in the database is inserted, the EndDate is Null.
The End Date is added into the excel sheet, and then I want the C# ASP.Net Application to UPDATE this information and update the SQL Table, not insert it as a new row.
End Data
ID StartDate EndDate OrderNumber
1 01/02/2015 02/02/2015 100
2 02/02/2015 03/02/2015 100
3 03/02/2015 04/02/2015 101
4 04/02/2015 05/02/2015 102
5 05/02/2015 06/02/2015 103
thanks for any help you can give me
The code you already have provides an efficient way of loading a file into SQL Server but doesn't provide any flexibility. Your options are to re-write your C# and do the INSERT/UPDATE one row at a time, or to change your approach slightly and bulkinsert the new data into a working table then call a stored procedure to do the actual INSERT/UPDATE from the working table to the real table - the latter would be my recommendation.
I believe you will need to iterate over the data reader and test for the date value and when found issue a single update statement, e.g.:
while (reader.Read())
{
//test if row exists
//then Update
//else Insert
}
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to get last inserted id?
I have a table Absences
| Id | Name | Job |
-------------------------
| 1 | James | 1 |
-------------------------
| 2 | Simon | 1 |
-------------------------
Where ID is an identity Primary Key incrementing by 1.
I'm accessing this table from a program in C# and I need to do the following :
Insert Into Absences (Name, Job) Values ('aName', 'aJob')
The problem is I need to get the Id column where i'm inserting at the same time because Nameand Job are not unique so I won't be able to retreive this exact column after.
Is it possible to add a select on the Id column in that query ?
Update
SqlConnection myConnection = new SqlConnection(#"SomeConnection");
myConnection.Open();
SqlCommand myCommand = myConnection.CreateCommand();
myCommand.CommandText = "Insert Into Absences (Name, Job) Values ('aName', 'aJob')";
int currentAbs = (int)myCommand.ExecuteScalar();
I get an error on the ExecuteScalar Line. Object reference is not set to and instance of object.
The SQL statement SCOPE_IDENTITY() will give you the value of the identity column of the newly inserted row from within the same scope.
SqlConnection myConnection = new SqlConnection(#"SomeConnection");
myConnection.Open();
SqlCommand myCommand = myConnection.CreateCommand();
myCommand.CommandText = "Insert Into Absences (Name, Job) Values ('aName', 'aJob'); SELECT SCOPE_IDENTITY();";
int currentAbs = (int)myCommand.ExecuteScalar();
Scope Identity Definition
If you use SqlCommand, then you can use
int lastId = (int)command.ExecuteScalar();
to retrieve the unique id of the inserted record.
Take a look at Microsoft page.
After this query you can select ##identity to get the last inserted id in mssql server.
One way would be to use SELECT ##IDENTITY immediately after you insert your record:
int id;
string query = "Insert Into Absences (Name, Job) Values ('aName', 'aJob')";
using (SqlCommand cmd = new SqlCommand(query, connection)) {
connection.Open();
// execute your INSERT query
cmd.ExecuteNonQuery();
// get the last-inserted ID
cmd.CommandText = "SELECT ##IDENTITY";
id = (int)cmd.ExecuteScalar();
}
I have a query - "SELECT PK1 FROM users";
I also have a datatable named myTable with three fields of type int;
-----------
| myTable |
-----------
| field1 |
| field2 |
| field3 |
-----------
For each row returned from my query, I would like to put the PK1 value into field2 in myTable leaving field1 and field3 null.
Here's some sample code I have tried so far, but it doesn't seem to be working. Instead, the field PK1 is appended to the end of myTable as a new field.
class Program
{
static void Main(string[] args)
{
string SQL = "SELECT PK1 FROM users";
SqlConnection connection = new SqlConnection([my connection string]);
DataTable myTable = new DataTable();
SqlDataAdapter adapter = new SqlDataAdapter();
adapter.SelectCommand = new SqlCommand(SQL, connection);
myTable.Columns.Add("field1", typeof(Int32));
myTable.Columns.Add("field2", typeof(Int32));
myTable.Columns.Add("field3", typeof(Int32));
DataTableMapping dataMapping = adapter.TableMappings.Add("myTable", "users");
dataMapping.ColumnMappings.Add("PK1", "field2");
adapter.Fill(myTable);
foreach (DataRow row in myTable.Rows)
{
Console.WriteLine("------------");
foreach (DataColumn column in myTable.Columns)
{
Console.WriteLine(column.ColumnName + " : " + row[column]);
}
}
Console.ReadKey();
}
}
Is there an issue in my code, or is there another way I can map the fields across?
This is a simplified example for the purposes of this question, as such mapping the field names would be ideal rather than inserting the value at a given position.
I think if it is your requirement then there is nothing wrong with this approach.
Change your query to the following, and you don't need the mapping.
string SQL = "SELECT PK1 as field2 FROM users";
it is adding the column because you are telling it to. You should either use the datamappingColumn feature or the table.column.add feature not both, they are redundant, and probably not necessary.
So if you leave the table.column.add()
Here is an example of how you would "bind" the column names to the columns you set in table, you specify the order in which you want them to appear, you can leave them blank by "" double quote or set them to null by using NULL.
this.table.AutoSetDataBoundColumnHeaders = true;
this.table.SetDataBinding(table, null, "PK1", null);