I want to update multiple rows like below
update mytable set s_id = {0} where id = {1}
(Here s_id is evaluated based on some complex logic).
For performance reason, updates should happen in batches. Is there any way to batch the update statements and execute the batch through single execute statements? I know in JAVA we can do this through JDBC. Is there similar way in C#?
Thanks in advance
Yes, you can use an SqlDataAdapter.
The SqlDataAdapter has InsertCommand and UpdateCommand properties which allow you to specify an SQLCommand to use to insert new rows into the database and an SqlCommand to update rows in the database respectively.
You can then pass a DataTable to the Update method of the dataadapter, and it will batch up the statements to the server - for rows in the DataTable that are new rows, it executes the INSERT command, for modified rows it executes the UPDATE command.
You can define the batch size using the UpdateBatchSize property.
This approach allows you to deal with large volumes of data, and allows you to nicely handle errors in different ways, i.e. if an error is encountered with a particular update, you can tell it to NOT throw an exception but to carry on with the remaining updates by setting the ContinueUpdateOnError property.
Yes, you can build a plain-text SQL command (parameterized for security), like this:
SqlCommand command = new SqlCommand();
// Set connection, etc.
for(int i=0; i< items.length; i++) {
command.CommandText += string.Format("update mytable set s_id=#s_id{0} where id = #id{0};", i);
command.Parameters.Add("#s_id" + i, items[i].SId);
command.Parameters.Add("#id" + i, items[i].Id);
}
command.ExecuteNonQuery();
Use a StringBuilder (System.Text.StringBuilder) to build your Sql, such as:
StringBuilder sql = new StringBuilder();
int batchSize = 10;
int currentBatchCount = 0;
SqlCommand cmd = null; // The SqlCommand object to use for executing the sql.
for(int i = 0; i < numberOfUpdatesToMake; i++)
{
int sid = 0; // Set the s_id here
int id = 0; // Set id here
sql.AppendFormat("update mytable set s_id = {0} where id = {1}; ", sid, id);
currentBatchCount++;
if (currentBatchCount >= batchSize)
{
cmd.CommandText = sql.ToString();
cmd.ExecuteNonQuery();
sql = new StringBuilder();
currentBatchCount = 0;
}
}
Create a set of those updates (with the id's filled in), separate them by semicolon in one string, set the resulting string to a SqlCommand's CommandText property, then call ExecuteNonQuery().
Related
I'm trying to find a way to have access to all the values in a row.
The following code returns one cell. If I change select id to select *, I have access to the row but how can I break it apart?
string find_user = "select id from users where userName = '" + un + "'";
using (SqlConnection con = new SqlConnection(cs))
{
using (SqlCommand cmd = new SqlCommand(find_user, con))
{
con.Open();
user_id = cmd.ExecuteScalar().ToString();
/* use to pass the info to all the pages */
Session.Add("u_id", user_id);
}
}
You cannot access additional columns using .ExecuteScalar(), per the docs:
Executes the query, and returns the first column of the first row in the result set returned by the query. Additional columns or rows are ignored.
Although it is not a route that I would recommend, you can iterate through the fields by using an index on a data reader:
SqlDataReader dataReader = cmd.ExecuteReader();
// for the query's result set, this while loop will go through all the records
while (dataReader.Read())
{
// for the current record, this for loop will go through all the fields
for (int i = 0; i < dataReader.FieldCount; i++)
{
var value = dataReader[i]; // do what you need with the data here
}
}
A better approach would be to specify the field names in the SQL query instead of using SELECT *, then get the values from the data reader by the specific field names (not relying on the order of the fields in the DB).
Also, you have a SQL injection vulnerability. You should look up what this means and how to parameterize a query.
I want to delete all rows in a datatable.
I use something like this:
foreach (DataRow row in dt.Rows)
{
row.Delete();
}
TableAdapter.Update(dt);
It works good but it takes lots of time if I have much rows.
Is there any way to delete all rows at once?
If you are running your code against a sqlserver database then
use this command
string sqlTrunc = "TRUNCATE TABLE " + yourTableName
SqlCommand cmd = new SqlCommand(sqlTrunc, conn);
cmd.ExecuteNonQuery();
this will be the fastest method and will delete everything from your table and reset the identity counter to zero.
The TRUNCATE keyword is supported also by other RDBMS.
5 years later:
Looking back at this answer I need to add something. The answer above is good only if you are absolutely sure about the source of the value in the yourTableName variable. This means that you shouldn't get this value from your user because he can type anything and this leads to Sql Injection problems well described in this famous comic strip. Always present your user a choice between hard coded names (tables or other symbolic values) using a non editable UI.
This will allow you to clear all the rows and maintain the format of the DataTable.
dt.Rows.Clear();
There is also
dt.Clear();
However, calling Clear() on the DataTable (dt) will remove the Columns and formatting from the DataTable.
Per code found in an MSDN question, an internal method is called by both the DataRowsCollection, and DataTable with a different boolean parameter:
internal void Clear(bool clearAll)
{
if (clearAll) // true is sent from the Data Table call
{
for (int i = 0; i < this.recordCapacity; i++)
{
this.rows[i] = null;
}
int count = this.table.columnCollection.Count;
for (int j = 0; j < count; j++)
{
DataColumn column = this.table.columnCollection[j];
for (int k = 0; k < this.recordCapacity; k++)
{
column.FreeRecord(k);
}
}
this.lastFreeRecord = 0;
this.freeRecordList.Clear();
}
else // False is sent from the DataRow Collection
{
this.freeRecordList.Capacity = this.freeRecordList.Count + this.table.Rows.Count;
for (int m = 0; m < this.recordCapacity; m++)
{
if ((this.rows[m] != null) && (this.rows[m].rowID != -1))
{
int record = m;
this.FreeRecord(ref record);
}
}
}
}
As someone mentioned, just use:
dt.Rows.Clear()
That's the easiest way to delete all rows from the table in dbms via DataAdapter. But if you want to do it in one batch, you can set the DataAdapter's UpdateBatchSize to 0(unlimited).
Another way would be to use a simple SqlCommand with CommandText DELETE FROM Table:
using(var con = new SqlConnection(ConfigurationSettings.AppSettings["con"]))
using(var cmd = new SqlCommand())
{
cmd.CommandText = "DELETE FROM Table";
cmd.Connection = con;
con.Open();
int numberDeleted = cmd.ExecuteNonQuery(); // all rows deleted
}
But if you instead only want to remove the DataRows from the DataTable, you just have to call DataTable.Clear. That would prevent any rows from being deleted in dbms.
Why dont you just do it in SQL?
DELETE FROM SomeTable
Just use dt.Clear()
Also you can set your TableAdapter/DataAdapter to clear before it fills with data.
TableAdapter.Update(dt.Clone());
//or
dt=dt.Clone();
TableAdapter.Update(dt);
//or
dt.Rows.Clear();
dt.AcceptChanges();
TableAdapter.Update(dt);
If you are really concerned about speed and not worried about the data you can do a Truncate. But this is assuming your DataTable is on a database and not just a memory object.
TRUNCATE TABLE tablename
The difference is this removes all rows without logging the row deletes making the transaction faster.
Here we have same question. You can use the following code:
SqlConnection con = new SqlConnection();
con.ConnectionString = ConfigurationManager.ConnectionStrings["yourconnectionstringnamehere"].ConnectionString;
con.Open();
SqlCommand com = new SqlCommand();
com.Connection = con;
com.CommandText = "DELETE FROM [tablenamehere]";
SqlDataReader data = com.ExecuteReader();
con.Close();
But before you need to import following code to your project:
using System.Configuration;
using System.Data.SqlClient;
This is the specific part of the code that can delete all rows is a table:
DELETE FROM [tablenamehere]
This must be your table name:tablenamehere
- This can delete all rows in table: DELETE FROM
I using MDaf just use this code :
DataContext db = new DataContext(ConfigurationManager.ConnectionStrings["con"].ConnectionString);
db.TruncateTable("TABLE_NAME_HERE");
//or
db.Execute("TRUNCATE TABLE TABLE_NAME_HERE ");
Here is a clean and modern way to do it using Entity FW and without SQL Injection or TSQL..
using (Entities dbe = new Entities())
{
dbe.myTable.RemoveRange(dbe.myTable.ToList());
dbe.SaveChanges();
}
Is there a Clear() method on the DataTable class??
I think there is. If there is, use that.
Datatable.clear() method from class DataTable
I'm trying to write a program using SQL and OleDB and I get an error while the Program is running.
the program first counts the number of rows in the table(access table which called 'tblCodons')
and storage the number as integer in j.
then the program stores all the rows (from a specific column which called 'codonsFullName') in comboBox1.
the code is below
I get this ERROR:System.Data.OleDb.OleDbException (0x80040E14): Invalid SQL statement;Required values' DELETE ',' INSERT ',' PROCEDURE ',' SELECT 'or' UPDATE
the code:
int j=0;
OleDbConnection conn1 = new OleDbConnection(connectionString);
conn1.Open();
string sqlCount= "SET #j= SELECT COUNT(tblCodons.codonsFullName) FROM tblCodons";
OleDbCommand counter = new OleDbCommand(sqlCount, conn1);
counter.ExecuteNonQuery();
conn1.Close();
OleDbConnection conn2 = new OleDbConnection(connectionString);
conn2.Open();
string sqlFill = "SELECT tblCodons.codonsFullName FROM tblCodons";
OleDbCommand fill = new OleDbCommand(sqlFill, conn2);
fill.ExecuteNonQuery();
OleDbDataReader dataReader = fill.ExecuteReader();
dataReader.Read();
for (int i = 0; i < j; i++)
{
comboBox1.Items.Add(dataReader.GetString(i));
}
You seem to need the count only for the loop. Also I do not understand why you are executing fill.ExecuteNonQuery() before executing is as a reader.
Also setting #j (if it did work) in a sql query has no scope to a local variable j in the code you are trying to set.
You should only need the following code (apologies for any syntax errors)
OleDbConnection conn2 = new OleDbConnection(connectionString);
conn2.Open();
string sqlFill = "SELECT tblCodons.codonsFullName FROM tblCodons";
OleDbCommand fill = new OleDbCommand(sqlFill, conn2);
OleDbDataReader dataReader = fill.ExecuteReader();
int j = 0;
if (dataReader.HasRows)
{
while(dataReader.Read())
{
comboBox1.Items.Add(dataReader.GetString(0));
j++;
}
}
Hope that helps
Leaving this answer here as an explanation for fixing your code as it currently exists, but also want to point out that I recommend going with Kamal's solution; it only queries the database once.
This line is probably your error:
string sqlCount= "SET #j= SELECT COUNT(tblCodons.codonsFullName) FROM tblCodons";
change to
string sqlCount= "SELECT COUNT(tblCodons.codonsFullName) FROM tblCodons";
You'll want to change your code to obtain the result of that first query like this:
j = counter.ExecuteScalar();
First, as Kamal Mentioned you can't directly set a variable from a sql query as you are trying to do and as the exception states only "SELECT" , "INSERT","UPDATE" and "DELETE" commands can be use in a query.
Second, I don't know why you need to get the record counts before getting the actual data but if it's really necessary you can write your query like this :
var query="SELECT COUNT(tblCodons.codonsFullName) FROM tblCodons;SELECT tblCodons.codonsFullName FROM tblCodons;";
Then you can execute both query using a single DataReader. When you execute DataReader.ExequteQuery() it will contain two results.the first one has access to the count and the second one has access to actual data.
Here's an example
I have an array of objects. Each object contains around 20 members. I need to loop through the array and insert the data from the object into my database. Is there a way of doing this that does not require me to put an INSERT statement within the body of my loop? I am using C# and SQL Server.
for(int i =0; i < arr.length; i++)
{
strSQL = "INSERT INTO myTable (field1...field20) VALUES (" + arr[i].field1 + "..." + arr[i].field20)
sqlCmd.execute(strSQL,sqlConn)
}
Why don't you use linq ?
Yes there is way.
First, +1 for lollancf37's answer.
And second, you can use StringBuilder. Build your query parameter with 1 query using StringBuilder, and execute 1 time
Enjoy
SqlCommand sqlcmd = null;
SqlParameter pField1 = new SqlParameter("#Field1", System.Data.SqlDbType.VarChar, 255);
...
SqlParameter pField20 = new SqlParameter("#Field20", System.Data.SqlDbType.VarChar, 255);
try{
sqlcmd = new SqlCommand("INSERT INTO myTable (field1...field20) VALUES (#Field1,...,#Field20)",sqlConn)
sqlcmd.Parameters.Add(pField1);
...
sqlcmd.Parameters.Add(pField20);
for(int i =0; i < arr.length; i++)
{
pField1.value = arr[i].field1;
...
pField20.value = arr[i].field20;
sqlCmd.ExecuteNonQuery();
}
}
catch (Exception ex)
{
LogError(ex.message)
}
finally
{
if (sqlconn != null && sqlconn.State != System.Data.ConnectionState.Closed)
sqlconn.Close();
if (sqlcmd != null)
sqlcmd.Dispose();
}
For simple commands, you could use a format string, something like this:
//assuming that the first param is a number, the second one a string etc...
string insertFormat = #"Insert into myTable((field1...field20)
VALUES ({0},'{1}',..,19})"
for(int i =0; i < arr.length; i++)
{
//assuming that arr[i] is string[]
strSQL = string.Format(insertFormat, arr[i]);
sqlCmd.execute(strSQL,sqlConn)
}
You must be aware, however, that you are leaving yourself wide open for SQL Injection (mandatory xkcd). In order to avoid that, and other issues, you could take a look at "advanced| solutions (using a stored procedure, using reflection to map the fields, using some ORM tool, using Linq-to-sql, etc)
strSQL = "INSERT INTO MYTABLE (COL01, COL01...COL42)";
for(int i = 0; i < arr.length; i++)
{
strSQL += "SELECT (" +arr[0] +", " +arr[1] +"..." +arr[42] +")";
if(i < arr.length - 1) {
strSQL += " UNION ALL ";
}
}
sqlCommand.Execute(strSQL, conn); // I forget how this bit goes, I am not a C# programmer by trade...
I find that inserting more than 500 records at a time like this makes the database run pretty slow.
Parameterised stored procedures.
Write the insert statement down in the db as a stored procedure, with parameters.
Create a method to populate the stored procedure, passing in the object as the parameter.
Call the method in the loop, populating an passing in an object instance each time.
I had a similar problem at one point, the most effective way I found to deal with it was create a 2-dimensional array for your data, and store the column name for each line of data in the array in addition to storing the data itself.
From there, it's pretty trivial to loop through it and use a StringBuilder to assemble the query. Additionally, instead of simply throwing the insert values on, put a parameter name on them (I just used the column name). Then, in the same loop, you can create a new SqlParameter.
If you're using .NET 4, just create an SqlParameter array and add your created parameters to it in your loop. After your loop ends, you can use the "AddRange' method of your command's parameter collection to simply add the parameter array to your command.
This way, you're able to build a fully dynamic query that sanitizes inputs.
Replaces Question: Update multiple rows into SQL table
Here's a Code Snippet to update an exam results set.
DB structure is as given, but I can submit Stored Procedures for inclusion (Which are a pain to modify, so I save that until the end.)
The question: Is there a better way using SQL server v 2005.,net 2.0 ?
string update = #"UPDATE dbo.STUDENTAnswers
SET ANSWER=#answer
WHERE StudentID =#ID and QuestionNum =#qnum";
SqlCommand updateCommand = new SqlCommand( update, conn );
conn.Open();
string uid = Session["uid"].ToString();
for (int i= tempStart; i <= tempEnd; i++)
{
updateCommand.Parameters.Clear();
updateCommand.Parameters.AddWithValue("#ID",uid);
updateCommand.Parameters.AddWithValue("#qnum",i);
updateCommand.Parameters.AddWithValue("#answer", Request.Form[i.ToString()]);
try
{
updateCommand.ExecuteNonQuery();
}
catch { }
}
A few things stand out:
You don't show where the SqlConnection is instantiated, so it's not clear that you're disposing it properly.
You shouldn't be swallowing exceptions in the loop - better to handle them in a top level exception handler.
You're instantiating new parameters on each iteration through the loop - you could just reuse the parameters.
Putting this together it could look something like the following (if you don't want to use a transaction, i.e. don't care if some but not all updates succeed):
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
using (SqlCommand updateCommand = new SqlCommand(update, conn))
{
string uid = Session["uid"].ToString();
updateCommand.Parameters.AddWithValue("#ID", uid);
updateCommand.Parameters.AddWithValue("#qnum", i);
updateCommand.Parameters.Add("#answer", System.Data.SqlDbType.VarChar);
for (int i = tempStart; i <= tempEnd; i++)
{
updateCommand.Parameters["#answer"] = Request.Form[i.ToString()];
updateCommand.ExecuteNonQuery();
}
}
}
Or to use a transaction to ensure all or nothing:
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
using (SqlTransaction transaction = conn.BeginTransaction())
{
using (SqlCommand updateCommand = new SqlCommand(update, conn, transaction))
{
string uid = Session["uid"].ToString();
updateCommand.Parameters.AddWithValue("#ID", uid);
updateCommand.Parameters.AddWithValue("#qnum", i);
updateCommand.Parameters.Add("#answer", System.Data.SqlDbType.VarChar);
for (int i = tempStart; i <= tempEnd; i++)
{
updateCommand.Parameters["#answer"] = Request.Form[i.ToString()];
updateCommand.ExecuteNonQuery();
}
transaction.Commit();
}
} // Transaction will be disposed and rolled back here if an exception is thrown
}
Finally, another problem is that you are mixing UI code (e.g. Request.Form) with data access code. It would be more modular and testable to separate these - e.g. by splitting your application into UI, Business Logic and Data Access layers.
For 30 updates I think you're on the right track, although the comment about the need for a using around updateCommand is correct.
We've found the best performing way to do bulk updates (>100 rows) is via the SqlBulkCopy class to a temporary table followed by a stored procedure call to populate the live table.
An issue I see is when you are opening your connection.
I would at least before every update call the open and then close the connection after the update.
If your loop takes time to execute you will have your connection open for a long time.
It is a good rule to never open your command until you need it.
You can bulk insert using OpenXML. Create an xml document containing all your questions and answers and use that to insert the values.
Edit: If you stick with your current solution, I would at least wrap your SqlConnection and SqlCommand in a using block to make sure they get disposed.
emit a single update that goes against a values table:
UPDATE s SET ANSWER=a FROM dbo.STUDENTAnswers s JOIN (
SELECT 1 as q, 'answer1' as a
UNION ALL SELECT 2, 'answer2' -- etc...
) x ON s.QuestionNum=x.q AND StudentID=#ID
so you just put this together like this:
using(SqlCommand updateCommand = new SqlCommand()) {
updateCommand.CommandType = CommandType.Text;
updateCommand.Connection = conn;
if (cn.State != ConnectionState.Open) conn.Open();
StringBuilder sb = new StringBuilder("UPDATE s SET ANSWER=a FROM dbo.STUDENTAnswers s JOIN (");
string fmt = "SELECT {0} as q, #A{0} as a";
for(int i=tempStart; i<tempEnd; i++) {
sb.AppendFormat(fmt, i);
fmt=" UNION ALL SELECT {0},#A{0}";
updateCommand.Parameters.AddWithValue("#A"+i.ToString(), Request.Form[i.ToString()]);
}
sb.Append(") x ON s.QuestionNum=x.q AND StudentID=#ID");
updateCommand.CommandText = sb.ToString();
updateCommand.Parameters.AddWithValue("#ID", uid);
updateCommand.ExecuteNonQuery();
}
This has the advantages of being an all other nothing operation (like if you'd wrapped several updates in a transaction) and will run faster since:
The table and associated indexes are looked at/updated once
You only pay for the latency between your application and the database server once, rather than on each update