I'm trying to make sure that both my insert and delete below work completely or not at all. I have my connection object outside of my transaction scope which I believe is correct by not 100% sure.
I do know that this code is not working as I intent. After the first part (the insert runs) and then I abort by terminating on a break point, the rows are indeed inserted even though I never called scope.complete.
Please point out the flaw in my thinking and logic here.
sqlConnection.Open();
int numFound = 1;
int max = 99;
int iteration = 0;
while (iteration < max && numFound > 0)
{
iteration++;
var ids = new List<int>();
using (var sqlCommand0 = new SqlCommand(sql0, sqlConnection))
{
using (SqlDataReader reader1 = sqlCommand0.ExecuteReader())
{
while (reader1.Read())
{
ids.Add(reader1.GetInt32(0));
}
}
}
numFound = ids.Count;
if (numFound > 0)
{
using (var scope = new TransactionScope())
{
string whereClause = $"WHERE Id IN ({string.Join(",", ids)})";
string sql1 = string.Format(sqlTemplate1, whereClause);
using (var sqlCommand1 = new SqlCommand(sql1, sqlConnection))
{
sqlCommand1.ExecuteNonQuery();
}
// BREAK POINT HERE - ABORTED PROGRAM AND sql1 had been committed.
var sql2 = "DELETE FROM SendGridEventRaw " + whereClause;
using (var sqlCommand2 = new SqlCommand(sql2, sqlConnection))
{
sqlCommand2.ExecuteNonQuery();
}
scope.Complete();
total += numFound;
Console.WriteLine("deleted: " + whereClause);
}
}
}
}
I think it's because you open your connection before starting your transaction. You could try to fix your issue by first starting your transaction and then opening your connection.
Just from what I am seeing and from what I am assuming is what you intend to happen is this:
If your first query gets some records, then the next query executes, hence the statement:
if (numFound > 0)
If that is the case, and where you put your breakpoint is true, of course the insert statement will fire. Reason is:
using (var sqlCommand1 = new SqlCommand(sql1, sqlConnection))
{
sqlCommand1.ExecuteNonQuery();
}
is within that if statement. You're saying "if there are any rows, execute the insert query."
If you're trying to actually get the scope object to do the query, then you're going to have to have all of the query construction happening within the object and then having scope.complete() doing the execution.
For example:
//In TransactionScope class
public string Complete(var ids, int numFound, SqlConnection sqlConnection, string sqlTemplate1)
{
string whereClause = $"WHERE Id IN ({string.Join(",", ids)})";
string sql1 = string.Format(sqlTemplate1, whereClause);
using (var sqlCommand1 = new SqlCommand(sql1, sqlConnection))
{
sqlCommand1.ExecuteNonQuery();
}
var sql2 = "DELETE FROM SendGridEventRaw " + whereClause;
using (var sqlCommand2 = new SqlCommand(sql2, sqlConnection))
{
sqlCommand2.ExecuteNonQuery();
}
return whereClause;
}
//in your Main class
if (num > 0)
{
string whereClause = scope.Complete(ids, numFound, sqlConnection, sqlTemplate1);
Console.WriteLine("deleted" + whereClause"." );
}
I am of course just going off of the assumptions I stated above. If I am incorrect, please let me know.
Hope it helps.
Related
I wonder how can i bulk insert instead of execute this method everytime.
It's getting slow when i try to insert 1000 rows:
queryText = "INSERT INTO `player_items` (`player_id`, `item_id`, `count`) VALUES (#player_id, #item_id, #count)";
for (int i = 0; i < player.invenotrySize; i++)
{
Item item = player.inventory.GetItem[i];
MySqlParameter[] parameters = {
new MySqlParameter("player_id", 1),
new MySqlParameter("item_id", item.data.id),
new MySqlParameter("count", item.amount),
};
ExecuteNonQuery(queryText, parameters);
}
public int ExecuteNonQuery(string queryText, params MySqlParameter[] parameters)
{
int affectedRows = 0;
using (MySqlConnection mySqlConnection = CreateConnection())
{
using (MySqlCommand mySqlCommand = new MySqlCommand(queryText, mySqlConnection))
{
mySqlCommand.CommandType = CommandType.Text;
mySqlCommand.Parameters.AddRange(parameters);
affectedRows = mySqlCommand.ExecuteNonQuery();
}
}
return affectedRows;
}
I think the optimal way is to insert everything as a huge row. E.g
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
But i have no idea how i can make a method to take care of this setup
You are opening and closing your connection for every single insert.
using (MySqlConnection mySqlConnection = CreateConnection())
This is a very expensive procedure, and therefore not really the way to work with a DB.
You should open your connection just once, and then close it when finished. Depending on what you app does this might be when you start your App (or before you do your first DB query) and then close it when exiting the App (or after you are certain there will be no more DB queries.
Then ideally you should also reuse the SqlCommand instance as well. But you need to make sure that you clear your parameters in between. So then you have something like this
int affectedRows = 0;
using (MySqlConnection mySqlConnection = CreateConnection())
{
string queryText = "INSERT INTO `player_items` (`player_id`, `item_id`, `count`) VALUES (#player_id, #item_id, #count)";
using (MySqlCommand mySqlCommand = new MySqlCommand(queryText, mySqlConnection))
{
mySqlCommand.CommandType = CommandType.Text;
for (int i = 0; i < player.invenotrySize; i++)
{
Item item = player.inventory.GetItem[i];
MySqlParameter[] parameters = {
new MySqlParameter("player_id", 1),
new MySqlParameter("item_id", item.data.id),
new MySqlParameter("count", item.amount)};
mySqlCommand.Parameters.Clear();
mySqlCommand.Parameters.AddRange(parameters);
affectedRows += mySqlCommand.ExecuteNonQuery();
}
}
}
It's not realy clean with your "ExecuteNonQuery" (do a multi row insert solution or just isolate/singleton the connection class like the solution above, will be better) but you can construct your whole query before execute instead of get/add connection, replace, execute foreach player.
queryText = "INSERT INTO `player_items` (`player_id`, `item_id`, `count`) VALUES";
for (int i = 0; i < player.invenotrySize; i++)
{
Item item = player.inventory.GetItem[i];
MySqlParameter[] parameters = {
new MySqlParameter("player_id_"+i, 1),
new MySqlParameter("item_id_"+i, item.data.id),
new MySqlParameter("count_"+i, item.amount),
};
queryText+= " (#player_id_"+i+", #item_id_"+i+", #count_"+i+"),";
}
//remove the last ,
queryText= queryText.Remove(queryText.Length - 1)+";";
ExecuteNonQuery(queryText, parameters);
Altnernate for to skip params if you are sure about your data.
Item item = player.inventory.GetItem[i];
queryText+= " (1, "+item.data.id+", "+item.amount+"),";
I have a very silly problem. I am doing a select, and I want that when the value comes null, return an empty string. When there is value in sql query, the query occurs all ok, but if there is nothing in the query, I have to give a sqlCommand.CommandTimeout greater than 300, and yet sometimes gives timeout. Have a solution for this?
public string TesteMetodo(string codPess)
{
var vp = new Classe.validaPessoa();
string _connection = vp.conString();
string query = String.Format("SELECT COUNT(*) FROM teste cliente WHERE cod_pess = {0}", codPess);
try
{
using (var conn = new SqlConnection(_connection))
{
conn.Open();
using (var cmd = new SqlCommand(query, conn))
{
SqlDataReader dr = cmd.ExecuteReader();
if(dr.HasRows)
return "";
return codPess;
}
}
}
You should probably validate in the UI and pass an integer.
You can combine the usings to a single block. A bit easier to read with fewer indents.
Always use parameters to make the query easier to write and avoid Sql Injection. I had to guess at the SqlDbType so, check your database for the actual type.
Don't open the connection until directly before the .Execute. Since you are only retrieving a single value you can use .ExecuteScalar. .ExecuteScalar returns an Object so must be converted to int.
public string TesteMetodo(string codPess)
{
int codPessNum = 0;
if (!Int32.TryParse(codPess, out codPessNum))
return "codPess is not a number";
var vp = new Classe.validaPessoa();
try
{
using (var conn = new SqlConnection(vp.conString))
using (var cmd = new SqlCommand("SELECT COUNT(*) FROM teste cliente WHERE cod_pess = #cod_pess", conn))
{
cmd.Parameters.Add("#cod_pess", SqlDbType.Int).Value = codPessNum;
conn.Open();
int count = (int)cmd.ExecuteScalar();
if (count > 0)
return "";
return codPess;
}
}
catch (Exception ex)
{
return ex.Message;
}
}
I have a program where I open a SqlConnection, load up a list of objects, modify a value on each object, then update the rows in the SQL Server database. Because the modification requires string parsing I wasn't able to do with with purely T-SQL.
Right now I am looping through the list of objects, and running a SQL update in each iteration. This seems inefficient and I'm wondering if there is a more efficient way to do it using LINQ
The list is called UsageRecords. The value I'm updating is MthlyConsumption.
Here is my code:
foreach (var item in UsageRecords)
{
string UpdateQuery = #"UPDATE tbl810CTImport
SET MthlyConsumption = " + item.MthlyConsumption +
"WHERE ID = " + item.Id;
SqlCommand update = new SqlCommand(UpdateQuery, sourceConnection);
update.ExecuteNonQuery();
}
Try this instead:
string UpdateQuery = #"UPDATE tbl810CTImport SET MthlyConsumption = #consumption WHERE ID = #itemId";
var update = new SqlCommand(UpdateQuery, sourceConnection);
update.Parameters.Add("#consumption", SqlDbType.Int); // Specify the correct types here
update.Parameters.Add("#itemId", SqlDbType.Int); // Specify the correct types here
foreach (var item in UsageRecords)
{
update.Parameters[0].Value = item.MthlyConsumption;
update.Parameters[1].Value = item.Id;
update.ExecuteNonQuery();
}
It should be faster because:
You don't have to create the command each time.
You don't create a new string each time (concatenation)
The query is not parsed at every iteration (Just changes the parameters values).
And it will cache the execution plan. (Thanks to #JohnCarpenter from the comment)
You can either use
SqlDataAdapter - See How to perform batch update in Sql through C# code
or what I have previously done was one of the following:
Tear down the ID's in question, and re-bulkinsert
or
Bulk Insert the ID + new value into a staging table, and update the table on SQL server:
update u
set u.MthlyConsumption = s.MthlyConsumption
from tbl810CTImport u
inner join staging s on
u.id = s.id
In a situation like this, where you can't write a single update statement to cover all your bases, it's a good idea to batch up your statements and run more than one at a time.
var commandSB = new StringBuilder();
int batchCount = 0;
using (var updateCommand = sourceConnection.CreateCommand())
{
foreach (var item in UsageRecords)
{
commandSB.AppendFormat(#"
UPDATE tbl810CTImport
SET MthlyConsumption = #MthlyConsumption{0}
WHERE ID = #ID{0}",
batchCount
);
updateCommand.Parameters.AddWithValue(
"#MthlyConsumption" + batchCount,
item.MthlyConsumption
);
updateCommand.Parameters.AddWithValue(
"#ID" + batchCount,
item.MthlyConsumption
);
if (batchCount == 500) {
updateCommand.CommandText = commandSB.ToString();
updateCommand.ExecuteNonQuery();
commandSB.Clear();
updateCommand.Parameters.Clear();
batchCount = 0;
}
else {
batchCount++;
}
}
if (batchCount != 0) {
updateCommand.ExecuteNonQuery();
}
}
It should be as simple as this . . .
private void button1_Click(object sender, EventArgs e)
{
SqlConnection con = new SqlConnection("Server=YourServerName;Database=YourDataBaseName;Trusted_Connection=True");
try
{
//cmd new SqlCommand( "UPDATE Stocks
//SET Name = #Name, City = #cit Where FirstName = #fn and LastName = #add";
cmd = new SqlCommand("Update Stocks set Ask=#Ask, Bid=#Bid, PreviousClose=#PreviousClose, CurrentOpen=#CurrentOpen Where Name=#Name", con);
cmd.Parameters.AddWithValue("#Name", textBox1.Text);
cmd.Parameters.AddWithValue("#Ask", textBox2.Text);
cmd.Parameters.AddWithValue("#Bid", textBox3.Text);
cmd.Parameters.AddWithValue("#PreviousClose", textBox4.Text);
cmd.Parameters.AddWithValue("#CurrentOpen", textBox5.Text);
con.Open();
int a = cmd.ExecuteNonQuery();
if (a > 0)
{
MessageBox.Show("Data Updated");
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
finally
{
con.Close();
}
}
Change the code to suit your needs.
I created an list named 'PTNList', and everything I needed added to it just fine. Now I am attempting to write code to retrieve each element from that list and run it against an SQL query. I have a feeling I'm not sure exactly how to about this. The CompareNumbers.txt file generates, but nothing is printed to it. Any help is greatly appreciated.
Below is the section of code I believe needs to be worked with.
using (FileStream fs = new FileStream("c:/temp/CompareNumbers.txt", FileMode.Append, FileAccess.Write))
using (StreamWriter sw = new StreamWriter(fs))
foreach (var ptn in PTNList)
{
//create sql for getting the count using "ptn" as the variable thats changing
//call DB with sql
//Get count from query, write it out to a file;
Console.WriteLine("Running Query");
string query2 = #"SELECT COUNT(PRODUCT_TYPE_NO)
AS NumberOfProducts
FROM dbo.PRODUCT
Where PRODUCT_TYPE_NO = " + ptn;
SqlCommand cmd2 = new SqlCommand(query2);
cmd2.Connection = con;
rdr = cmd2.ExecuteReader();
while (rdr.Read())
{
sw.WriteLine(rdr["NumberOfProducts"]);
}
rdr.Close();
}
You haven't used apostrophes around the values. But you should use parameters anyway. You could use one query instead of one for every type. For example with this approach:
string sql = #"SELECT COUNT(PRODUCT_TYPE_NO) AS NumberOfProducts
FROM dbo.PRODUCT
Where PRODUCT_TYPE_NO IN ({0});";
string[] paramNames = PTNList.Select(
(s, i) => "#type" + i.ToString()
).ToArray();
string inClause = string.Join(",", paramNames);
using (SqlCommand cmd = new SqlCommand(string.Format(sql, inClause)))
{
for (int i = 0; i < paramNames.Length; i++)
{
cmd.Parameters.AddWithValue(paramNames[i], PTNList[i]);
}
// con.Open(); // if not already open
int numberOfProducts = (int) cmd.ExecuteScalar();
}
Update: maybe you really just want to loop them and get their count. Then you don't need this complex approach. But you should still use sql-parameters to prevent sql-injection and other issues like missing apostrophes etc.
You'll want to convert the column back to a type, e.g.
sw.WriteLine(rdr["NumberOfProducts"] as string);
Also, note that your query is prone to SqlInjection attacks and should be parameterized, and that SqlCommand is also disposable. You can squeeze a bit more performance by reusing the SqlCommand:
string query2 = #"SELECT COUNT(PRODUCT_TYPE_NO)
AS NumberOfProducts
FROM dbo.PRODUCT
Where PRODUCT_TYPE_NO = #ptn";
using (var cmd2 = new SqlCommand(query2))
{
cmd2.Connection = con;
cmd2.Parameters.Add("#ptn", SqlDbType.Varchar);
foreach (var ptn in PTNList)
{
cmd2.Parameters["#ptn"].Value = ptn;
Console.WriteLine("Running Query");
using var (rdr = cmd2.ExecuteReader())
{
if (rdr.Read())
{
sw.WriteLine(rdr["NumberOfProducts"] as string);
}
}
}
}
Are you sure your query give an result and sw.WriteLine is executed? I would redesign your code like this, becfause if you have an error in your data query, you might get into trouble. I always like to use this (schema):
IDataReader reader = null;
try
{
// create every thing...
}
catch(Exception ex)
{
// catch all exceptions
}
finally()
{
if(reader != null)
{
reader.Close();
}
}
And use the same for your connection, so that you can be sure, it is closed correct.
I have table that contains 350632 records currently. I have recently added a new column to the table which I am trying to populate using this code in C#:
List<int> listOfInts = new List<int>();
dbConnect.Open();
int counter = 1;
string toExecute = "select * from tempwords";
string insertQuery = "update tempwords set rownum=#toInsert";
using (SQLiteTransaction transaction = dbConnect.BeginTransaction())
{
using (SQLiteCommand newCommand = new SQLiteCommand(toExecute, dbConnect))
{
using (SQLiteDataReader reader = newCommand.ExecuteReader())
{
while (reader.Read())
{
listOfInts.Add(counter);
counter++;
}
}
}
transaction.Commit();
dbConnect.Dispose();
}
Console.WriteLine(listOfInts.Count.ToString());
dbConnect.Open();
int iterator = 0;
using (SQLiteTransaction transactionx = dbConnect.BeginTransaction())
{
using (SQLiteCommand command = new SQLiteCommand(insertQuery, dbConnect))
{
command.Transaction = transactionx;
while (iterator <= listOfInts.Count - 1)
{
command.Parameters.AddWithValue("#toInsert", listOfInts[iterator]);
command.ExecuteNonQuery();
iterator++;
Console.WriteLine((iterator + 1).ToString() + Environment.NewLine);
}
}
transactionx.Commit();
dbConnect.Dispose();
}
I think the logic is fine and it would all be done properly but the update is so slow(even though I have an index onn the rownum column). Is there any way I can speed it up to some realistic time?
Thanks in advance.
This command:
update tempwords set rownum=#toInsert
updates all 360632 rows (with the same value).
When you execute this command 360632 times, you end up updating 122942799424 rows.
If you want to update only a single row with each command execution, you have to tell the database which row that is:
update tempwords set rownum = #toInsert where _id = #id_of_the_row