We Keep Coding

Sql Server Connection C#: max number of insert queries

I've a problem with SqlConnection in C#. I do a large number of INSERT NonQuery, but in any case SqlConnection save in the database always the first 573 rows. This is the method I use for queries. In this method there is a lock because I use different thread to save the data.
public void InsertElement(string link, string titolo, string text)
{
string conString = "*****************";
using (SqlConnection connection = new SqlConnection(conString))
{
connection.Open();
text = text.Replace("\"", "");
DateTime localDate = DateTime.Now;
lock (thisLock)
{
string query = "IF (NOT EXISTS(SELECT * FROM Result " +
" WHERE Link = '" + link + "')) " +
" BEGIN " +
" INSERT INTO Result ([Titolo],[Link],[Descrizione],[DataRicerca],[FKDatiRicercheID]) " +
" VALUES('" + titolo + "', '" + link + "', '" + text + "', '" + localDate + "', 1) " +
" END";
if (connection != null)
{
SqlCommand cmd = new SqlCommand(query, connection);
cmd.ExecuteNonQuery();
}
}
}
}
This is the code of the loop that call the method InsertElement()
public void Save()
{
string[] DatiLetti;
string url = "";
while (result.Count > 0)
{
try
{
url = result.Last();
result.RemoveAt(result.Count - 1);
DatiLetti = ex.DirectExtractText(url);
if (DatiLetti[0].Length > 2)
{
ssc.InsertGare(url, DatiLetti[0], DatiLetti[1]);
}
}
catch (Exception exc)
{
logger.Error("Exception SpiderSave> " + exc);
}
}
}
Result is a volatile array that is progressively filled from other thread. I'm sure that the array contains more than 573 items.
I try to search one solution, but all the answers say that the number of database connections for SQLServer is over 32K at a time and I've already checked this number in my database. Is there anyone who can help me understand the problem?

Don't open a connection for every insert. Use one connection, then pass that connection through to your insert, like this :
public void InsertElement(string link, string titolo, string text, SqlConnection conn)
{
text = text.Replace("\"", "");
DateTime localDate = DateTime.Now;
lock (thisLock)
{
string query = "IF (NOT EXISTS(SELECT * FROM Result " +
" WHERE Link = '" + link + "')) " +
" BEGIN " +
" INSERT INTO Result ([Titolo],[Link],[Descrizione],[DataRicerca],[FKDatiRicercheID]) " +
" VALUES('" + titolo + "', '" + link + "', '" + text + "', '" + localDate + "', 1) " +
" END";
if (connection != null)
{
SqlCommand cmd = new SqlCommand(query, connection);
cmd.ExecuteNonQuery();
}
}
}
I recommend also looking at paramatizing your query, as well as using bulk inserts, and not individual inserts

If you are executing InsertElement() once for each rows of data to insert, then the execution will be too slow for large no. of rows. (Also, you are creating SqlConnection for each query execution.) Try adding many rows at once using a single INSERT query:
INSERT INTO tablename
(c1,c2,c3)
VALUES
(v1,v2,v3),
(v4,v5,v6)
...

C# SQLite Database During Update

My SQLite query hangs then locks during my ExecuteNonQuery() in WriteToDB() below. It only seems to lock during the UPDATE and has no problem with the INSERT. This is only running in a single thread. When it hangs, I can see the journal being created in the SQLite database directory as if it keeps trying to write. It throws a SQLiteException with ErrorCode=5, ResultCode=Busy.
public String WriteToDB()
{
String retString = "";
//see if account exists with this email
String sql = "";
bool aExists = AccountExists();
if (!aExists)
{
sql = "INSERT INTO accounts (email, password, proxy, type, description) VALUES ('" + Email + "', '" + Password + "', '" + Proxy + "', 'dev', '" + Description + "');";
retString = "Added account";
}
else
{
sql = "UPDATE accounts SET password='" + Password + "', proxy='" + Proxy + "', description='" + Description + "' WHERE (email='" + Email + "' AND type='dev');";
retString = "Updated account";
}
using (SQLiteConnection dbconn = new SQLiteConnection("Data Source=" + Form1.DBNAME + ";Version=3;"))
{
dbconn.Open();
using (SQLiteCommand sqlcmd = new SQLiteCommand(sql, dbconn))
{
sqlcmd.ExecuteNonQuery(); //this is where it locks. Only on update.
}
}
return retString;
}
//Test to see if Email exists as account
public bool AccountExists()
{
int rCount = 0;
String sql = "SELECT COUNT(email) FROM accounts WHERE email='" + Email + "' AND type='dev';";
using (SQLiteConnection dbconn = new SQLiteConnection("Data Source=" + Form1.DBNAME + ";Version=3;"))
{
dbconn.Open();
using (SQLiteCommand sqlcmd = new SQLiteCommand(sql, dbconn))
{
rCount = Convert.ToInt32(sqlcmd.ExecuteScalar());
}
}
if (rCount > 0)
return true;
return false;
}

Oh man I feel dumb. I thought I posted all relevant code but all the code I posted works just fine. I had:
SQLiteDataReader dbReader = sqlcmd.ExecuteReader()
instead of
using (SQLiteDataReader dbReader = sqlcmd.ExecuteReader())
In another function. I thought it was an issue with the UPDATE because that was the place where the lock took place. Thanks for the responses and hopefully this reminds reminds everyone to use using() blocks with SQLite the first time!

An open data reader

i recieve this message when i run the action: "There is already an open DataReader associated with this Command which must be closed first."
my code is:public void UpdatePoints(string rightScore, string rightWinner)
{
cmd.CommandText = "select * from Users_Details";
cmd.Connection = connection;
connection.Open();
rdr = cmd.ExecuteReader();
while (rdr.Read())
{
int points=0;
string sql;
string hisScore = (string)rdr["lastbetscore"];
string hisWinner = (string)rdr["lastbetwinner"];
if (rightScore == hisScore)
points = points + 30;
if (rightWinner == hisWinner)
{
points = points + 20;
}
sql = "update Users_Details set lastgame_points='" + points + "', gamesplayed='" + ((int)rdr["gamesplayed"] + 1) + "',currentpoints='" + ((int)rdr["currentpoints"] + points) + "',pointsPG='" + (((int)rdr["currentpoints"] + points) / ((int)rdr["gamesplayed"] + 1)) + "' where username='" + (string)rdr["username"] + "'";
cmd.CommandText = sql;
cmd.ExecuteScalar();
}
rdr.Close();
connection.Close();
}

The error message is pretty specific about what you doing wrong. You cannot reuse the command or the connection for another command while you're reading data from it. You must firstly read all the data to some List or another data structure and then update db with each element of this List.
Also, consider to execute your statements in transaction
var transaction = connection.BeginTransaction();
...
transaction.Commit();
This will speed up your updates since transaction will be created and commited only once but other way transaction will be created implicitely on each update

You should create a new Command instance instead of reusing the old one here:
sql = "update Users_Details set lastgame_points='" + points + "', gamesplayed='" + ((int)rdr["gamesplayed"] + 1) + "',currentpoints='" + ((int)rdr["currentpoints"] + points) + "',pointsPG='" + (((int)rdr["currentpoints"] + points) / ((int)rdr["gamesplayed"] + 1)) + "' where username='" + (string)rdr["username"] + "'";
cmd.CommandText = sql;
cmd.ExecuteScalar();
In addition, read up on command parameters, formatting the sql string yourself opens you up to SQL injection attacks.

Can I use SQLCLR stored procedure to update a column of a database table ( using some compiled dll)

I wanted to update the values of a few columns of a database table, using queries or stored procedure, but wanted to use my C# library to alter the value.
For eg, I want the columns A,B,C of table T to be replaced with Encrypt(A), Encrypt(B) and Encrypt(C) where Encrypt is a part of a C# library. I could have done it in a simple console application, but I have to do this process for a lot of columns in lot of tables.
Could I use a SQLCLR stored procedure / query to do this process in SQL Server Management Studio? It will be really great if someone could assist in this.
public class SP
{
[Microsoft.SqlServer.Server.SqlFunction()]
public static void Enc()
{
using (SqlConnection connection = new SqlConnection("context connection=true"))
{
connection.Open();
SqlCommand command;
SqlCommand command1;
for (int i = 0; i < 1; i++)
{
command = new SqlCommand("SELECT " + tableFieldArray[i, 1].ToString() + " FROM " + tableFieldArray[i, 0].ToString(), connection);
SqlDataReader reader = command.ExecuteReader();
using (reader)
{
while (reader.Read())
{
if (!reader.IsDBNull(0) && !String.IsNullOrEmpty(reader.GetString(0)))
{
//SqlContext.Pipe.Send("Data = " + reader.GetString(0) + "; Encrypted = " + Encrypt(reader.GetString(0)));
SqlContext.Pipe.Send("UPDATE " + tableFieldArray[i, 0].ToString() + " SET "
+ tableFieldArray[i, 1].ToString() + " = '" + Encrypt(reader.GetString(0)) + "' "
+ "WHERE " + tableFieldArray[i, 1].ToString() + " = '" + reader.GetString(0) + "'");
//query = "UPDATE " + tableFieldArray[i, 0].ToString() + " SET "
// + tableFieldArray[i, 1].ToString() + " = '" + Encrypt(reader.GetString(0)) + "' "
// + "WHERE " + tableFieldArray[i, 1].ToString() + " = '" + reader.GetString(0) + "'";
command1 = new SqlCommand("UPDATE " + tableFieldArray[i, 0].ToString() + " SET "
+ tableFieldArray[i, 1].ToString() + " = '" + Encrypt(reader.GetString(0)) + "' "
+ "WHERE " + tableFieldArray[i, 1].ToString() + " = '" + reader.GetString(0) + "'",connection);
}
}
}
SqlCommand command1 = new SqlCommand(query , connection);
command1.ExecuteNonQuery();
}
connection.Close();
}
}
public static string Encrypt(string TextFromForm)
{
//implementation
}
}
}

You can use SQLCLR to call encryption from C#, though this is the wrong approach. If you need to do a custom algorithm, you should encapsulate that into a SQLCLR function so that it can be used in an UPDATE statement or even an INSERT or SELECT or anywhere. Something like:
public class SP
{
[Microsoft.SqlServer.Server.SqlFunction(IsDeterministic = true)]
public static SqlString EncryptByAES(SqlString TextToEncrypt)
{
return DoSomething(TextToEncrypt.Value);
}
}
Then you can use that function as follows:
UPDATE tb
SET tb.FieldA = EncryptByAES(tb.FieldA)
FROM dbo.TableName tb
WHERE tb.FieldA some_test_to_determine_that_FieldA_is_not_alreay_encrypted;
BUT, before you write a custom encryption algorithm, you might want to check out the several built-in paired ENCRYPTBY / DECRYPTBY functions that might do exactly what you need:
ENCRYPTBYASYMKEY / DECRYPTBYASYMKEY
ENCRYPTBYCERT / DECRYPTBYCERT
ENCRYPTBYKEY / DECRYPTBYKEY
ENCRYPTBYPASSPHRASE / DECRYPTBYPASSPHRASE

Error while using MySqlTransaction for multiple inserts

I have a form in windows where I am doing insert statement for header and detail.
I am using MySqlTransaction for the form. When there is no error in header and detail the transaction gets committed but when there is an error in insert query of detail then the following error comes while rollback
There is already an open DataReader associated with this Connection
which must be closed first.
Here is my code.
public string Insert_Hardening_Measures_HdrANDDtl(BL_Vessel_Hardening_Measures objHdr, List<BL_Vessel_Hardening_Measures> objDtl)
{
string success = "true";
string success1 = "";
MySqlConnection MySqlConnection1 = new MySqlConnection(strCon);
MySqlConnection1.Open();
MySqlTransaction MyTransaction = MySqlConnection1.BeginTransaction();
MySqlCommand MyCommand = new MySqlCommand();
MyCommand.Transaction = MyTransaction;
MyCommand.Connection = MySqlConnection1;
try
{
MyCommand.CommandText = "insert into hardening_measures_hdr (Hardening_Measures_Hdr_id,Month,Year) values (" + objHdr.Hardening_Measures_Hdr_id + ",'" + objHdr.Month + "','" + objHdr.Year + "')";
MyCommand.ExecuteNonQuery();
for (int i = 0; i < objDtl.Count; i++)
{
MyCommand.CommandText = "insert into hardening_measures_dtl (Hardening_Measures_Dtl_id,Hardening_Measures_Hdr_id,Hardening_Measures_Mst_id,Value) values (" + objDtl[i].Hardening_Measures_Dtl_id + "," + objDtl[i].Hardening_Measures_Hdr_id + ",'" + objDtl[i].Hardening_Measures_Mst_id + ",'" + objDtl[i].Value + "')";
MyCommand.ExecuteNonQuery();
}
MyTransaction.Commit();
MySqlConnection1.Close();
}
catch
{
MyTransaction.Rollback();
}
return success;
}
Anybody who have come through this kind of problem please suggest something