Sql query: use where in or foreach? - c#

I'm using query, where the piece is:
...where code in ('va1','var2'...')
I have about 50k of this codes.
It was working when I has 30k codes, but know I get:
The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partition
I think that problem is related with IN...
So now I'm planning use foreach(string code in codes)
...where code =code
Is it good Idea ??

I suggest you create a temporary table containing all the codes you want to match, and join against that instead.
However, we haven't really seen enough of your code to comment for sure.

First, create a temp table, call it #tmp0, with just one column. Then:
SqlConnection conn = new SqlConnexion(connection_string);
SqlCommand cmd = new SqlCommand("INSERT INTO #tmp0 (code) VALUE (#code)", conn);
conn.Open();
foreach (string s in bigListOfCodes)
{
cmd.Parameters.Clear();
cmd.Parameters.AddWithValue("#code", s);
cmd.ExecuteNonQuery();
}
cmd = new SqlCommand("SELECT * FROM tbl " +
"WHERE code IN (SELECT CODE FROM #tmp0)", conn);
SqlDataReader rs = cmd.ExecuteReader();
while (rs.Read())
{
/* do stuff */
}
cmd = new SqlCommand("DROP TABLE #tmp0", conn);
cmd.ExecuteNonQuery();
conn.Close();
I know this seems like a lot of work for the server, but it's really pretty fast.

I'm not sure where you got those 50k values, but if it is from another query, just JOIN to this table and get all the data at one time from one query.

Related

How do I delete an entire row from a database

How would I delete a row from a sql database, either with stored procedures or without, right now I have tried without, using a button press.
This is what I have so far, _memberid has been sent over from a differnt form from the database(For context).
private void btnDelete_Click(object sender, EventArgs e)
{
SqlCommand cmd = new SqlCommand();
cmd.Connection = Lib.SqlConnection;
cmd.CommandType = CommandType.Text;
cmd.CommandText = "Delete * From Members where MemberId = " + _memberId;
SqlDataAdapter adapter = new SqlDataAdapter();
adapter.DeleteCommand = cmd;
adapter.Fill(MembersDataTable); // Im fairly sure this is incorrect but i used it from old code
DialogResult = DialogResult.OK;
}
If you're trying to do a simple ADO.Net-based delete, then it would be somehting like his:
private void DeleteById(int memberId)
{
// or pull the connString from config somewhere
const string connectionString = "[your connection string]";
using (var connection = new SqlConnection(connectionString))
{
connection.Open();
using (var command = new SqlCommand("DELETE FROM Members WHERE MemberId = #memberId", connection))
{
command.Parameters.AddWithValue("#memberId", memberId);
command.ExecuteNonQuery();
}
}
Use parameter to prevent SQL injection.
There are essentially three main things I'm seeing...
One
You don't need the * in the query. DELETE affects the whole row, so there's no need to specify columns. So just something like:
DELETE FROM SomeTable WHERE SomeColumn = 123
Two
There's no need for a SqlDataAdapter here, all you need to do is execute the query. For example:
cmd.ExecuteNonQuery();
The "non query" is basically a SQL command which doesn't query data for results. Inserts, updates, and deletes are generally "non queries" in this context. What it would return is simply the number of rows affected, which you can use to double-check that it matches what you expect if necessary.
Three
Don't do this:
cmd.CommandText = "Delete From Members where MemberId = " + _memberId;
This kind of string concatenation leads to SQL injection. While it looks intuitively like you're using _memberId as a query value, technically you're using it as executable code. It's less likely (though not impossible) to be a problem for numeric values, but it's a huge problem for string values because it means the user can send you any string and you'll execute it as code.
Instead, use query parameters. For example, you might do something like this:
cmd.CommandText = "Delete From Members where MemberId = #memberId";
cmd.Parameters.Add("#memberId", SqlDbType.Int);
cmd.Parameters["#memberId"].Value = _memberId;
This tells the database engine itself that the value is a value and not part of the executing query, and the database engine knows how to safely handle values.
You could use a DataAdapter, but since you aren't using a datatable, it's just easier to do it without like this:
var sql = "DELETE FROM Members WHERE MemberId=#MemberId";
using(var cmd = new SqlCommand(sql, Lib.SqlConnection))
{
cmd.Connection.Open();
cmd.Parameters.Add("#MemberId",SqlDbType.Int).Value = _memberId;
cmd.ExecuteNonQuery();
}
And if you are using Dapper, you can do this:
Lib.SqlConnection.Execute("DELETE FROM Members WHERE MemberId=#MemberId", new {MemberId=_memberId});
If you are still using DataTables, I would highly recommend you look into using this (or something like this) to simplify your database accesses. It'll make CRUD logic on a database a breeze, and your code will me a lot more maintainable because you can get rid of all the odd needs to do casting, boxing/unboxing, and reduce the chances of runtime bugs because of the use of magic strings that happens so often with DataTables (column names). Once you start working with POCO classes, you'll hate having to use DataTables. That said, there are a few places where DataTables are a better solution (unknown data structures, etc), but those are usually pretty rare.

C# Sql Command for Insering into a database

I am new to Visual C# programming language and recently i was trying to make a application that is supposed to insert into a local database of users some data but every times my code runs and the insertion works fine the database does not update.This is the code that i am using
try
{
cn.Open();
SqlCommand insert = new SqlCommand();
insert.CommandText = "insert into Clienti (Nume,Prenume,Parola,Email) values(#Nume,#Prenume,#Parola,#Email)";
insert.Connection = cn;
insert.Parameters.AddWithValue("#Nume", register_nume.Text);
insert.Parameters.AddWithValue("#Prenume", register_prenume.Text);
insert.Parameters.AddWithValue("#Parola", register_password.Text);
insert.Parameters.AddWithValue("#Email", register_email.Text);
insert.ExecuteNonQuery();
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
MessageBox.Show("Added succesfully");
}
catch (Exception ex)
{
MessageBox.Show(""+ex);
}
I already tried the property Copy to output and it doesn't seems to work.
I am sorry for any grammar mistakes that i made,I would be grateful for any help.
insert.ExecuteNonQuery();
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
You only need the ExecuteNonQuery, it will run the INSERT. You need to use ExecuteReader instead only when you're running a statement that produces result sets (eg. SELECT). So it should be:
insert.ExecuteNonQuery();
Its because you have to now read the database using a select statement, you cant use an INSERT SQL statement to read.
You could add the following immediately after your insert.
using(var selectCmd = new SqlCommand("SELECT Nume,Prenume,Parola,Email FROM Clienti WHERE Nume = #Nume", cn))
{
selectCmd.Parameters.AddWithValue("#Nume", register_nume.Text);
using(SqlDataReader reader = selectCmd.ExecuteReader())
{
while (reader.Read()) { }
}
}
That said if you want to know IF the row was inserted or how many records were inserted ExecuteNonQuery returns the number of rows affected. You could change that part of the code like this:
var recordsAffected = insert.ExecuteNonQuery();
if(recordsAffected > 0)
MessageBox.Show("Added succesfully");
else
MessageBox.Show("Nothing happened");
Although in this particular case it would not make sense because if nothing was inserted it would probably be caused by an Exception.
Some side notes
Always wrap types that implement IDisposable in using blocks (see code above as example). It ensures that resources are always released as soon as you are done with them even if an Exception is thrown.
Never swallow Exceptions! Either recover from one and log it or do not catch it at all. If you swallow it you will never know if/why your code broke.
This part of your code is preparing the SQL command that you are running
cn.Open();
SqlCommand insert = new SqlCommand();
insert.CommandText = "insert into Clienti (Nume,Prenume,Parola,Email) values(#Nume,#Prenume,#Parola,#Email)";
insert.Connection = cn;
insert.Parameters.AddWithValue("#Nume", register_nume.Text);
insert.Parameters.AddWithValue("#Prenume", register_prenume.Text);
insert.Parameters.AddWithValue("#Parola", register_password.Text);
insert.Parameters.AddWithValue("#Email", register_email.Text);
The SQL command that your code is running, is an INSERT statement, which only adds a new record to your table.
This statement runs the command that you setup earlier:
insert.ExecuteNonQuery();
If I understand correctly, here you are trying to read the data from the table again:
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
Problem is, your command is NOT set for reading. To read data, you need to use a SELECT statement. Something like this:
insert.CommandText = "SELECT Nume,Prenume,Parola,Email from Clienti" ;
So, to read the data after executing the insert, you should do this:
insert.CommandText = "SELECT Nume,Prenume,Parola,Email from Clienti" ;
SqlDataReader reader = insert.ExecuteReader();
while (reader.Read()) { }
You ar executing 2 times the query:
insert.ExecuteNonQuery();
SqlDataReader reader = insert.ExecuteReader();
Try to get the affected rows
int rows = insert.ExecuteNonQuery();
I would suggest creating a stored procedure in your database and just execute the SP.
It's always better to execute SP from code and leave the SQL programming in the DB.

Query is much faster in c# above System.Data.SQLite if first load SQLite table into .NET DataTable and then use DataTable.Select. why?

I have tables in SQLite database.
I use C# and System.Data.SQLite DLL to access the database.
My test shows that the following query is very slow:
string sql = "select column1, column2 from table_1 where column1=1 and column2=2"
SQLiteCommand mycommand = new SQLiteCommand(cnn);
mycommand.CommandText = sql;
SQLiteDataReader reader = mycommand.ExecuteReader();
tb.Load(reader);
reader.Close();
But if I first load the whole table into .NET DataTable like below, and then use DataTable.Select(), it's is much faster:
string sql = #"SELECT * FROM '" + data_table_name + "'" + ";\n";
SQLiteCommand mycommand = new SQLiteCommand(cnn);
mycommand.CommandText = sql;
SQLiteDataReader reader = mycommand.ExecuteReader();
tb.Load(reader);
reader.Close();
DataRow[] rows = tb.Select("column1=1 AND column2=2");
The difference is even more significant if have multiple queries after loading the whole table.
This is essentially cache the whole table in .NET. I tried different kind of tables with different sizes and primary keys, all behave the same.
For now, I know we would use such optimization, but wondering if there are other workaround.
Also, I figured out that if the index of a table is string rather than integer, as long as the string is short (tested up to 10 characters string), the direct SQL query is actually faster without caching the table. However, after caching the table, using integer index is faster than string index.
Anybody can explain the behavior?

Syntax error while trying to fetch data from MySql

So I am trying to fetch a value from the database, selecting the row using WHERE INT.
conn = new MySqlConnection(DBdetails.connStr);
conn.Open();
query = "SELECT * FROM tables WHERE table=#tafel";
MySqlCommand cmd = new MySqlCommand(query, conn);
cmd.Parameters.AddWithValue("#tafel", tafel);
cmd.ExecuteNonQuery();
However it wont pass 'cmd.ExecuteNonQuery()', it throws a error saying the syntax isnt right like: "near table=1", "near table=2"
I tried fetching a other one in the same table that is a var char and it worked perfectly.
Don't really see what I am doing wrong. The 'table' column is a int and 'tafel' is a int to.
Thanks!
Put your field name table in backticks (table is a reserved word in MySQL) :
query = "SELECT * FROM `tables` WHERE `table` = #tafel";
As others said, table is a reserved word in MySQL. You need to use quote with it like
query = "SELECT * FROM tables WHERE `table` = #tafel";
However, the best solution is to change the name to a nonreserved word.
Also use using statement to dispose your MySqlConnection and MySqlCommand like;
using(MySqlConnection conn = new MySqlConnection(DBdetails.connStr))
using(MySqlCommand cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT * FROM tables WHERE `table` = #tafel";
cmd.Parameters.AddWithValue("#tafel", tafel);
conn.Open();
cmd.ExecuteNonQuery();
}
By the way, I don't understand why you use ExecuteNonQuery with SELECT statement. It just executes your query. It doesn't even return any value.
If you want to get the result of your query, you can use ExecuteReader method which returns SqlDataReader as your result rows.

How do I count the number of rows returned in my SQLite reader in C#?

I'm working in Microsoft Visual C# 2008 Express and with SQLite.
I'm querying my database with something like this:
SQLiteCommand cmd = new SQLiteCommand(conn);
cmd.CommandText = "select id from myTable where word = '" + word + "';";
cmd.CommandType = CommandType.Text;
SQLiteDataReader reader = cmd.ExecuteReader();
Then I do something like this:
if (reader.HasRows == true) {
while (reader.Read()) {
// I do stuff here
}
}
What I want to do is count the number of rows before I do "reader.Read()" since the number returned will affect what I want/need to do. I know I can add a count within the while statement, but I really need to know the count before.
Any suggestions?
The DataReader runs lazily, so it doesn't pick up the entirety of the rowset before beginning. This leaves you with two choices:
Iterate through and count
Count in the SQL statement.
Because I'm more of a SQL guy, I'll do the count in the SQL statement:
cmd.CommandText = "select count(id) from myTable where word = '" + word + "';";
cmd.CommandType = CommandType.Text;
int RowCount = 0;
RowCount = Convert.ToInt32(cmd.ExecuteScalar());
cmd.CommandText = "select id from myTable where word = '" + word + "';";
SQLiteDataReader reader = cmd.ExecuteReader();
//...
Note how I counted *, not id in the beginning. This is because count(id) will ignore id's, while count(*) will only ignore completely null rows. If you have no null id's, then use count(id) (it's a tad bit faster, depending on your table size).
Update: Changed to ExecuteScalar, and also count(id) based on comments.
What you request is not feasible -- to quote Igor Tandetnik, my emphasis:
SQLite produces records one by one, on request, every time you call sqlite3_step.
It simply doesn't know how many there are going to be, until on some sqlite3_step
call it discovers there are no more.
(sqlite3_step is the function in SQLite's C API that the C# interface is calling here for each row in the result).
You could rather do a "SELECT COUNT(*) from myTable where word = '" + word + "';" first, before your "real" query -- that will tell you how many rows you're going to get from the real query.
Do a second query:
cmd.CommandText = "select count(id) from myTable where word = '" + word + "';";
cmd.CommandType = CommandType.Text;
SQLiteDataReader reader = cmd.ExecuteReader();
Your reader will then contain a single row with one column containing the number of rows in the result set. The count will have been performed on the server, so it should be nicely quick.
If you are only loading an id column from the database, would it not be easier to simply load into a List<string> and then work from there in memory?
Normally i would do
select count(1) from myTable where word = '" + word + "';";
to get the result as fast as possible. In the case where id is an int then it won't make much difference. If it was something a bit bigger like a string type then you'll notice a difference over a large dataset.
Reasoning about it count(1) will include the null rows. But i'm prepared to be corrected if i'm wrong about that.
Try this,
SQLiteCommand cmd = new SQLiteCommand(conn);
cmd.CommandText = "select id from myTable where word = '" + word + "';";
SQLiteDataReader reader = cmd.ExecuteReader();
while (reader.HasRows)
reader.Read();
int total_rows_in_resultset = reader.StepCount;
total_rows_in_resultset gives you the number of rows in resultset after processing query
remember that if you wanna use the same reader then close this reader and start it again.
Here is my full implementation in a static method.
You should be able to plug this into your class (replace _STR_DB_FILENAME & STR_TABLE_NAME with your database file name and table name).
/// <summary>
/// Returns a count of the number of items in the database.
/// </summary>
/// <returns></returns>
public static int GetNumberOfItemsInDB()
{
// Initialize the count variable to be returned
int count = 0;
// Start connection to db
using (SqliteConnection db =
new SqliteConnection("Filename=" + _STR_DB_FILENAME))
{
// open connection
db.Open();
SqliteCommand queryCommand = new SqliteCommand();
queryCommand.Connection = db;
// Use parameterized query to prevent SQL injection attacks
queryCommand.CommandText = "SELECT COUNT(*) FROM " + _STR_TABLE_NAME;
// Execute command and convert response to Int
count = Convert.ToInt32(queryCommand.ExecuteScalar());
// Close connection
db.Close();
}
// return result(count)
return count;
}
Note: To improve performance, you can replace '' in "SELECT COUNT()…." with the column name of the primary key in your table for much faster performance on larger datasets.
but I really need to know the count before
Why is that ? this is usually not necessary, if you use adequate in-memory data structures (Dataset, List...). There is probably a way to do what you want that doesn't require to count the rows beforehand.
You do have to count with select count... from...
This will make your application slower. However there is an easy way to make your app faster and that way is using parameterized queries.
See here: How do I get around the "'" problem in sqlite and c#?
(So besides speed parameterized queries have 2 other advantages too.)
SQLiteCommand cmd = new SQLiteCommand(conn);
cmd.CommandText = "select id from myTable where word = '" + word + "';";
SQLiteDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
total_rows_in_resultset++;
}
Surely a better way to get a row count would be something like this:-
SQLiteDataReader reader = SendReturnSQLCommand(dbConnection, "SELECT COUNT(*) AS rec_count FROM table WHERE field = 'some_value';");
if (reader.HasRows) {
reader.Read();
int count = Convert.ToInt32(reader["rec_count"].ToString());
...
}
That way you don't have to iterate over the rows

Categories