I'm using a C# function to query the DB based on SqlConnection, SqlCommand and SqlDataAdapter but performances are quite bad when I try to retrieve a huge number of rows (few millions).
using (SqlConnection mySqlConnection = new SqlConnection(connectionString))
using (SqlCommand mySqlCommand = new SqlCommand(query, mySqlConnection)){
mySqlConnection.Open();
DataSet myDataSet = new DataSet();
using (SqlDataAdapter mySqlDataAdapter = new SqlDataAdapter(mySqlCommand)) {
affectedRow = mySqlDataAdapter.Fill(myDataSet);
}
}
Is there a way to optimize this query for big data set?
Just as a comparison this piece of VBA code requires only 4-5 versus 20-22 seconds of above C#
Set cnt = New ADODB.Connection
Set rst = New ADODB.Recordset
Set cmd = New ADODB.Command
cnt.Open
Set rst = cnt.Execute(queryString)
Dim nDimArray As Variant
nDimArray = rst.GetRows
cnt.Close
Don't use DataSet. Use a SqlReader which is similar to what you were doing in VBA.
using (var mySqlConnection = new SqlConnection(connectionString))
using (var mySqlCommand = new SqlCommand(query, mySqlConnection))
{
mySqlConnection.Open();
var reader = mySqlCommand.ExecuteReader();
while (reader.Read())
{
}
reader.Close();
}
More information here: https://msdn.microsoft.com/en-us/library/9kcbe65k%28v=vs.110%29.aspx
You should be able to do this as well, which will clean up after yourself just in case you have exceptions thrown during the read:
using (var mySqlConnection = new SqlConnection(connectionString))
{
using (var mySqlCommand = new SqlCommand(query, mySqlConnection))
{
mySqlConnection.Open();
using (var reader = mySqlCommand.ExecuteReader())
{
while (reader.Read())
{
}
}
}
}
Related
I would like to get some help : I can't get values form SQL Database to WPF textbox.
I tried by myself many codes that didn't worked, and this one looks fine but ".ToString" argument is wrong.
What should i put instead ?
using (SqlConnection con = new SqlConnection(MyConnString))
{
SqlCommand sqlCmd = new SqlCommand("SELECT DATE_A FROM Donnees_Accueil", con);
con.Open();
SqlDataReader sqlReader = sqlCmd.ExecuteReader();
while (sqlReader.Read())
{
hourA.Text = sqlReader["Date_A"].ToString;
}
sqlReader.Close();
}
Thank you in advance,
Zancrew.
Well, ToString() is a method, not property, that's why () is required; if you want to concat all the records:
using (SqlConnection con = new SqlConnection(MyConnString))
{
con.Open();
using (SqlCommand sqlCmd = new SqlCommand("SELECT DATE_A FROM Donnees_Accueil", con))
{
using (SqlDataReader sqlReader = sqlCmd.ExecuteReader())
{
StringBuilder sb = new StringBuilder();
while (sqlReader.Read())
{
sb.Append(Convert.ToString(sqlReader["Date_A"]));
}
hourA.Text = sb.ToString();
}
}
}
If you want to get the 1st record only:
using (SqlConnection con = new SqlConnection(MyConnString))
{
con.Open();
using (SqlCommand sqlCmd = new SqlCommand("SELECT DATE_A FROM Donnees_Accueil", con))
{
using (SqlDataReader sqlReader = sqlCmd.ExecuteReader())
{
if (sqlReader.Read())
hourA.Text = Convert.ToString(sqlReader["Date_A"]);
else
hourA.Text = "";
}
}
}
I tried to get data in a drop down list and it's not working. I don't understand what's the problem.
string connString = #" Data Source=(LocalDB)\v11.0;AttachDbFilename='C:\Users\oshri\Documents\Stock scores.mdf';Integrated Security=True;Connect Timeout=30";
string queryLecturer = "select name_student from student";
SqlConnection conn = new SqlConnection(connString);
//SqlCommand cmdL = new SqlCommand(queryLecturer, conn);
conn.Open();
//SqlCommand SQLCommand = new SqlCommand();
//cmdL.CommandType = CommandType.Text;
//SQLCommand.CommandText = queryLecturer;
//conn.Close();
SqlDataAdapter adapter = new SqlDataAdapter(queryLecturer, conn);
adapter.Fill(subjects);
DropDownListID.DataSource = subjects;
DropDownListID.DataBind();
DropDownListID.DataBind();
conn.Close();
You are assigning a DataSet containing DataRowView items to your drop down. Your drop down (is it a System.Windows.Forms.ComboBox?) is not smart enough to extract the real values from this DataSet. Instead, use a SqlDataReader to read your string values and add them to a list that you can use as data source for your drop down.
string connString = #"Data Source=...";
string queryLecturer = "select name_student from student";
using (var conn = new SqlConnection(connString))
using (var cmd = new SqlCommand(queryLecturer)) {
conn.Open();
using (SqlDataReader reader = cmd.ExecuteReader()) {
var list = new List<string>();
while (reader.Read()) {
list.Add(reader.GetString(0)); // 0 is the column index.
}
DropDownListID.DataSource = list;
}
}
The using statements automatically close the connection and dispose resources at the end of the statement blocks. They do so even if the statement blocks are left prematurely because of an exception or because of a break or return statement.
I am keep trying and still don't know how to fix this, keep getting same error. I want to get data from the database to txt file.
using (conn)
{
conn.Open();
SqlCommand cmd = new SqlCommand(command)
cmd.Connection = (SqlConnection)conn;
using (cmd)
{
SqlDataReader reader = cmd.ExecuteReader();
using(reader)
{
while (reader.Read())
{
dt.Load(reader);
}
}
using (StreamWriter sw = new StreamWriter(txtPath.Text + fileName))
{
// write to text file from Datatable dt
} }
Try insted your code, something like this:
DataTable myDataTable = new DataTable();
using (SqlConnection myConnection = new SqlConnection(yourDBconn)
{
yourDBconn.Open();
using (SqlDataAdapter myAdapter = new SqlDataAdapter(strYourQuery, yourDBconn))
{
myAdapter.Fill(myDataTable);
}
}
}
Here you are. Tested.
var customers = new List<Tuple<int, string>>(); // the list of ID, Name
using (var con = new SqlConnection("your connection string"))
{
using(SqlCommand cmd = new SqlCommand("select ID, Name from Customer", con))
{
con.Open();
using(SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
customers.Add(new Tuple<int, string>(
(int)reader["ID"], reader["Name"].ToString()));
}
}
}
}
// Store customers to file
Is this the following code healthy? Or I don't need to use the using keyword as the SqlDataAdapter will handle closing the connection?
public static DataSet Fetch(string sp, SqlParameter [] prm)
{
using (SqlConnection con = new SqlConnection(ConStr))
{
using (SqlCommand cmd = con.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = sp;
cmd.Parameters.AddRange(prm);
using (SqlDataAdapter dta = new SqlDataAdapter(cmd))
{
DataSet dst = new DataSet();
dta.Fill(dst);
return dst;
}
}
}
}
#MarkGravell I need a suggestions here, I am really looking to use DataReader, but I was looking all the time to use the using keyword to ensure closing the connections. Where with DataReader we can not use it because it will close the connection if we want to return the DataReader back to some method.
So do you think the following technique is fine with DataReader and the using keyword:
public static SqlDataReader Fetch(string sp, SqlParameter [] prm)
{
SqlCommand cmd = new SqlConnection(ConStr).CreateCommand();
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = sp;
cmd.Parameters.AddRange(prm);
cmd.Connection.Open();
return cmd.ExecuteReader(CommandBehavior.CloseConnection);
}
using (SqlDataReader dtrPrize = Sql.Fetch("SelectPrize", new SqlParameter[] { new SqlParameter("id", id) }))
{
dtrPrize.Read();
Prize prize = new Prize();
prize.id = (int)dtrPrize[dtrPrize.GetOrdinal("id")];
prize.artitle = (string)dtrPrize[dtrPrize.GetOrdinal("artitle")];
prize.entitle = (string)dtrPrize[dtrPrize.GetOrdinal("entitle")];
prize.ardetail = (string)dtrPrize[dtrPrize.GetOrdinal("ardetail")];
prize.endetail = (string)dtrPrize[dtrPrize.GetOrdinal("endetail")];
prize.image = (string)dtrPrize[dtrPrize.GetOrdinal("image")];
prize.theme = (string)dtrPrize[dtrPrize.GetOrdinal("theme")];
prize.price = (int)dtrPrize[dtrPrize.GetOrdinal("price")];
prize.audience = (int)dtrPrize[dtrPrize.GetOrdinal("audience")];
prize.type = (byte)dtrPrize[dtrPrize.GetOrdinal("type")];
prize.status = (byte)dtrPrize[dtrPrize.GetOrdinal("status")];
prize.voucher = (string)dtrPrize[dtrPrize.GetOrdinal("voucher")];
prize.supplierid = (int)dtrPrize[dtrPrize.GetOrdinal("supplierid")];
prize.created = (DateTime)dtrPrize[dtrPrize.GetOrdinal("created")];
prize.updated = (DateTime)dtrPrize[dtrPrize.GetOrdinal("updated")];
return prize;
}
Healthy-ish; personally I'd say the unhealthy bit is the bit where it makes use of DataSet and DataAdapter, but that is perhaps just my personal bias.
Yes, you should dispose the adapter etc here (which is what the using does for you, obviously).
As a trivial pointless tidy, you can stack the usings - just makes it a little less verbose:
using (SqlConnection con = new SqlConnection(ConStr))
using (SqlCommand cmd = con.CreateCommand())
{
It will be enough to leave just the first using (the one on the Connection) because disposing the connection will dispose everything you need disposed.
However, there is no harm disposing everything, just a bit more code.
Or what's the recommended way to read and write at the same time? Having two connections open is the only way? This is on a plain Windows Forms C# (.NET 3.5) app
using (SqlConnection conn = new SqlConnection(connStr)) {
SqlCommand cmdRead = new SqlCommand("select stools from foo", conn);
SqlDataReader rdr = cmdRead.ExecuteReader();
SqlCommand cmdWrite = new SqlCommand("insert into bar values (#beer)", conn);
SqlParameter beer = new SqlParameter("#beer", SqlDbType.Int);
cmdWrite.Parameters.Add(beer);
while(rdr.Read()) {
int stools = rdr.GetInt32(0);
cmdWrite.Parameters["#beer"].value = stools;
//Next line fails for an open data reader associated to the "command"
cmdWrite.ExecuteNonQuery();
}
rdr.Close()
}
This, OTOH, works, but looks ugly to me (besides of opening an extra connection)
using (SqlConnection connR = new SqlConnection(connStr)) {
using (SqlConnection connW = new SqlConnection(connStr)) {
SqlCommand cmdRead = new SqlCommand("select stools from foo", connR);
SqlDataReader rdr = cmdRead.ExecuteReader();
SqlCommand cmdWrite = new
SqlCommand("insert into bar values (#beer)", connW);
SqlParameter beer = new SqlParameter("#beer", SqlDbType.Int);
cmdWrite.Parameters.Add(beer);
while(rdr.Read()) {
int stools = rdr.GetInt32(0);
cmdWrite.Parameters["#beer"].value = stools;
cmdWrite.ExecuteNonQuery();
}
rdr.Close()
}
}
In this simple case, read all the stools, store them in a list, close the reader and then write them will work (as long as there aren't many stools in the database), but in more complex cases it starts being unwieldy and memory hungry, so that's also not desirable.
With SQL 2005 and later, you can use Multiple Active Result Sets (MARS):
http://msdn.microsoft.com/en-us/library/ms345109%28SQL.90%29.aspx
http://msdn.microsoft.com/en-us/library/ms131686.aspx
For this to work, you need to enable it in your connection string.
Why not having two methods: one for reading and one for writing, each using its own connection drawn from the connection pool.
Reading
using (SqlConnection conn = new SqlConnection(connStr))
using (SqlCommand cmdRead = new SqlCommand("select stools from foo", conn))
{
conn.Open();
using (SqlDataReader rdr = cmdRead.ExecuteReader())
{
while(rdr.Read())
{
int stools = rdr.GetInt32(0);
}
}
}
Writing
using (SqlConnection conn = new SqlConnection(connStr))
using (SqlCommand cmdWrite = new SqlCommand("insert into bar values (#beer)", conn))
{
conn.Open();
SqlParameter beer = new SqlParameter("#beer", SqlDbType.Int);
cmdWrite.Parameters.Add(beer);
cmdWrite.Parameters["#beer"].value = stools;
cmdWrite.ExecuteNonQuery();
}
And then just call them on different threads. This makes methods clearer and more specific.
You can't write while the DataReader is open, creating a second connection is needed in this situation, and IMHO it's not a design issue.
Well the obvious question is why are you doing this in code at all?
You're selecting from one table to write into another - that screams insert query to me.
Doubtless the real world example is more complex in which case you need two connections because datareaders work by holding the connection open 'til you're done and that's more or less that.