I am using the following code do get all data records from a MS SQL database and I try to update every single record. The code is used in a WebService. The issue is, that the code runs fine if I have 1000 data records but now I have 20000 data records an the code first returned with an timeout. Then I set the cmd.CommandTimeout to zero to have no timeout. Now when I invoke the function in the IE WebSvc the IE window is still blank and still try to load something but nothing happens. Only 150 datarecords are updated.
Do you have any idea where the issue might be ? Is the code not the best, so what should I change ?
Thank you very much!
WorldSignia
MyCode:
private string AddNewOrgBez()
{
try
{
SqlConnection sqlconn = new SqlConnection(this.connectionString);
SqlCommand cmd;
SqlDataReader reader;
sqlconn.Open();
cmd = new SqlCommand("SELECT * FROM dbo.mydata", sqlconn);
cmd.CommandTimeout = 0;
reader = cmd.ExecuteReader();
while (reader.Read())
{
// Felder holen
string okuerzel = reader["O_KURZ"].ToString();
string bezeichnung = reader["O_BEZ"].ToString();
string[] lines = CreateNewOrgBez(bezeichnung);
string sqlcmd = "UPDATE dbo.mydata SET WEB_OBEZ1 = '" + lines[0] + "', WEB_OBEZ2 = '" + lines[1] + "', WEB_OBEZ3 = '" + lines[2] + "' WHERE O_KURZ = '" + okuerzel + "'";
SqlConnection sqlconn2 = new SqlConnection(this.connectionString);
sqlconn2.Open();
SqlCommand cmd2 = new SqlCommand(sqlcmd, sqlconn2);
cmd2.CommandTimeout = 0;
cmd2.ExecuteNonQuery();
sqlconn2.Close();
}
reader.Close();
sqlconn.Close();
return "OK";
}
catch (Exception ex)
{
return ex.Message;
}
}
You are leaking every SqlCommand here - I suggest you review your use of SqlClient classes to find the ones that are IDisposable and restructure your code to ensure they are always freed, using the using construct.
For example, this ensures Dispose gets called even if there is an exception in the bracketed code:
using (SqlCommand cmd2 = new SqlCommand(sqlcmd, sqlconn2))
{
cmd2.CommandTimeout = 0;
cmd2.ExecuteNonQuery();
}
Using a new SqlConnection for every UPDATE is expensive too, this should be done outside the loop. Redundant connection establishment is likely the explanation for your timeout.
Take note of #ck's comment that for efficiency this type of piecemeal client-side operation is not as good as doing the heavy lifting server-side. You should be able to get this code working better, but that does not mean it's the ideal/fastest solution.
I found the issue.
You need first to get all the data records, for example in a new DataTable. The structure I used does not work, because it reads data from the database and also updates the database. After changing it to a new structure it works.
You were using two different connections to read and to update, and one of them was blocking another. This is why when you read all your data first, it began to work.
I doubt if your running into OutOfMemoryException. Can you profile your application and check the memory usage?
Since you are just overwriting the variables in While loop, why don't you try taking them out of the loop.
Related
I am developing an asp.net web application and I am trying to add a user xp system to it. I have a SQL Server database connected to it and I am trying to make a function that will give 5 experience points to the user.
I queried to the user that is logged in, accessed the user_xp column, and I am trying to add +5 to the old session variable for xp, then send that back into the database to be stored. Here is my code, I am not sure what is wrong with it.
void generateXp()
{
try
{
SqlConnection con = new SqlConnection(strcon);
if (con.State == ConnectionState.Closed)
{
con.Open();
}
SqlCommand cmd = new SqlCommand("UPDATE member_master_tbl SET user_xp = #user_xp WHERE " +
"user_name = '" + Session["username"].ToString().Trim() + "'", con);
int xp = 5;
int current_xp = Convert.ToInt32(Session["user_xp"]);
int new_xp = xp + current_xp;
string new_xp2 = Convert.ToString(new_xp);
cmd.Parameters.AddWithValue("user_xp", new_xp2);
}
catch (Exception ex)
{
}
}
Try renaming the SQL parameter to #user_xp.
cmd.Parameters.AddWithValue("#user_xp", new_xp2);
I don't have an accessible database to test. Also, you need to add the command to execute the query at the end.
cmd.ExecuteNonQuery()
That being said, it's a good practice to learn to separate DB queries to stored procedures or functions.
As others noted, you simply forgot to do a execute non query to run the command that you setup.
However, you can write things this way. You don't mention or note what the data type the experience points column is - I assumed "int".
So, your code block can be written this way:
using (SqlCommand cmd = new SqlCommand("UPDATE member_master_tbl SET user_xp = #user_xp WHERE user_name = #user",
new SqlConnection(strcon)))
{
cmd.Parameters.Add("#user_xp", SqlDbType.Int).Value = 5 + Session("user_xp");
cmd.Parameters.Add("#user", SqlDbType.NVarChar).Value = Session("username");
cmd.Connection.Open();
cmd.ExecuteNonQuery();
}
note how the command object has a connection object (so we don't need a separate one).
And while several people here "lamented" the string concentration to build the sql and warned about sql injection?
Actually, the introduction of # parameters for both values cleans up the code. So you get nice parameters - nice type checking, and you don't have to remember to add/use/have things like quotes around teh string, but not for numbers.
And I let .net cast the number expression from session() - this also likely is ok.
Also the "using block" also correctly cleans up the command object and also the connection object - so the using block is a good idea here.
I have a post action method which takes in a username and password from a form. In this action method I am using a function to execute a mysql command to a database
This is the function to read from the database
public static MySqlDataReader GetDataFromDB(string command)
{
using (MySqlConnection con = new MySqlConnection(ConStr))
{
com = new MySqlCommand(command, con);
con.Open();
MySqlDataReader rdr1 = com.ExecuteReader();
while (rdr1.Read())
{
}
return rdr1;
}
}
This is how i tried to use this function in my action method
MySqlDataReader rdr2 = Helper.GetDataFromDB("select * from library.Accounts where username = '" + username + "' and password = '" + password + "'");
I put a breakpoint in the helper method and could see that it was reading from the database fine with it storing all the expected row data in the internal 'rdr1' object however it isnt returning anything at the end as the 'rdr2' object is empty once it's finished. I don't know if something is wrong or if im just being an idiot and so i would appretiate if anyone can tell me where im going wrong.
A Data Reader such as MySqlDataReader is Forward Only. This means that once the data has been read and moved on to the next record, or the end of the data, it cannot go back to the beginning.
If you remove this block of code:
while (rdr1.Read())
{
}
You should find it will return the Data Reader at the start, and therefore be able to read the data you loaded from the database.
In addition, you should know that when the using () {...} block finishes the connection will be closed, along with the Data Readers ability to read the data.
If you do want to return the Data Reader, remove the using() {...} block to be a simple statement like:
MySqlConnection con = new MySqlConnection(ConStr);
If you do want to navigate backwards and forwards, filter or sort the data after it has been loaded, you should LOAD the data in to a DataTable from the Data Reader.
Example:
DataTable dt = new DataTable();
dt.Load(rdr1);
I have a WebService that updates my access table from some terminals (10).
When I try to update I get this error from the error log:
Could not Update; Currently locked
Some terminals succeed and some do not.
I update like this:
using (Conn = new OleDbConnection(Work_Connect))
{
Conn.Open();
foreach (DataRow R in ds.Tables["MyCount"].Rows)
{
U_ID = ID;
U_Bar = R["Bar"].ToString().Trim();
U_Qty = R["Qty"].ToString().Trim();
U_Des = R["Des"].ToString().Trim();
SQL = "INSERT INTO MyTbl(ID,Bar,Qty,Des)VALUES('";
SQL += Convert.ToInt32(ID) + "','" + U_Bar + "','" + Convert.ToDouble(U_Qty) + "','" + U_Des + "')";
OleDbCommand Cmd2 = new OleDbCommand(SQL, Conn);
Cmd2.CommandText = SQL;
Cmd2.ExecuteNonQuery();
}
}
GC.Collect();
return true;
MsAccess has serious drawbacks for multi-user update. The Jet engine is not a database server, and will manage concurrence based on file system locking. If your problem is with a web service, I'd move the update to the server part, and implement queuing of simultaneous requests there. Thus, only the server, one process, will have access to the Access data. The other option is to use a real database server that will do that work for you. SQL Server Express is the usual option because it's easy to integrate, it's free as in beer, and is solid.
Also, if your problem happens always from the same terminals, that is, some terminals can never update anything, check the file access rights of these terminals' users to the database file, the lock file, and the database and lock file directory. Write rights are required for all of them.
Suggestions:
Convert your query to a parameterized query to avoid any potential strangeness with quoting. (You are converting text to numbers and then enclosing them in single-quotes in the SQL statement. That makes no sense.)
Don't force garbage collection on each call. According to the MSDN article here: "It is possible to force garbage collection by calling Collect, but most of the time, this should be avoided because it may create performance issues."
Try something like this instead:
using (Conn = new OleDbConnection(Work_Connect))
{
Conn.Open();
foreach (DataRow R in ds.Tables["MyCount"].Rows)
{
U_ID = ID;
U_Bar = R["Bar"].ToString().Trim();
U_Qty = R["Qty"].ToString().Trim();
U_Des = R["Des"].ToString().Trim();
SQL = "INSERT INTO MyTbl (ID,Bar,Qty,Des) VALUES (?,?,?,?)";
using(OleDbCommand Cmd2 = new OleDbCommand(SQL, Conn))
{
// Cmd2.CommandText = SQL; redundant, the 'new' set the .CommandText
Cmd2.Parameters.AddWithValue("?", Convert.ToInt32(ID));
Cmd2.Parameters.AddWithValue("?", U_Bar);
Cmd2.Parameters.AddWithValue("?", Convert.ToDouble(U_Qty));
Cmd2.Parameters.AddWithValue("?", U_Des);
Cmd2.ExecuteNonQuery();
}
}
Conn.Close();
}
// GC.Collect(); // disabled for test purposes
return true;
I am trying to get column information in C# from a SQL table on SQL Server. I am following the example in this link: http://support.microsoft.com/kb/310107 My program strangely gets hung up when it tries to close the connection. If the connection is not closed, the program exits without any Exceptions. Here's my code:
SqlConnection connection = new SqlConnection(#"MyConnectionString");
connection.Open();
SqlCommand command = new SqlCommand("SELECT * FROM MyTable", connection);
SqlDataReader reader = command.ExecuteReader(CommandBehavior.KeyInfo); // If this is changed to CommandBehavior.SchemaOnly, the program runs fast.
DataTable table = reader.GetSchemaTable();
Console.WriteLine(table.Rows.Count);
connection.Close(); // Alternatively If this line is commented out, the program runs fast.
Putting the SqlConnection inside a using block also causes the application to hang unless CommandBehavior.KeyInfo is changed to CommandBehavior.SchemaOnly.
using (SqlConnection connection = new SqlConnection(#"MyConnectionString"))
{
connection.Open();
SqlCommand command = new SqlCommand("SELECT * FROM MyTable", connection);
SqlDataReader reader = command.ExecuteReader(CommandBehavior.KeyInfo); // If this is changed to CommandBehavior.SchemaOnly, the program runs fast even here in the using
DataTable table = reader.GetSchemaTable();
Console.WriteLine(table.Rows.Count);
}
The table in question has over 3 million rows, but since I am only obtaining the Schema information, I would think this wouldn't be an issue. My question is: Why does my application get stuck while trying to close a connection?
SOLUTION: Maybe this isn't optimal, but it does work; I inserted a command.Cancel(); statement right before Close is called on connection:
SqlConnection connection = new SqlConnection(#"MyConnectionString");
connection.Open();
SqlCommand command = new SqlCommand("SELECT * FROM MyTable", connection);
SqlDataReader reader = command.ExecuteReader(CommandBehavior.KeyInfo); // If this is changed to CommandBehavior.SchemaOnly, the program runs fast.
DataTable table = reader.GetSchemaTable();
Console.WriteLine(table.Rows.Count);
command.Cancel(); // <-- This is it.
connection.Close(); // Alternatively If this line is commented out, the program runs fast.
I saw something like this, long ago. For me, it was because I did something like:
SqlCommand command = new SqlCommand("SELECT * FROM MyTable", connection);
SqlDataReader reader = command.ExecuteReader();
// here, I started looping, reading one record at a time
// and after reading, say, 100 records, I'd break out of the loop
connection.Close(); // this would hang
The problem is that the command appears to want to complete. That is, go through the entire result set. And my result set had millions of records. It would finish ... eventually.
I solved the problem by adding a call to command.Cancel() before calling connection.Close().
See http://www.informit.com/guides/content.aspx?g=dotnet&seqNum=610 for more information.
It looks right to me overall and I think you need a little optimization. In addition to the above suggestion regarding avoiding DataReader, I will recommend to use connection pooling. You can get the details from here :
http://www.techrepublic.com/article/take-advantage-of-adonet-connection-pooling/6107854
Could you try this?
DataTable dt = new DataTable();
using(SqlConnection conn = new SqlConnection("yourConnectionString"))
{
SqlCommand cmd = new SqlCommand("SET FMTONLY ON; " + yourQueryString + "; SET FMTONLY OFF;",conn);
conn.Open();
dt.Load(cmd.ExecuteReader());
}
SET FMTONLY ON/OFF from MSDN seems the way to go
There is an specific way to do this, using SMO (SQL Server management objects)
You can get the collection of tables in the database, and then read the properties of the table you're interested in (columns, keys, and all imaginable properties)
This is what SSMS uses to get and set properties of all database objects.
Look at this references:
Database.Tables Property
Table class
This is a full example of how to get table properties:
Retrieving SQL Server 2005 Database Info Using SMO: Database Info, Table Info
This will allow you to get all the possible information from the database in a very easy way. there are plenty of samples in VB.NET and C#.
I would try something like this. This ensures all items are cleaned up - and avoids using DataReader. You don't need this unless you have unusually large amounts of data that would cause memory issues.
public void DoWork(string connectionstring)
{
DataTable dt = new DataTable("MyData");
using (var connection = new SqlConnection(connectionstring))
{
connection.Open();
string commandtext = "SELECT * FROM MyTable";
using(var adapter = new SqlDataAdapter(commandtext, connection))
{
adapter.Fill(dt);
}
connection.Close();
}
Console.WriteLine(dt.Rows.Count);
}
I want to use an Access database for my Windows Forms application. (written with C#)
I have used OleDb namespace for connecting, and I'm able to select the records from the source using the OleDbConnection and ExecuteReader objects.
However, I can't insert, update or delete records yet.
My code is the following:
OleDbConnection con = new OleDbConnection(strCon);
try
{
string con="Provider=Microsoft.Jet.OLEDB.4.0;Data Source=xyz.mdb;Persist Security Info=True";
con.Open();
OleDbCommand com = new OleDbCommand("INSERT INTO DPMaster(DPID, DPName, ClientID, ClientName) VALUES('53', 'we', '41', 'aw')", con);
int a = com.ExecuteNonQuery();
//OleDbCommand com = new OleDbCommand("SELECT * FROM DPMaster", con);
//OleDbDataReader dr = com.ExecuteReader();
//while (dr.Read())
//{
// MessageBox.Show(dr[2].ToString());
//}
MessageBox.Show(a.ToString());
}
catch
{
MessageBox.Show("cannot");
}
If I the commentted block is executed, the application works fine. But the insert block doesn't.
Knowing this, why I am unable to insert, update or delete database records?
the problem that I encountered myself is as:
You've added the mdb file to your solution and every time you run the program it will be copied into debug folder.
So you can select from it but deleting rows doesn't affect the original file you have in your solution.
Check for it.
First, never strangle your exception. It is better to let your exception bubble up so that you may get important information regarding what is not working properly. It is better to write:
con.Open();
OleDbCommand com = new OleDbCommand("INSERT INTO DPMaster(DPID,DPName,ClientID,ClientName) VALUES('53','we','41','aw')", con);
int a = com.ExecuteNonQuery();
than
try {
con.Open();
OleDbCommand com = new OleDbCommand("INSERT INTO DPMaster(DPID,DPName,ClientID,ClientName) VALUES('53','we','41','aw')", con);
int a=com.ExecuteNonQuery();
} catch {
MessageBox.Show("cannot");
}
Second, make use of using blocks as much as possible, since those blocks will dispose the no longer needed objects. So your code should look like this:
using (OleDbConnection con = new OleDbConnection(conStr))
using (OleDbCommand com = new OleDbCommand("INSERT INTO DPMaster(DPID,DPName,ClientID,ClientName) VALUES('53','we','41','aw')", con) {
con.Open();
int a = com.ExecuteNonQuery();
MessageBox.Show(a.ToString());
}
With this code, you will more likely get to know what is going wrong while the exception will bubble up, plus, as soon as you'll quit the scope of the using blocks, resources used will be freed up as your objects will get disposed.