We Keep Coding

Where to close SqlDataReader object and SqlConnection object?

I call a function which returns a SqlDataReader object to calling statement. I am confused where should I close the SqlDataReader object and SqlConnection object? In function or after calling it?
This is the function call:
SqlDataReader dr2= RetrieveSearcher();
pid = dr2[0].ToString();
This is the function:
protected SqlDataReader RetrieveSearcher()
{
String Q = "select price from tb3 where pid='12';
cn = new SqlConnection("data source=.\\sqlexpress; integrated security=true; initial catalog=singh");
cn.Open();
cmd = new SqlCommand(Q,cn);
dr1 = cmd.ExecuteReader();
dr1.Read();
return dr1;
}

Always use parameterized queries to avoid sql injection attacks and increase performance (most db servers can reuse execution plans with proper queries)
Never leave a connection open any longer than necessary!
Do not share db connections! Create it, use it, destroy it.
Wrap everything that implements IDisposable in a using block like Connections, Commands, DataReaders, etc. This ensures no resources remain open even in the event of an exception.
Use correct types in your db schema and read those types, do not blanket-convert everything to/from string! Example price seems like it should really be a decimal or numeric value and not a string so do not store it as a string and do not read it back as a string.
Retrieve the connection strings by name from the app.config or web.config (depending on the application type), do not hard code the strings into your connections or anywhere else.
About your logic
Change your method to return a custom type like a piece of data. This ensures proper SoS (Separation of Concerns). Do not return a DataReader! This will abstract the whole database call from the caller which is what you should strive for.
protected SomeType RetrieveSearcherData(string pid)
{
const string Q = "SELECT price FROM tb3 WHERE pid = #pid";
using(var cn=new SqlConnection())
using(var cmd=new SqlCommand(Q,cn))
{
// I do not know what pid is but use tho correct type here as well and specify that type using SqlDbType
cmd.Parameters.Add(new SqlParameter("#pid", SqlDbType.VarChar, 100) { Value = pid});
cn.Open();
using(var dr1= cmd.ExecuteReader())
{
if(dr1.Read())
{
var result = dr1.GetDecimal(0);
// read something and return it either in raw format or in some object (use a custom type)
}
else
return null; // return something else that indicates nothing was found
}
}
}

Do you really want to open a connection each time you call into this function? Having one thread deal with multiple connections is a sure fire way to get deadlocks.
If you still want to do #1, I'd recommend having your RetrieveSearcher return the data it needs in a List<T> or heck, just return a DataTable and deal with that. That way the function can close the connection that it opened.
If you still REALLY want to return a SqlDataReader then you need to make sure that you can close the connection that you opened. SqlDataReader doesn't expose a SqlConnection directly, so you can't directly close the connection after you leave the RetrieveSearcher method. However, you can do this:
dr1 = cmd.ExecuteReader(CommandBehavior.CloseConnection);
That will close the connection when the reader is closed. So, then you can do:
using (SqlDataReader dr2 = RetrieveSearcher()) {
pid=dr2[0].ToString();
}
I'm assuming of course that you REALLY need more than just one string. :) If you REALLY only need one string you just be returning the string and calling cmd.ExecuteScalar();

App_Code SQL Inject/Select

I need some guidance on the following if possible please
Explanation
I have a main project.cs file in the App_Code which contains main functions. One of these functions is a SQL_Inject which inserts data into the database.
I then have multiple pages that utilize this function from multiple client machines at the same time.
Question
The answer i am after is, is this a safe method of choice? Or should i be creating a new connection separately on each .cs page.
Reason/Problem
Reason this is becoming a concern, we are currently a small company but growing. It has happened that a page crashes due to the SQL Connection is still open. I am worried its due to two connections trying to be made at the same time. I am not sure if this is the issue or if it comes from something else.
//GLOBAL DECLARATIONS
//DB CONNECTIONS - retrieve from config file
public static string ConProjectms = System.Configuration.ConfigurationManager.ConnectionStrings["conProject"].ConnectionString;
//DB CONNECT TO SQL
public static SqlConnection SqlConn = new SqlConnection();
public static SqlCommand SqlCmd = new SqlCommand();
public static SqlDataReader SqLdr;
public static string SqlStr;
public static string ConnString;
public static void SqlInject(string query, string dataBase)
{
SqlConn.ConnectionString = ConProjectms;
//Set the Connection String
SqlConn.Open();
//Open the connection
SqlCmd.Connection = SqlConn;
//Sets the Connection to use with the SQL Command
SqlCmd.CommandText = query;
//Sets the SQL String
SqlCmd.ExecuteNonQuery();
//put Data
SqlClose();
}
public static void SqlClose()
{
if (SqlConn.State != ConnectionState.Open) return;
SqlConn.Close();
SqlCmd.Parameters.Clear();
}

SQL can handle multiple connections at the same time. However, you're code is very likely to be be run by two clients at the same time, and they'll be using the same connection not two separate connections. That's bad thing #1.
SQL Server does a fantastic job of connection pooling - and I assume other DBs have similar capabilities. In such a world, you shouldn't try to keep and reuse any of your data-related objects around - but create them as you need them and when SQL sees that you're using a connection it's created before and since freed up, it'll use that. You don't have to do anything weird to get this functionality.
With that in mind, your static objects should mostly go away, and your SQLInject method might look something like this:
public static void SqlInject(string query, string dataBase)
{
var connectionString =
System
.Configuration
.ConfigurationManager
.ConnectionStrings["conProject"]
.ConnectionString;
using ( var connection = new SqlConnection( connectionString ) )
{
connection.Open( );
using ( var command = connection.CreateCommand( ) )
{
command.CommandText = query;
command.CommandType = CommandType.Text;
command.ExecuteNonQuery( );
}
}
}
Notice that you don't have to worry about closing the connection per se; the using blocks handle the disposition of your open, active objects. This is largely how folks are doing direct SQL from c#. By the way, neither your code nor mine uses the dataBase argument. Maybe you're supposed to edit the base connection string with it??
But wait - there's more!
Having said all that, and since you raised a concern about security, you should know that this isn't safe code at all - yours or mine. SqlInject is probably a good name, because it allows pretty much anything in the query argument (which, BTW, if you're doing ExecuteNonQuery, then maybe query isn't a good name).
You're far far better allowing arguments to a library of known statements (maybe stored procedures), validating those arguments, and using SQL Injection attack mitigation to parameterize your known statements (look up that phrase and you'll find an abundance of examples and advice).
Just for yuks, here's a scaffold of what you might consider:
public static void SqlInject(string commandName, params[] object commandArgs )
{
//--> no point in going on if we got no command...
if ( string.IsNullOrEmpty( commandName ) )
throw new ArgumentNullException( nameof( commandName ) );
var connectionString =
System
.Configuration
.ConfigurationManager
.ConnectionStrings["conProject"]
.ConnectionString;
using ( var connection = new SqlConnection( connectionString ) )
{
connection.Open( );
using ( var command = connection.CreateCommand( ) )
{
command.CommandType = CommandType.Text;
command.CommandText = "select commandText from dbo.StatementRepository where commandName = #commandName";
command.Parameters.AddWithValue( "#commandName", commandName );
var results = command.ExecuteScalar( );
if ( results != null && results != DbNull.Value )
{
//--> calling a separate method to validate args, that returns
//--> an IDictionary<string,object> of parameter names
//--> and possibly modified arguments.
//--> Let this validation method throw exceptions.
var validatedArgs = ValidateArgs( commandName, commandArgs );
command.Parameters.Clear( );
command.CommandText = query;
foreach( var kvp in validatedArgs )
{
command.Parameters.AddWithValue( kvp.Key, kvp.Value );
}
command.ExecuteNonQuery( );
}
else
{
throw new InvalidOperationException( "Invalid command" );
}
}
}
}
I didn't attempt to write an actual argument validating method, because that's all wrapped up in your application logic...but I wanted to give you an idea of how you might get to a safer state.

There's no reason why database code inside App_Code shouldn't work. It sounds more like your connection pooling doesn't work very well. Look at the connection string, IIS settings and the performance of your database. If for some reason connection pooling is not possible, then the running time of the query becomes the problem.

Best Practices for SQL Statements/Connections in Get() Request

For simple lookups, I need to perform some SQL statements on a DB2 machine. I'm not able to use an ORM at the moment. I have a working example through this code, however I'm wondering if it can be optimized more as this would essentially create a connection on each request. And that just seems like bad programming.
Is there a way I can optimize this Get() request to leave a connection open? Nesting using statements seems dirty, as well. How should I handle the fact that Get() really wants to return an object of User no matter what; even in error? Can I put this connection in the start of the program so that I can use it over and over again? What are some of the best practices for this?
public class UsersController : ApiController
{
String constr = WebConfigurationManager.ConnectionStrings["DB2Connection"].ConnectionString;
public User Get([FromUri] User cst)
{
if (cst == null)
{
throw new HttpResponseException(HttpStatusCode.NotFound);
}
else
{
using (OdbcConnection DB2Conn = new OdbcConnection(constr))
{
DB2Conn.Open();
using (OdbcCommand com = new OdbcCommand(
// Generic SQL Statement
"SELECT * FROM [TABLE] WHERE customerNumber = ?", DB2Conn))
{
com.Parameters.AddWithValue("#var", cst.customerNumber);
using (OdbcDataReader reader = com.ExecuteReader())
{
try
{
while (reader.Read())
{
cst.name = (string)reader["name"];
return cst;
}
}
catch
{
throw;
}
}
}
}
return cst;
}
}
}
I found a great question that doesn't really have detailed answers, I feel like similar solutions exist for both of these questions...

And that just seems like bad programming.
Why do you think that?
The underlying system should be maintaining connections in a connection pool for you. Creating a connection should be very optimized already.
From a logical perspective, what you're doing now is exactly what you want to be doing. Create the connection, use it, and dispose of it immediately. This allows other threads/processes/etc. to use it from the connection pool now that you're done with it.
This also avoids the myriad of problems which arise from manually maintaining your open connections outside of the code that uses them.
Is there a way I can optimize this Get() request to leave a connection open?
Have you measured an actual performance problem? If not, there's nothing to optimize.
And there's a very good chance that hanging on to open connections in a static context in your web application is going to have drastic performance implications.
In short... You're already doing this correctly. (Well, except for that unnecessary try/catch. You can remove that.)
Edit: If you're just looking to improve the readability of the code (which itself is a matter of personal preference), this seems readable to me:
public User Get([FromUri] User cst)
{
if (cst == null)
throw new HttpResponseException(HttpStatusCode.NotFound);
using (var DB2Conn = new OdbcConnection(constr))
using (var com = new OdbcCommand("SELECT * FROM [TABLE] WHERE customerNumber = ?", DB2Conn))
{
com.Parameters.AddWithValue("#var", cst.customerNumber);
DB2Conn.Open();
using (OdbcDataReader reader = com.ExecuteReader())
while (reader.Read())
{
cst.name = (string)reader["name"]
return cst;
}
}
return cst;
}
Note that you can further improve it by re-addressing the logic of that SQL query. Since you're fetching one value from one record then you don't need to loop over a data reader. Just fetch a single literal and return it. Note that this is free-hand and untested, but it might look something like this:
public User Get([FromUri] User cst)
{
if (cst == null)
throw new HttpResponseException(HttpStatusCode.NotFound);
using (var DB2Conn = new OdbcConnection(constr))
using (var com = new OdbcCommand("SELECT name FROM [TABLE] WHERE customerNumber = ? FETCH FIRST 1 ROWS ONLY", DB2Conn))
{
com.Parameters.AddWithValue("#var", cst.customerNumber);
DB2Conn.Open();
cst.name = (string)com.ExecuteScalar();
}
return cst;
}

#David's answer addresses your actual questions perfectly but here's some other observations that may make your code a little more pallatable to you:
remove the try/catch block - all you're doing is re-throwing the exception which is what will happen if you don't use a try/catch at all. Don't catch the exception unless you can do something about it. (I see now that #David's answer addresses that - either it was added after I read it or I missed it - my apologies for the overlap but it's worth reinforcing)
Change your query to just pull name and use ExecuteScalar instead of ExecuteReader. You are taking the name value from the first record and exiting the while loop. ExecuteScalar returns the value from the first column in the first record, so you can eliminate the while loop and the using there.

Timeout error when result of a transaction is used as parameter

I have following code that uses SqlTransaction
string connectionString = ConfigurationManager.AppSettings["ConnectionString"];
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlTransaction transaction = connection.BeginTransaction();
int logID = HelperClass.InsertException(connection, 1, DateTime.Now, "Y", "test", "test", 1, 1, transaction);
LogSearch logSearch = new LogSearch();
logSearch.LogID = 258;
Collection<Log> logs = HelperClass.GetLogs(logSearch, connectionString);
}
This code is throwing the following exception.
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
However if I pass a hard coded value for LogID, there is no exception.
QUESTION
Why does exception come when I pass logID (result from InsertException()) ?
Please explain why there is NO exception when I pass a hard coded value as LogID
Note: InsertException() uses a connection with SqlTransaction whereas GetLogs() uses a new connection without any transaction
UPDATED QUESTION
The Business Layer code does not use Transaction. I need to call the Business Layer methods in my Unit Testing code shown above (for integration testing). How can we apply transaction to UT code (for integration testing) even though the Business Layer does not use transaction? From #jbl answer, it seems like, it is not at all possible to use transaction in Unit Testing. How can we apply transaction for UT code.
CODE
public static class HelperClass
{
public static Collection<Log> GetLogs(LogSearch logSearch, string connectionString)
{
Collection<Log> logs = new Collection<Log>();
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
string commandText = "SELECT * FROM Application_EX WHERE application_ex_id = #application_ex_id";
using (SqlCommand command = new SqlCommand(commandText, connection))
{
command.CommandType = System.Data.CommandType.Text;
//Parameter value setting
command.Parameters.AddWithValue("#application_ex_id", logSearch.LogID);
using (SqlDataReader reader = command.ExecuteReader())
{
if (reader.HasRows)
{
while (reader.Read())
{
}
}
}
}
}
return logs;
}
public static Int16 InsertException(SqlConnection connection, Int16 applicationID, DateTime createdDate, string isInternalLocationIndicator, string exceptionDescription, string operation, Int16 severityLevelNumber, Int16 exceptionTypeCode, SqlTransaction transaction)
{
Int16 newLogID = 0;
string commandText = #"INSERT INTO Application_Ex
VALUES (#severity_level_nbr, #appl_service_id, #ex_internal_appl_ind,
#application_ex_txt,#ex_location_txt,#create_ts,#application_ex_code);
SELECT SCOPE_IDENTITY() AS [LogIdentity];";
using (SqlCommand command = new SqlCommand(commandText, connection, transaction))
{
command.CommandType = System.Data.CommandType.Text;
command.Parameters.AddWithValue("#severity_level_nbr", severityLevelNumber);
command.Parameters.AddWithValue("#appl_service_id", applicationID);
command.Parameters.AddWithValue("#ex_internal_appl_ind", isInternalLocationIndicator);
command.Parameters.AddWithValue("#application_ex_txt", exceptionDescription);
command.Parameters.AddWithValue("#ex_location_txt", operation);
command.Parameters.AddWithValue("#create_ts", createdDate);
command.Parameters.AddWithValue("#application_ex_code", exceptionTypeCode);
newLogID = Convert.ToInt16(command.ExecuteScalar(), CultureInfo.InvariantCulture);
}
return newLogID;
}
}

QUESTION
Why does exception come when I pass logID (result from InsertException()) ?
Please explain why there is NO exception when I pass a hard coded value as LogID
ANSWERS
When a new record is inserted with using a transaction, that means the new record is not finally committed until you yourself commit the transaction. Until that time the new record is LOCKED, which means that any query that touches that new record will be suspended until a timeout occurs. In your case, the call to the method GetLogs is executed while the transaction is still running, and that method searches for the newly inserted record-id. Since the row is locked, your GetLogs call will wait until the timeout occurs, which results in a timeout exception.
In the case of a hard coded value, the call to GetLogs will search for the existing record with the corresponding id. Since you are searching for a PK-value, SQL does not have to search all rows, since it is the PK. Therefore, the existing row is found, and returned, all in a separate transaction, since the transactions are not overlapping in the data they touch.
Suppose your method GetLogs was searching the table on another column, a non-pk column (say for example application_message), then the whole table would have to be read to find a row (or rows) with the corresponding value for application_message. This would result in a query that always touches the newly inserted locked row, and then also with a hard coded (application_message) value your would get a timeout exception. This I added just to clarify the locking, and when SQL does or does not need to touch the locked row(s).
Hope this helps.

I guess that's because HelperClass.GetLogs(logSearch, connectionString); instantiate a new connection out of scope of your transaction :
You may, at your will :
have your helper class accept the connection object holding the transaction instead of a connection string
or replace "SELECT * FROM Application_EX WHERE application_ex_id = #application_ex_id" with "SELECT * FROM Application_EX with (nolock) WHERE application_ex_id = #application_ex_id"
note that the second case would sometimes return incorrect values, and would not return values you are currently inserting in your transaction
Hope this will help

What's the most DRY-appropriate way to execute an SQL command?

I'm looking to figure out the best way to execute a database query using the least amount of boilerplate code. The method suggested in the SqlCommand documentation:
private static void ReadOrderData(string connectionString)
{
string queryString = "SELECT OrderID, CustomerID FROM dbo.Orders;";
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = new SqlCommand(queryString, connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
try
{
while (reader.Read())
{
Console.WriteLine(String.Format("{0}, {1}", reader[0], reader[1]));
}
}
finally
{
reader.Close();
}
}
}
mostly consists of code that would have to be repeated in every method that interacts with the database.
I'm already in the habit of factoring out the establishment of a connection, which would yield code more like the following. (I'm also modifying it so that it returns data, in order to make the example a bit less trivial.)
private SQLConnection CreateConnection()
{
var connection = new SqlConnection(_connectionString);
connection.Open();
return connection;
}
private List<int> ReadOrderData()
{
using(var connection = CreateConnection())
using(var command = connection.CreateCommand())
{
command.CommandText = "SELECT OrderID FROM dbo.Orders;";
using(var reader = command.ExecuteReader())
{
var results = new List<int>();
while(reader.Read()) results.Add(reader.GetInt32(0));
return results;
}
}
}
That's an improvement, but there's still enough boilerplate to nag at me. Can this be reduced further? In particular, I'd like to do something about the first two lines of the procedure. I don't feel like the method should be in charge of creating the SqlCommand. It's a tiny piece of repetition as it is in the example, but it seems to grow if transactions are being managed manually or timeouts are being altered or anything like that.
edit: Assume, at least hypothetically, there's going to have to be a bunch of different types of data being returned. And consequently the solution can't be just one one-size-fits-all method, there will have to be a few different ones depending, at minimum, on whether ExecuteNonQuery, ExecuteScalar, ExecuteReader, ExecuteReaderAsync, or any of the others are being called. I'd like to cut down on the repetition among those.

Tried Dapper?
Granted this doesn't get you a DataReader but you might just prefer it this way once you've tried it.
It's about the lightest-weight an ORM can be while still being called an ORM. No more methods to map between DataReader and strong types for me.
Used right here on all the StackExchange sites.
using (var conn = new SqlConnection(cs))
{
var dogs = connection.Query("select name, age from dogs");
foreach (dynamic dog in dogs)
{
Console.WriteLine("{0} age {1}", dog.name, dog.age);
}
}
or
using (var conn = new SqlConnection(cs))
{
var dogs = connection.Query<Dog>("select Name, Age from dogs");
foreach (Dog dog in dogs)
{
Console.WriteLine("{0} age {1}", dog.Name, dog.Age);
}
}
class Dog
{
public string Name { get; set; }
public int Age { get; set; }
}

If you want to roll data access on your own, this pattern of help methods could be one way to remove duplication:
private List<int> ReadOrderData()
{
return ExecuteList<int>("SELECT OrderID FROM dbo.Orders;",
x => x.GetInt32("orderId")).ToList();
}
private IEnumerable<T> ExecuteList(string query,
Func<IDataRecord, T> entityCreator)
{
using(var connection = CreateConnection())
using(var command = connection.CreateCommand())
{
command.CommandText = query;
connection.Open();
using(var reader = command.ExecuteReader())
{
while(reader.Read())
yield return entityCreator(reader);
}
}
}
You'll have to add support for parameters and this might not compile, but the pattern is what I'm trying to illustrate.

What I typically do is use a custom class that I wrote a while back that accepts a SQL string, and optionally a list of parameters and it returns a DataTable.
Since the thing that changes between invocations is typically just the SQL that is optimal IMHO.
If you truly do need to use a DataReader you can do something like this:
public void ExecuteWithDataReader(string sql, Action<DataReader> stuffToDo) {
using (SqlConnection connection = new SqlConnection(connectionString)) {
using (SqlCommand command = new SqlCommand(sql, connection)) {
connection.Open();
using (SqlDataReader reader = command.ExecuteReader()) {
try {
while (reader.Read()) {
stuffToDo(reader);
}
}
finally {
reader.Close();
}
}
}
}
}
private static void ReadOrderData(string connectionString) {
string sql = "SELECT OrderID, CustomerID FROM dbo.Orders;";
ExecuteWithDataReader(sql, r => Console.WriteLine(String.Format("{0}, {1}", r[0], r[1])));
}

The first two line are the most important thing you need...
but if you still wish to do it, you can turn them to a database handler class, yes it will become more of code, but in refactoring concept, every thing will move to the related topic...
try to write a singleton class, that receive a command and do action, so return result of type SqlDataReader reader...

Doing this in comments was too much.
I would suggest that the boilerplate code around
using(conn = new sqlconnection)
using(cmd = new sqlcommand) {
// blah blah blah
}
isn't something to be lightly removed and instead would encourage that you keep it exactly where it's at. Resources, especially unmanaged ones, should be opened and released at the closest point to execution as possible IMHO.
In no small part due to the ease with which other developers will fail to follow the appropriate clean up conventions.
If you do something like
private SQLConnection CreateConnection()
{
var connection = new SqlConnection(_connectionString);
connection.Open();
return connection;
}
Then you are inviting another programmer to call this method and completely fail to release the resource as soon as the query is executed. I don't know what kind of app you are building, but in a web app such a thing will lead to memory / connection / resource errors of types that are difficult to debug, unless you've been through it before.
Instead, I'd suggest you look into a lightweight ORM such as Dapper.net or similar to see how they approached it. I don't use dapper, but I hear it's pretty good. The reason I don't use it is simply that we don't allow inline sql to be executed against our databases (but that's a very different conversation).
Here's our standard:
public static DataTable StatisticsGet( Guid tenantId ) {
DataTable result = new DataTable();
result.Locale = CultureInfo.CurrentCulture;
Database db = DatabaseFactory.CreateDatabase(DatabaseType.Clients.ToString());
using (DbCommand dbCommand = db.GetStoredProcCommand("reg.StatsGet")) {
db.AddInParameter(dbCommand, "TenantId", DbType.Guid, tenantId);
result.Load(db.ExecuteReader(dbCommand));
} // using dbCommand
return result;
} // method::StatisticsGet
We make heavy use of Enterprise Library. It's short, simple and to the point and very well tested. This method just returns a datatable but you could easily have it return an object collection.. or nothing.