I have a class like
class ReadData{
public IdataReader Execute(string sql){
// Ado.net code here
return cmd.ExecuteReader();
}
}
This is the sample implementation and it works fine.
I am calling like this in caller
class caller{
void CallMethod(){
var reader = Execute("Sql query here");
while(reader.Read()){
logic here
}
//Here i need to get the out params after reading the resultset.
//But the impplementation should in the class ReadData.
//because that class has implementation to get the out params for the
//other type means, execute without resultset get only output params
}
}
Some possible ways like calling the first method with callback and in that once the data is read completely then read the out params.
I don't know how to implement that stuff.
Any possible better ways to do ?
Please help me on this..
public void Execute(string sql, Action<IDataRecord> action)
{
using(var connection = new ...)
{
connection.Open();
using(var command = new ...)
{
using(var reader = command.ExecuteReader())
{
while(reader.Read())
{
action(reader);
}
}
}
}
}
This would allow you to do something like this:
var entries = List<object>();
Execute("Sql query here", row => entries.Add(row["Field"]));
Or you could try a more linqy appraoch:
public IEnumerable<IDataRecord> Execute(string sql)
{
using(var connection = new ...)
{
connection.Open();
using(var command = new ...)
{
using(var reader = command.ExecuteReader())
{
while(reader.Read())
{
yield return reader;
}
}
}
}
}
Which would allow something like this:
var list = Execute("Sql query here").Where(row => (int)row["Field"] == 17)).ToList();
However, this has some weird effects with defered execution if you don't materialize it properly.
Related
I'm learning how Dapper is working behind the scenes.
However I saw this pattern of disposing which is not understood to me.
Roughly in general — this is how QueryAsync is implemented :
/*1*/ public async Task<IEnumerable<T>> QueryAsync<T>(string sql, Func<IDataRecord, T> projector, DbConnection _conn, dynamic param = null)
/*2*/ {
/*3*/
/*4*/ DbDataReader reader = null;
/*5*/ bool wasClosed = _conn.State == ConnectionState.Closed;
/*6*/ try
/*7*/ {
/*8*/
/*9*/ using (var cmd = _conn.CreateCommand())
/*10*/ {
/*11*/ if (param!=null)
/*12*/ foreach (var prop in param.GetType().GetProperties(BindingFlags.Instance | BindingFlags.Public))
/*13*/ {
/*14*/ var parameter = cmd.CreateParameter();
/*15*/ parameter.ParameterName = prop.Name;
/*16*/ parameter.Value = prop.GetValue(param, null);
/*17*/ cmd.Parameters.Add(parameter);
/*18*/ }
/*19*/
/*20*/ await _conn.OpenAsync().ConfigureAwait(false);
/*21*/ cmd.CommandTimeout = 100000;
/*22*/ cmd.CommandText = sql;
/*23*/ reader = await cmd.ExecuteReaderAsync().ConfigureAwait(false);
/*24*/ List<T> buffer = new List<T>();
/*25*/ while (await reader.ReadAsync().ConfigureAwait(false)) buffer.Add(projector(reader));
/*26*/ return buffer;
/*27*/ }
/*28*/
/*29*/ }
/*30*/ finally
/*31*/ {
/*32*/ using (reader) { }
/*33*/ if (wasClosed) _conn.Close();
/*34*/ }
/*35*/ }
I can understand why he didn't use using over the connection , that's because he wanted to conditionally close the connection via the wasClosed variable.
For doing it - he must use the try/finally clause. ( so the conditional closing will be in the finally clause)
But my question is about line #32.
Instead of doing using at the finally clause , he could do:
using (DbDataReader reader = await cmd.ExecuteReaderAsync().ConfigureAwait(false))
{
List<T> buffer = new List<T>();
while (await reader.ReadAsync().ConfigureAwait(false)) buffer.Add(projector(reader));
return buffer;
}
So the finally clause is left with :
finally
{
//using (reader) { } //removed
if (wasClosed) _conn.Close();
}
Question
I've seen this using clause in a finally clause many times in dapper.
I must be missing something here, But what does this pattern achieve that my suggestion does not?
I'm no #MarcGravell, but I think there is one thing you're missing. The code you pasted doesn't exactly match the link you reference. The relevant code path looks like this:
try
{
if (command.Buffered)
{
List<T> buffer = new List<T>();
while (await reader.ReadAsync(cancel).ConfigureAwait(false))
{
buffer.Add((T)func(reader));
}
while (await reader.NextResultAsync().ConfigureAwait(false)) { }
command.OnCompleted();
return buffer;
}
else
{
// can't use ReadAsync / cancellation; but this will have to do
wasClosed = false; // don't close if handing back an open reader;
// rely on the command-behavior.
var deferred = ExecuteReaderSync<T>(reader, func, command.Parameters);
reader = null; // to prevent it being disposed before the caller gets to see it
return deferred;
}
}
finally
{
using (reader) { } // dispose if non-null
if (wasClosed) cnn.Close();
}
The method can either return a buffered result (indicated by the command.Buffered flag) or a deferred iterator. If Marc was to wrap the reader with a using statement and return an iterator, it (the reader) would of been disposed by the time the call-site executed it. By setting the reader to null (in the line before he returns the deferred result) he prevents the reader from being disposed, because the using in the finally block would be translated to this:
finally
{
IDisposable disposable = reader;
try
{
}
finally
{
if (dispoable != null)
{
disposable.Dispose();
}
}
}
When he sets the reader to null, it isn't disposed, and the reference exists in the iterator is still alive, pointing to the reader. This way, he can both dispose the reader in the normal codepath, but keep it alive if a deferred iterator was requested.
I am using TransactionScope in my repository unit tests to rollback any changes made by tests.
Setup and teardown procedures for tests look like this:
[TestFixture]
public class DeviceRepositoryTests {
private static readonly string ConnectionString =
ConfigurationManager.ConnectionStrings["TestDB"].ConnectionString;
private TransactionScope transaction;
private DeviceRepository repository;
[SetUp]
public void SetUp() {
transaction = new TransactionScope(TransactionScopeOption.Required);
repository = new DeviceRepository(ConnectionString);
}
[TearDown]
public void TearDown() {
transaction.Dispose();
}
}
Problematic test consists of code which inserts records to database and CUT that retrieves those records.
[Test]
public async void GetAll_DeviceHasSensors_ReturnsDevicesWithSensors() {
int device1Id = AddDevice();
AddSensor();
var devices = await repository.GetAllAsync();
// Asserts
}
AddDevice and AddSensor methods open sql connection and insert a row into a database:
private int AddDevice() {
var sqlString = "<SQL>";
using (var connection = CreateConnection())
using (var command = new SqlCommand(sqlString, connection)) {
var insertedId = command.ExecuteScalar();
Assert.AreNotEqual(0, insertedId);
return (int) insertedId;
}
}
private void AddSensor() {
const string sqlString = "<SQL>";
using (var connection = CreateConnection())
using (var command = new SqlCommand(sqlString, connection)) {
var rowsAffected = command.ExecuteNonQuery();
Assert.AreEqual(1, rowsAffected);
}
}
private SqlConnection CreateConnection() {
var result = new SqlConnection(ConnectionString);
result.Open();
return result;
}
GetAllAsync method opens a connection, executes query, and for each fetched row opens new connection to fetch child objects.
public class DeviceRepository {
private readonly string connectionString;
public DeviceRepository(string connectionString) {
this.connectionString = connectionString;
}
public async Task<List<Device>> GetAllAsync() {
var result = new List<Device>();
const string sql = "<SQL>";
using (var connection = await CreateConnection())
using (var command = GetCommand(sql, connection, null))
using (var reader = await command.ExecuteReaderAsync()) {
while (await reader.ReadAsync()) {
var device = new Device {
Id = reader.GetInt32(reader.GetOrdinal("id"))
};
device.Sensors = await GetSensors(device.Id);
result.Add(device);
}
}
return result;
}
private async Task<List<Sensor>> GetSensors(int deviceId) {
var result = new List<Sensor>();
const string sql = "<SQL>";
using (var connection = await CreateConnection())
using (var command = GetCommand(sql, connection, null))
using (var reader = await command.ExecuteReaderAsync()) {
while (await reader.ReadAsync()) {
// Fetch row and add object to result
}
}
return result;
}
private async Task<SqlConnection> CreateConnection() {
var connection = new SqlConnection(connectionString);
await connection.OpenAsync();
return connection;
}
}
The problem is that when GetSensors method calls SqlConnection.Open I get following exception:
System.Transactions.TransactionAbortedException : The transaction has aborted.
----> System.Transactions.TransactionPromotionException : Failure while attempting to promote transaction.
----> System.Data.SqlClient.SqlException : There is already an open DataReader associated with this Command which must be closed first.
----> System.ComponentModel.Win32Exception : The wait operation timed out
I could move code that fetches child object out of the first connection scope (this would work), but let's say I don't want to.
Does this exception mean that it is impossible to open simultaneous connections to DB inside single TransactionScope?
Edit
GetCommand just calls SqlCommand contructor and do some logging.
private static SqlCommand GetCommand(string sql, SqlConnection connection, SqlParameter[] parameters) {
LogSql(sql);
var command = new SqlCommand(sql, connection);
if (parameters != null)
command.Parameters.AddRange(parameters);
return command;
}
The issue is that two DataReader objects can't be open at the same time against the database (unless MARS is enabled). This restriction is by design. As I see it you have a few options:
Enable MARS on your connection string; add this MultipleActiveResultSets=True
Don't use the DataReader if it's really not necessary. But the way you've got your code written, it's pretty necessary.
Populate the Sensor property after loading the devices.
Use Dapper, it can do all of this (including populate the Sensor) and likely faster.
Using Dapper you could do something like this (and you wouldn't need GetSensors):
public async Task<List<Device>> GetAllAsync() {
var result = new List<Device>();
const string sql = "<SQL>";
using (var connection = await CreateConnection())
using (var multi = connection.QueryMultiple(sql, parms)) {
result = multi.Read<Device>().ToList();
var sensors = multi.Read<Sensors>().ToList();
result.ForEach(device => device.Sensors =
sensors.Where(s => s.DeviceId = device.Id).ToList());
}
return result;
}
Here your sql would look like this:
SELECT * FROM Devices
SELECT * FROM Sensors
See the Multi Mapping documentation for Dapper.
I'm trying to determine if there are glaring errors in a code base, or not.
The code in question calls a third party dll which returns an IDataReader. If the code uses the reader without disposing of it, it won't be explicitly returned to the pool, correct?
Here's the calling code:
IDataReader rdr = db.ExecSPGetDataReader("dbo.someStoredProcedure", paramList);
if (rdr.Read())
{
List<nameValuePair> formValues = Utils.nameValuePairs(rdr["valuepairs"].ToString());
foreach (nameValuePair nvp in formValues)
{
if (nvp.name.ToLower() == "name")
{
outString = nvp.value;
break;
}
}
}
Here's the decompiled third party dll code:
public IDataReader ExecSPGetDataReader(string sp, List<param> paramList)
{
IDataReader dataReader;
using (DbCommand dbC = this.setDBCommand(sp, paramList, true))
{
IDataReader dr = this._db.ExecuteReader(dbC);
this.setOutputParams(dbC, paramList);
dataReader = dr;
}
return dataReader;
}
It looks like the command gets disposed of, probably for the purpose of disposing the connection, but if that's true, how can anything be read from the returned IDataReader?
If the code uses the reader without disposing of it, it won't be explicitly returned to the pool, correct?
That is correct. Change the code to this:
using (IDataReader rdr =
db.ExecSPGetDataReader("dbo.someStoredProcedure", paramList))
{
if (rdr.Read())
{
List<nameValuePair> formValues =
Utils.nameValuePairs(rdr["valuepairs"].ToString());
foreach (nameValuePair nvp in formValues)
{
if (nvp.name.ToLower() == "name")
{
outString = nvp.value;
break;
}
}
}
}
The using statement will ensure that Dispose is called.
This is a simplified example, to isolate the purpose of the question. In my actual scneario, the ColumnReader returned by GetColumnReader will actually do more work than merely ReadLine.
If I run the following program, I will get an error when I try to call Reader(), because of course the TextReader has already been disposed by the using statement.
public class Play{
delegate string ColumnReader();
static ColumnReader GetColumnReader(string filename){
using (TextReader reader = new StreamReader(filename)){
var headers = reader.ReadLine();
return () => reader.ReadLine();
}
}
public static void Main(string[] args){
var Reader = GetColumnReader("Input.tsv");
Console.WriteLine(Reader());
}
}
Alternatively, I can remove the "using" and directly declare the TextReader, which would function, but now we no longer have a guarantee that the TextReader will be eventually closed.
Is there a way to add a "destructor" to the returned lambda function where I might be able to Dispose of the TextReader as soon as the lambda function goes out of scope (no more references)?
I also welcome other suggestions but wish to keep the basic closure structure (that is, fits into the scope of the question).
If you would not require lambda expression you could create enumerable instead.
Potentially moving using inside the =>{} may work in real code... Still probably not what you are looking for:
static ColumnReader GetColumnReader(string filename) {
return () => {
using (TextReader reader = new StreamReader(filename)) {
var headers = reader.ReadLine();
return reader.ReadLine();
}
};
}
Version with IEnumerable (if you alway finish iteration):
static IEnumerable<string> GetColumnReader(string filename) {
using (TextReader reader = new StreamReader("aa")) {
var headers = reader.ReadLine();
yield return reader.ReadLine();
}
}
You'll need to create custom IDisposable iterator if you want to support iterations to the middle of the enumeration. See how foreach handles iterators that implement IDisposable to handle such cases.
Essentially you need the scope of the disposable element outside of the delegate itself. In these situations I would make the delegate accept the disposable instance (I.e. TextReader) rather than a file name.
public class Play {
delegate string ColumnReader();
static ColumnReader GetColumnReader(string filename) {
return () => {
using (TextReader reader = new StreamReader(filename)) {
var headers = reader.ReadLine();
return reader.ReadLine();
}
};
}
public static void Main(string[] args) {
var Reader = GetColumnReader("Input.tsv");
Console.WriteLine(Reader());
}
}
Obviously, this will open / read one line / close the file every time you call the returned delegate.
If you need to open it once, and then keep it open while reading through several lines, you'll be better off with an iterator block, similar to this:
public class Play {
static IEnumerable<string> ReadLines(string filename) {
using (TextReader reader = new StreamReader(filename)) {
var headers = reader.ReadLine(); // I'm guessing you want to ignore this??
while (true) {
string line = reader.ReadLine();
if (line == null)
yield break;
yield return line;
}
}
}
public static void Main(string[] args) {
foreach (string line in ReadLines("Input.tsv"))
Console.WriteLine(line);
}
}
If you really want to preserve the closure semantics, you will need to add an argument for it. Something like bellow, but you have to take care of calling the dispose command.
public class Play {
enum ReaderCommand {
Read,
Close
}
delegate string ColumnReader(ReaderCommand cmd);
static ColumnReader GetColumnReader(string filename) {
TextReader reader = new StreamReader(filename);
var headers = reader.ReadLine();
return (ReaderCommand cmd) => {
switch (cmd) {
case ReaderCommand.Read:
return reader.ReadLine();
case ReaderCommand.Close:
reader.Dispose();
return null;
}
return null;
};
}
public static void Main(string[] args) {
var Reader = GetColumnReader("Input.tsv");
Console.WriteLine(Reader(ReaderCommand.Read));
Console.WriteLine(Reader(ReaderCommand.Read));
Reader(ReaderCommand.Close);
Console.ReadKey();
}
}
How is this any easier than simply returning the TextReader? Seems to me you're making things much more complicated just to achieve a particular coding style.
The onus will always be on the caller to dispose of whatever is returned correctly.
I'm sure your project will give you plenty of opportunity to flex your muscles - this time just keep it simple!
I really like the yield solution. I have wirten a simple code, it shows it works well, the resouce can be disposed, after client out of for-each.
static void Main(string[] args)
{
using (Resource resource = new Resource())
{
foreach (var number in resource.GetNumbers())
{
if (number > 2)
break;
Console.WriteLine(number);
}
}
Console.Read();
}
public class Resource : IDisposable
{
private List<int> _numbers = new List<int> { 1, 2, 3, 4, 5, 6, 7 };
public IEnumerable<int> GetNumbers()
{
foreach (var number in _numbers)
yield return number;
}
public void Dispose()
{
Console.WriteLine("Resource::Dispose()...");
}
}
I have the following code shape. It seems that I'm misunderstanding the C# method return values. How is it possible that a "full" enumerator gets returned as an empty one?
class ThingDoer
{
public NpgsqlDataReader DoQuery()
{
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader();
return dataread; // Debugger confirms that six data are enumerable here.
}
}
...
class OtherThing
{
public void higherLevelFunction()
{
NpgsqlDataReader result = myThingDoer.DoQuery();
result.Read(); // No data! result's enumerable returns nothing!
}
}
You don't detail where your connection is coming from. Assuming it's something like:
public NpgsqlDataReader DoQuery()
{
using(NpgsqlConnection = GetConnectionCode())
{
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader();
return dataread;
}//Connection closes at this using-scope being left because that triggers Dispose()
}
Then change it to:
public NpgsqlDataReader DoQuery()
{
bool ownershipPassed = false;
NpgsqlConnection conn = GetConnectionCode();
try
{
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader(CommandBehavior.CloseConnection);
ownershipPassed = true;
return dataread;
}
finally
{
if(!ownershipPassed)//only if we didn't create the reader than takes charge of the connection
conn.Dispose();
}
}
Then where you use the reader, you have to dispose it to in turn dispose the connection's underlying connection to the database:
public void higherLevelFunction()
{
using(NpgsqlDataReader result = myThingDoer.DoQuery())
result.Read();
}
NpgsqlCommand c = new NpgsqlCommand(...);
NpgsqlDataReader dataread = c.ExecuteReader();
The above lines are very local to the method DoQuery. So as soon as the control comes out of the method, every object created inside to this method loses its scope. Hence you're losing the data because it's a reference type you're referring to in the caller method.