I am using the following generic data helper function to run queries and return them as a collection of DbDataRecords.
public static IEnumerable<DbDataRecord> GetDataRecords(string Query, Dictionary<string, object> Parameters, CommandType CommandType = CommandType.Text)
{
using (MySqlConnection Connection = CreateConnection())
{
using (MySqlCommand cmd = new MySqlCommand(Query, Connection))
{
cmd.CommandType = CommandType;
foreach (var Parameter in Parameters)
{
cmd.Parameters.AddWithValue(Parameter.Key, Parameter.Value);
}
Connection.Open();
using (MySqlDataReader Reader = cmd.ExecuteReader())
{
foreach (DbDataRecord record in Reader)
{
yield return record;
}
}
Connection.Close();
}
}
}
This works great for multiple results, but I'd also like to create a function that returns a single record from the reader as a single DbDataRecord. What I cant figure out is how to convert a single row from a reader into a DbDataRecord. Here is what I have so far:
public static DbDataRecord GetDataRecord(string Query, Dictionary<string, object> Parameters, CommandType CommandType = CommandType.Text)
{
DbDataRecord Record = null;
using (MySqlConnection Connection = CreateConnection())
{
using (MySqlCommand cmd = new MySqlCommand(Query, Connection))
{
cmd.CommandType = CommandType;
foreach (var Parameter in Parameters)
{
cmd.Parameters.AddWithValue(Parameter.Key, Parameter.Value);
}
Connection.Open();
using (MySqlDataReader Reader = cmd.ExecuteReader())
{
if(Reader.Read() != false)
{
Record = ???;
}
}
Connection.Close();
}
}
return Record;
}
I have seen lots of examples that show how to return one column (ExecuteScalar), that is not what I am looking for. I also know how to get individual column values. Again, that is not what I am looking to do.
Basically I want to be replace the following foreach loop
foreach (DbDataRecord record in Reader)
{
yield return record;
}
with something that converts a single reader row into a DBDataRecord
You could return IDataRecord instead, that could be sufficient, then you can return the MySqlDataReader directly. Or you have to cast it accordingly:
So this is not possible directly:
Record = (DbDataRecord)Reader;
but in this way:
Record = (DbDataRecord)(IDataRecord)Reader;
Related
I currently have a DB library used for database access and I use it across several projects. I currently use the following code to get a recordset.
METHODS
public static IDataReader GetRs(string sql)
{
using (var con = NewSqlConnection())
{
con.Open();
return GetRs(sql, con);
}
}
public static IDataReader GetRs(string sql, SqlConnection dbconn)
{
using (var cmd = new SqlCommand(sql, dbconn))
{
int tries = 1;
while (tries <= 3)
{
try
{
if (dbconn.State == ConnectionState.Closed)
{
dbconn.Open();
}
DataTable myTable = new DataTable();
var reader = cmd.ExecuteReader();
myTable.Load(reader);
return myTable.CreateDataReader();
//return cmd.ExecuteReader();
}
catch (SqlException ex)
{
if (ex.Message.Contains("Timeout expired") || ex.Number == 1205) // Deadlock
{
Thread.Sleep(1000);
if (tries == 3)
{
throw ex;
}
tries += 1;
cmd.CommandTimeout *= 10;
}
else
{
throw ex;
}
}
}
}
throw new Exception("Could not get RecordSet");
}
USAGE
public static void Test()
{
using(var reader = GetRs("SELECT Col FROM TABLE"))
{
while(reader.Read())
{
// do stuff with data here e.g. var value = reader[0];
}
}
}
While this method works, as you can see it loads the entire dataset into memory thus causing issues with scaling.
I tried replacing the following code in the GetRs(string sql, SqlConnection con) method
DataTable myTable = new DataTable();
var reader = cmd.ExecuteReader();
myTable.Load(reader);
return myTable.CreateDataReader();
and tried returning just the return cmd.ExecuteReader();
However an error is thrown on the while (reader.Read()) - Invalid attempt to call read when the reader is closed. I am guessing this is because the SqlConnection property is disposed (and hence closed) after returning the IDataReader.
I'm aware that I can wrap the GetRs method with a new sql connection but this means rewriting a lot of my code, and I was hoping that I would be able to dispose the reader AND the connection with my using(var reader = GetRs()) method.
Is there any way I can still use these methods without loading the whole dataset into memory?
You can inject in your code with a Action<DataTableReader> parameter to you GetRs call.
Try this:
public static void GetRs(string sql, Action<DataTableReader> consumer)
{
using (var con = NewSqlConnection())
{
con.Open();
GetRs(sql, con, consumer);
}
}
public static void GetRs(string sql, SqlConnection dbconn, Action<DataTableReader> consumer)
{
using (var cmd = new SqlCommand(sql, dbconn))
{
if (dbconn.State == ConnectionState.Closed)
{
dbconn.Open();
}
DataTable myTable = new DataTable();
var reader = cmd.ExecuteReader();
myTable.Load(reader);
consumer(myTable.CreateDataReader());
}
}
(I removed your try/catch code for clarity.)
Then you call it like this:
public static void Test()
{
GetRs("SELECT Col FROM TABLE", reader =>
{
while(reader.Read())
{
// do stuff with data here e.g. var value = reader[0];
}
});
}
Your connection object was closed by the following line
using (var con = NewSqlConnection())
{
con.Open();
return GetRs(sql, con);
}//Connection object gets released here
But still, you are returning the reader object which was created by SqlCommand object using the above connection object. Hence the reader object is closed.
I'm currently executing my stored procedure below, and it works perfectly. But I can't specify the command timeout.
var results = await _dbContext.DbContext.Database.SqlQuery<GetOutputDto>(#"[dbo].[GetOutput] " + parameterString, list.ToArray()).ToListAsync();
Now I've change this to the below, and wondering what's the best way to convert the result to an object. I have over 30 properties, so setting each value would be quite tedious. Was wondering if there's a clean solution as Entity Framework solution.
using (var conn = new SqlConnection(_dbContextProvider.DbContext.Database.Connection.ConnectionString))
{
conn.Open();
SqlCommand cmd = new SqlCommand(#"[dbo].[GetOutput]", conn);
cmd.CommandTimeout = 60;
cmd.CommandType = CommandType.StoredProcedure;
foreach (var item in list)
{
cmd.Parameters.Add(item);
}
cmd.ExecuteNonQuery();
cmd.Connection.Close();
// How to get the result to entity in a clean manner.
}
Using System.reflection in those situation is really handy.
public static List<T> Convert<T>(IDataReader dr) where T : class, new()
{
List<T> list = new List<T>();
T obj = default(T);
while (dr.Read()) {
obj = Activator.CreateInstance<T>();
foreach (PropertyInfo prop in obj.GetType().GetProperties()) {
if (!object.Equals(dr[prop.Name], DBNull.Value)) {
prop.SetValue(obj, dr[prop.Name], null);
}
}
list.Add(obj);
}
return list;
}
using (var conn = new SqlConnection(_dbContextProvider.DbContext.Database.Connection.ConnectionString))
{
conn.Open();
SqlCommand cmd = new SqlCommand(#"[dbo].[GetOutput]", conn);
cmd.CommandTimeout = 60;
cmd.CommandType = CommandType.StoredProcedure;
foreach (var item in list)
{
cmd.Parameters.Add(item);
}
using ( var reader = cmd.ExecuteReader() ){
List<Entity> result = Convert<Entity>(reader); // convert to entity.
cmd.Connection.Close();
}
}
I would in all honesty send over as an array and convert to table type within SQL and do the dirty work on the server side. Also a good way to be able to specify the timeout can be done by either the connection strings within your config file or you can also pass that same parameter over to sql with a WAITFOR DELAY.
Cheers!
Not that hard, do it like this
note, this is lazy eval so it should perform well when there is user IO, still fairly fast in other cases, I've used it in data ETL projects with many records.
public static IEnumerable<dynamic>( /* params */)
{
// build command object here.
using (SqlDataReader reader = cmd.ExecuteReader())
{
if (reader.Read()) // read the first one to get the columns collection
{
var cols = reader.GetSchemaTable()
.Rows
.OfType<DataRow>()
.Select(r => r["ColumnName"]);
do
{
dynamic t = new System.Dynamic.ExpandoObject();
foreach (string col in cols)
{
((IDictionary<System.String, System.Object>)t)[col] = reader[col];
}
yield return t;
} while (reader.Read());
}
}
// remember to close connection
}
From my simple DB framework https://gist.github.com/hoganlong/b7f5c5e8dde61ae3cd6f
I have been trying to read single Record using AutoMapper. But I couldn't get through using the below code,
public ChartOfAccount GetSingleCOA( )
{
string queryString = "select * from ChartofAccounts where AccNo=423";
using (var connection = new SqlConnection(conStr))
using (var command = new SqlCommand(queryString, connection))
{
connection.Open();
using (var reader = command.ExecuteReader())
if (reader.HasRows)
{
reader.Read();
return Mapper.Map<IDataRecord, ChartOfAccount>(reader);
}
}
return null;
}
simply changing the line to
return Mapper.DynamicMap<IDataReader, ChartOfAccount>(reader);
solved the issue
Consider the FetchData method below. It is designed to avoid duplicating the database query code every time you want to fetch some data from the database.
List<MyData> myData = new List<MyData();
FetchData((IDataReader reader) =>
{
myData.Add(new MyData(reader.GetString(0), reader.GetInt32(1)));
}, "usp_get_my_data");
It works, but it would be nice if this object creation could somehow live inside the FetchData method.
Is there a better approach?
Perhaps FetchData can be modified to return a list of some type X directly?
protected void FetchData(Action<IDataReader> processor, String query)
{
using (var connection = CreateConnection())
{
connection.ConnectionString = ConnectionString;
connection.Open();
using (var command = connection.CreateCommand())
{
command.CommandType = CommandType.StoredProcedure;
command.CommandText = query;
using (IDataReader reader = command.ExecuteReader())
{
while (reader.read())
{
processor(reader);
}
}
}
}
}
Using generics maybe?
protected IEnumerable<T> FetchData<T>(Func<IDataReader, T> processor, String query)
{
using (var connection = CreateConnection())
{
connection.ConnectionString = ConnectionString;
connection.Open();
using (var command = connection.CreateCommand())
{
command.CommandType = CommandType.StoredProcedure;
command.CommandText = query;
using (IDataReader reader = command.ExecuteReader())
{
while (reader.read())
{
yield return processor(reader);
}
}
}
}
}
Then you can use the enumeration however you want:
var myData = FetchData<MyData>(reader => new MyData(reader.GetString(0), reader.GetInt32(1)), "usp_get_my_data").ToList();
You might take a look at Dapper.Net which is single file ORM with three different helpers:
Execute a query and map the results to a strongly typed List
Execute a query and map it to a list of dynamic objects
Execute a Command that returns no results
Design patterns used to build this miro-ORM are so helpful. Dapper.Net is currently being used in StackOverflow.
The list will grow and shrink depending on how many items I have in my database.
I need to populate a list not a listbox. I understand I will need to open a connection.
using (var conn = new SqlConnection(Properties.Settings.Default.DBConnectionString))
{
using (var cmd = conn.CreateCommand())
{
conn.Open();
List<string> TagList = new List<string>();
for (int i = 0; i < TagList.Count; i++)
TagList[i].Add("Data from database");
cmd.ExecuteNonQuery();
}
}
I'm really not sure how to do this and I'm sure my method up here looks very wrong so I really need help.
Could someone show me what I'm doing wrong?
public IEnumerable<string> GetTagList()
{
using (var connection = new SqlConnection(Properties.Settings.Default.DBConnectionString))
using (var cmd = connection.CreateCommand())
{
connection.Open();
cmd.CommandText = "select Tag from TagsTable"; // update select command accordingly
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
yield return reader.GetString(reader.GetOrdinal("Tag"));
}
}
}
}
then you can call it as below
List<string> tags = GetTagList().ToList();
I would like to share my solution, hope helps someone in the future:
public List<string> getFromDataBase()
{
List<string> result = new List<string>();
using(SqlConnection con = new SqlConnection("connectionString"))
{
con.Open();
DataTable tap = new DataTable();
new SqlDataAdapter(query, con).Fill(tap);
result = tap.Rows.OfType<DataRow>().Select(dr => dr.Field<string>("columnName")).ToList();
}
return result;
}
This would do as it is (if I didn't do any typos...)
private void LoadList()
{
List<string> tagsList = new List<string>();
using (IDbConnection connection = new SqlConnection(Properties.Settings.Default.DBConnectionString))
{
connection.Open();
using (IDbCommand command = connection.CreateCommand())
{
command.CommandText = "SELECT TAGCOLUMN FROM TAGSTABLE";
using (IDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
if (!reader.IsDBNull(0))
tagsList.Add(reader.GetString(0));
}
reader.Close();
}
}
connection.Close();
}
}
EDIT:
Of course you have to change the select statement to the correct one from your database.
I just used a pseudo one to show you what to put there.