In my application I need to display a list of records returned by different stored procedures. Each store procedure returns different types of records (ie the number of columns and the column type are different).
My original thought was to create a class for each type of records and create a function which would execute the corresponding stored procedure and return List< MyCustomClass>. Something like this:
public class MyCustomClass1
{
public int Col1 { get; set; } //In reality the columns are NOT called Col1 and Col1 but have proper names
public int Col2 { get; set; }
}
public static List<MyCustomClass1> GetDataForReport1(int Param1)
{
List<MyCustomClass1> output = new List<MyCustomClass1>();
using (SqlConnection cn = new SqlConnection("MyConnectionString"))
using (SqlCommand cmd = new SqlCommand("MyProcNameForReport1", cn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#Param1", SqlDbType.Int).Value = Param1;
SqlDataReader rdr=cmd.ExecuteReader();
int Col1_Ordinal = rdr.GetOrdinal("Col1");
int Col2_Ordinal = rdr.GetOrdinal("Col2");
while (rdr.Read())
{
output.Add(new MyCustomClass1
{
Col1 = rdr.GetSqlInt32(Col1_Ordinal).Value,
Col2 = rdr.GetSqlInt32(Col2_Ordinal).Value
});
}
rdr.Close();
}
return output;
}
This works fine but as I don't need to manipulate those records in my client code (I just need to bind them to a graphical control in my application layer) it doesn't really make sense to do it this way as I would end up with plenty of custom classes that I wouldn't actually use. I found this which does the trick:
public static DataTable GetDataForReport1(int Param1)
{
DataTable output = new DataTable();
using (SqlConnection cn = new SqlConnection("MyConnectionString"))
using (SqlCommand cmd = new SqlCommand("MyProcNameForReport1", cn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#Param1", SqlDbType.Int).Value = Param1;
output.Load(cmd.ExecuteReader());
}
return output;
}
This returns a DataTable which I can bind to whatever control I use in my application layer. I'm wondering if using a DataTable is really needed.
Couldn't I return a list of objects created using anonymous classes:
public static List<object> GetDataForReport1(int Param1)
{
List<object> output = new List<object>();
using (SqlConnection cn = new SqlConnection("MyConnectionString"))
using (SqlCommand cmd = new SqlCommand("MyProcNameForReport1", cn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#Param1", SqlDbType.Int).Value = Param1;
SqlDataReader rdr=cmd.ExecuteReader();
int Col1_Ordinal = rdr.GetOrdinal("Col1");
int Col2_Ordinal = rdr.GetOrdinal("Col2");
while (rdr.Read())
{
output.Add(new
{
Col1 = rdr.GetSqlInt32(Col1_Ordinal).Value,
Col2 = rdr.GetSqlInt32(Col2_Ordinal).Value
});
}
rdr.Close();
}
return output;
}
Any other ideas? Basically I just want the function to return 'something' which I can bind to a graphical control and I prefer not to have to create custom classes as they wouldn’t really be used. What would be the best approach?
The good news is: this isn't the first time the entities vs datasets issue has come up. It's a debate much older than my own programming experience. If you don't want to write DTO's or custom entities then your options are DataTables/DataSets or you could reinvent this wheel again. You wouldn't be the first and you won't be the last. The argument for DTO's/Entities is that you don't have the performance overheads that you get with DataSets. DataSets have to store a lot of extra information about the datatypes of the various columns, etc...
One thing, you might be happy to hear is that if you go the custom entity route and you are happy for your object property names to match the column names returned by your sprocs, you can skip writing all those GetDataForReport() mapping functions where you are using GetOrdinal to map columns to properties. Luckily, some smart monkeys have properly sussed that issue here.
EDIT: I was researching an entirely different problem today (binding datasets to silverlight datagrids) and came across this article by Vladimir Bodurov, which shows how to transform an IEnumerable of IDictionary to an IEnumerable of dynamically created objects (using IL). It occured to me that you could easily modify the extension method to accept a datareader rather than the IEnumerable of IDictionary to solve your issue of dynamic collections. It's pretty cool. I think it would accomplish exactly what you were after in that you no longer need either the dataset or the custom entities. In effect you end up with a custom entity collection but you lose the overhead of writing the actual classes.
If you are lazy, here's a method that turns a datareader into Vladimir's dictionary collection (it's less efficient than actually converting his extension method):
public static IEnumerable<IDictionary> ToEnumerableDictionary(this IDataReader dataReader)
{
var list = new List<Dictionary<string, object>>();
Dictionary<int, string> keys = null;
while (dataReader.Read())
{
if(keys == null)
{
keys = new Dictionary<int, string>();
for (var i = 0; i < dataReader.FieldCount; i++)
keys.Add(i, dataReader.GetName(i));
}
var dictionary = keys.ToDictionary(ordinalKey => ordinalKey.Value, ordinalKey => dataReader[ordinalKey.Key]);
list.Add(dictionary);
}
return list.ToArray();
}
If you're going to XmlSerialize them and want that to be efficient, they can't be anonymous. Are you going to send back the changes? Anonymous classes wont be much use for that.
If you're really not going to do anything with the data other than gridify it, a DataTable may well be a good answer for your context.
In general, if there's any sort of interesting business login going on, you should be using Data Transfer Objects etc. rather than just ferrying around grids.
If you use anonymous classes, you are still defining each type. You just get to use Type Inference to reduce the amount of typing, and the class type will not clutter any namespaces.
The downside of using them here is that you want to return them out of the method. As you noted, the only way to do this is to cast them to objects, which makes all of their data inaccessable without reflection! Are you never going to do any processing or computation with this data? Display one of the columns as a picture instead of a value, or hide it based on some condition? Even if you are only ever going to have a single conditional statement, doing it with the anonymous class version will make you wish you hadn't.
Related
I have a mapping function that reads from the data loaded from the database.
It maps onto a class that is through generic.
The problem is that it takes 3 minutes for looping through a 10000 records.
I would like to improve its performance I am looking for a solution for this?
public static List<T> GetList<T>(string query = null, string whereClause = null, Dictionary<string, string> Params = null)
{
var results = new List<T>();
var properties = GetReadableProperties<T>();
query = QueryMaker.SelectQuery<T>(query, whereClause);
using (SqlConnection Connection = new SqlConnection(ConnectionString))
{
Connection.Open();
SqlCommand cmd = GetSqlCommandWithParams(query, Connection, Params);
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
// From here
while (reader.Read())
{
var item = Activator.CreateInstance<T>();
foreach (var property in properties)
{
DBReader(item, property, reader);
}
results.Add(item);
}
// To here. It takes 3 minutes. reading a 10000 record from database into reader isn't as slow as this code is.
}
Connection.Close();
return results;
}
}
This is the DBReader function:
private static void DBReader(Object _Object, PropertyInfo property, SqlDataReader reader)
{
if (!reader.IsDBNull(reader.GetOrdinal(property.Name)))
{
Type convertTo = Nullable.GetUnderlyingType(property.PropertyType) ?? property.PropertyType;
property.SetValue(_Object, Convert.ChangeType(reader[property.Name], convertTo), null);
}
else
{
property.SetValue(_Object, null);
}
}
I can see a number of things you could theoretically improve. For example:
store the result of GetOrdinal to reuse when getting values.
use a generic new() instead of Activator.CreateInstance.
Rather than using reflection, you could precompile an expression into a Func<> and invoke it for each row.
Call the appropriate Get...() methods based on the desired type to avoid boxing and unboxing.
But all of these are micro-optimizations and even combining them all together is unlikely to make a significant difference. The amount of time you're describing (minutes for tens of thousands of rows) is unlikely to be caused by any of these kinds of issues. It's most likely coming from I/O operations somewhere, but it's impossible to be certain without profiling the code to see where the most time is being spent. Profile your code, and then use those results to inform what approach you take.
I'm attempting to make a method that returns a ObservableCollection by providing a object type & a table name.
The method should use this information to make a ObservableCollection which retrieves all rows from table and puts it into the previous ObservableCollection with the desired object type.
public class DatabaseManager
{
public async Task<ObservableCollection<object>> GetObjectsAsync(object object_name, string table_name)
{
Type type = object_name.GetType();
IList<PropertyInfo> props = new List<PropertyInfo>(type.GetProperties());
ObservableCollection<object> oc = new ObservableCollection<object>();
using (SqlConnection conn = new SqlConnection(Constants.ConnectionString))
{
await conn.OpenAsync();
using (SqlCommand cmd = conn.CreateCommand())
{
cmd.CommandText = "select * from " + table_name;
using (SqlDataReader reader = await cmd.ExecuteReaderAsync())
{
while (await reader.ReadAsync())
{
object x = new object();
foreach (PropertyInfo prop in props)
{
// I want to create a universal object that i can add to my observable collection here.
object propValue = prop.GetValue(object_name, null);
// Code Here
}
oc.Add(x);
}
}
}
}
return oc;
}
}
The goal is to make a method for getting a ObservableCollection from a Database Table with a none specific object type.
I would appreciate any help / recommendations, I might have the wrong mind set of how to do this, but if it is at all possible please tell me.
C# is a statically typed language, so generally you cannot build an object without knowing its type. In other words, when you do this
object x = new object();
you literally get an object of type System.Object, with no additional properties.
There are two ways of working around this:
Use dynamic and ExpandoObject - this would let you add properties on the fly, but the resulting objects would need to be used as dynamic, or
Take the type of the object that will be produced by the query - your code can "link" table name to objects stored in the table. If you decide to go this route, consider using a lightweight ORM library, such as StackExchange's Dapper.
I am working with pulling data from a DB in WPF and doing it with multiple different tables, which I have modeled as POCO's in C#. I want to be able to create a method that is flexible enough to handle any of these POCO's being passed in, and then return an observable collection of the same class to the caller.
I have something set up that I haven't tested yet, but I know this probably isn't the best way to implement this so I wanted to get some advice on how to best do this before I even bothered troubleshooting or attempting to get this to work:
public static ObservableCollection<object> SQLAuthentication(ObservableCollection<object> myCollection, object myClass, String sql)
{
var conn = new SqlConnection();
var paramList = GenerateSQLParameters(myClass, null);
var tempModel = Global.GenerateNewInstance(myClass);
//get the type
Type model = tempModel.GetType();
var prop = model.GetProperties();
PropertyInfo pi;
using (getConnection(conn))
{
conn.Open();
SqlCommand cmd;
SqlDataReader reader;
cmd = new SqlCommand(sql, conn);
reader = cmd.ExecuteReader();
while (reader.Read())
{
//set the values for each property in the model
foreach (var p in prop)
{
pi = tempModel.GetType().GetProperty(p.Name);
pi.SetValue(tempModel, reader[p.Name]);
}
myCollection.Add(tempModel);
}
reader.Close();
cmd.Dispose();
}
return myCollection;
}
You can use generic types in the method signature its self, you dont need to use object collection... and as im writing this John B hit the nail on its head
public static ObservableCollection<T> SQLAuthentication(ObservableCollection<T> myCollection, T myClass, String sql) where T: class
#John, if you post it as an answer ill remove this since you did technically shout it out first, just leave a comment if you do so i get the notification
I use ExecuteReader().
It returns only the last result. I wanted to display the result like an array in tbid_1.Text, tbid_1.Text, tbid_1.Text etc.
public void Select(FrmVIRGO frm)
{
string query = "SELECT* FROM tb_patient_information ";
if (this.OpenConnection() == true)
{ //Create Command
MySqlCommand cmd = new MySqlCommand(query, connection);
//Create a data reader and Execute the command
MySqlDataReader dataReader = cmd.ExecuteReader();
while (dataReader.Read())
{
// I think use like this
frm.tbid_1.Text = dataReader["id_patient"][1].ToString(); //... id_patient1
frm.tbid_2.Text = dataReader["id_patient"][2].ToString(); //... id_patient2
frm.tbid_3.Text = dataReader["id_patient"][3].ToString(); //... id_patient3
}
//close Data Reader
dataReader.Close();
//close Connection
this.CloseConnection();
}
}
Your code appears to be expecting that once you've called dataReader.Read(), you can access all of the records by an index.
Your data reader is an instance of IDataReader, the interface that most .NET data access libraries use to represent the concept of "reading the results of a query". IDataReader only gives you access to one record at a time. Each time you call dataReader.Read(), the IDataReader advances to the next record. When it returns false, it means that you have reached the end of the result set.
For example, you could transform your code above to something like this:
dataReader.Read(); // dataReader is at 1st record
frm.tbid_1.Text = dataReader["id_patient"].ToString();
dataReader.Read(); // dataReader is at 2nd record
frm.tbid_2.Text = dataReader["id_patient"].ToString();
dataReader.Read(); // dataReader is at 3rd record
frm.tbid_3.Text = dataReader["id_patient"].ToString();
Note that this is not the way you should do it, I'm just using it to illustrate the way a DataReader works.
If you are expecting exactly 3 records to be returned, you could use something similar to the code above. I would modify it to verify that dataReader.Read() returns true before you read data from each record, however, and handle a case where it doesn't in a meaningful way (e.g., throw an exception that explains the error, log the error, etc.).
Generally though, if I am working with raw ADO.Net (as opposed to using an OR/M) I prefer to convert each record in the IDataReader to a dictionary beforehand, and work with those.
For example, you could write the following extension method for DataReader:
public static class DataReaderExtensions
{
public static IList<IDictionary<string, object>> ListRecordsAsDictionaries(this IDataReader reader)
{
var list = new List<IDictionary<string, object>>();
while (reader.Read())
{
var record = new Dictionary<string, object>();
for (var i = 0; i < reader.FieldCount; i++)
{
var key = reader.GetName(i);
var value = reader[i];
record.Add(key, value);
}
list.Add(record);
}
return list;
}
}
This method iterates over the IDataReader, and sticks the values from each row into a Dictionary<string, object>. I have found this pattern to generally be fairly useful when dealing with raw ADO stuff.
This approach has a couple of caveats:
You don't get access to the records (the Dictionary instances) until all of the data has been received from the server. If you are handling records individually as the DataReader makes them available, you may actually be able to start processing the data while some of it is still in-transit. (Note: This could be fixed by making this method return IEnumerable<IDictionary<string, object>> instead of IList<IDictionary<string, object>>, and using yield return to yield each record as it becomes available.)
If you are iterating over a large data set, you may not want to instantiate that many dictionaries. It may be better to just handle each record individually instead.
You lose access to some of the information that DataReader can provide about the record(s) (e.g., you can't use DataReader.GetDataTypeName as you are iterating over the records).
This isn't related to a particular issue BUT is a question regarding "best practise".
For a while now, when I need to get data straight from the database I've been using the following method - I was wondering if there's a faster method which I don't know about?
DataTable results = new DataTable();
using (SqlConnection connection = new SqlConnection(ConfigurationManager.ConnectionStrings["Name"]))
{
connection.Open();
using (SqlCommand command = new SqlCommand("StoredProcedureName",connection))
{
command.CommandType = CommandType.StoredProcedure;
/*Optionally set command.Parameters here*/
results.Load(command.ExecuteReader());
}
}
/*Do something useful with the results*/
There are indeed various ways of reading data; DataTable is quite a complex beast (with support for a number of complex scenarios - referential integrity, constraints, computed values, on-the-fly extra columns, indexing, filtering, etc). In a lot of cases you don't need all that; you just want the data. To do that, a simple object model can be more efficient, both in memory and performance. You could write your own code around IDataReader, but that is a solved problem, with a range of tools that do that for you. For example, you could do that via dapper with just:
class SomeTypeOfRow { // define something that looks like the results
public int Id {get;set;}
public string Name {get;set;}
//..
}
...
var rows = connection.Query<SomeTypeOfRow>("StoredProcedureName",
/* optionalParameters, */ commandType: CommandType.StoredProcedure).ToList();
which then very efficiently populates a List<SomeTypeOfRow>, without all the DataTable overheads. Additionally, if you are dealing with very large volumes of data, you can do
this in a fully streaming way, so you don't need to buffer 2M rows in memory:
var rows = connection.Query<SomeTypeOfRow>("StoredProcedureName",
/* optionalParameters, */ commandType: CommandType.StoredProcedure,
buffered: false); // an IEnumerable<SomeTypeOfRow>
For completeness, I should explain optionalParameters; if you wanted to pass #id=1, #name="abc", that would be just:
var rows = connection.Query<SomeTypeOfRow>("StoredProcedureName",
new { id = 1, name = "abc" },
commandType: CommandType.StoredProcedure).ToList();
which is, I think you'll agree, a pretty concise way of describing the parameters. This parameter is entirely optional, and can be omitted if no parameters are required.
As an added bonus, it means you get strong-typing for free, i.e.
foreach(var row in rows) {
Console.WriteLine(row.Id);
Console.WriteLine(row.Name);
}
rather than having to talk about row["Id"], row["Name"] etc.