We Keep Coding

Can I define custom type mapping for parameters and fields in PetaPoco?

I'm currently trying to pick a C# ORM to use with my PostgreSQL database, and I'm interested in the micro-ORMs, since they allow me to better utilize the power of Postgres(and since full blown ORMs are hard to configure. While Dapper simply works, trying to deal with NHibernate has left a forehead shaped dent in my screen...)
Anyways, currently PetaPoco has the lead, but there is one feature I need and can't figure if it has(to be fair - I couldn't find it in the other ORMs either) - mapping of custom types.
My PostgreSQL database uses the hstore and Postgis extensions, which define custom types. I don't expect any ORM to support those types(it's hard enough to find one that supports PostgreSQL!) but I want to be able to provide my own mappers for them, so when I get them as columns or send them as parameters PetaPoco will automatically use my mappers.
Is this even possible? The closest I could find is IDbParameter support, but those are built-in types and I need to write mappers for extension types that are not part of the list...

Based on Schotime's comment, I came with half a solution - how to parse the hstore from the query results into the object. I'm leaving this question open in case someone wants to get the other solution.
I need to define my own mapper. Obviously I want to use PetaPoco's default mapping for regular types, so it's only natural to inherit PetaPoco.StandardMapper - but that won't work, because StandardMapper implements PetaPoco.IMapper's fields without the virtual attribute - so I can't override them(I can only overshadow them, but that's not really helping).
What I did instead was to implement IMapper directly, and delegate regular types to an instance of PetaPoco.IMapper:
public class MyMapper:PetaPoco.IMapper{
private PetaPoco.StandardMapper standardMapper=new PetaPoco.StandardMapper();
public PetaPoco.TableInfo GetTableInfo(Type pocoType){
return standardMapper.GetTableInfo(pocoType);
}
public PetaPoco.ColumnInfo GetColumnInfo(PropertyInfo pocoProperty){
return standardMapper.GetColumnInfo(pocoProperty);
}
public Func<object, object> GetFromDbConverter(PropertyInfo TargetProperty, Type SourceType){
if(TargetProperty.PropertyType==typeof(HStore)){
return (x)=>HStore.Create((string)x);
}
return standardMapper.GetFromDbConverter(TargetProperty,SourceType);
}
public Func<object, object> GetToDbConverter(PropertyInfo SourceProperty){
if(SourceProperty.PropertyType==typeof(HStore)){
return (x)=>((HStore)x).ToSqlString();
}
return standardMapper.GetToDbConverter(SourceProperty);
}
}
The HStore object is constructed similarly to the one in Schotime's gist.
I also need to register the mapper:
PetaPoco.Mappers.Register(Assembly.GetAssembly(typeof(MainClass)),new MyMapper());
PetaPoco.Mappers.Register(typeof(HStore),new MyMapper());
Now, all of this works perfectly when I read from the query - but not when I write query parameters(even though I defined GetToDbConverter. It seems my mapper simply isn't called when I'm writing query parameters. Any idea how to do that?

Is this an appropriate use of generics and C#'s dynamic data type?

The problem I'm having is thus, we're building a data access layer using our existing ORM (it's an old one called Gentle) with the idea of moving to something like Fluent NHibernate. There are a few queries where we have to add custom clauses to the SqlBuilder in our existing setup, so for instance when retrieving some person objects we might be adding a clause like:
"PersonId in (SELECT PersonId from Orders where OrderValue > " + orderValue + " and OrderName = " + orderName
The point being that the parameters are being added directly in a string rather than as a parameterised query, it is possible in Gentle to add it as a parameterised query and this is what I've been working on. All our DALs inherit from a base GentleDAL, this is the class that actually constructs the Gentle query, adds the clauses and parameters etc. To add a parameterised clause in Gentle you have to do two things with your SqlBuilder object, you have to call sb.AddConstraint(string clause) to add your clause, and then for each parameter you have to call sb.AddParameter(string name, Type type), you can then construct your SqlStatement object from this, and only after that can you set the value for your parameter where you call stmt.SetParameter(string name, object value).
The way I have represented these parameters/clauses is I have created a class called GentleClauseCollection, this contains the clauses and parameters and has Add and Get methods for both of these things. Clauses are just strings and are stored internally in a List, the parameters are stored in a GentleParameter class which uses generics. The full code for GentleParameter is as follows.
public class GentleParameter<TParamType>
{
public string Name { get; private set; }
public TParamType Value { get; private set; }
public Type ParameterType {get { return typeof (TParamType); }}
public GentleParameter(string parameterName, TParamType parameterValue)
{
Name = parameterName;
Value = parameterValue;
}
}
There is no collection in .NET that I'm aware of that would let me store GentleParameter for different values of TParamType in the same collection, however it can be done using the DLR. In my GentleCollection class I store the parameters in a List and I get the parameters from this class as an IEnumerable. The Add method in my class is able to only allow GentleParameter's to be added so I know that my parameters will always have a Name, Value and ParameterType field which I can access.
My questions are: Given I could sacrifice the generics and change my parameter class Value property to be 'object' instead of T, have I overcomplicated things by using dynamic, what are the pros and cons of both approaches? Is there a third way to do this that I haven't thought of and how significant a performance impact am I likely to see by using dynamic given that all the method calls using the dynamic objects will be compiled at run time?
Thanks in advance for your help.

As sb.SetParameter is not generic and awaits an object, I would not make GentleParameter generic and hence I would not use the DLR.

Using dynamic doesn't seem over complicated to me. Method calls are resolved at runtime and it's cached the resulting invocation probably averages about 10x slower (we are talking nanoseconds). So it depends on how you are going to use it if it makes sense.
If you are always going to use it as type Object than yes you don't need to be using type dynamic not that it would have hurt anything.
If you want to be able to access properties than yes you should use dynamic, the result code will look cleaner than anything else you could do.
But even using dynamic you don't necessarily have to call the properties themselves dynamically, if you want to have as much static typing is possible you can have dynamic resolve a helper method that takes a generic form of your GentleParameter and do your work inside that.
...
private void HelperDoStuffWithGenericParam<T>(GentleParameter<T>param){
//Do stuff you have the static typing
}

Using Dynamic LINQ (or Generics) to query/filter Azure tables

So here's my dilemma. I'm trying to utilize Dynamic LINQ to parse a search filter for retrieving a set of records from an Azure table. Currently, I'm able to get all records by using a GenericEntity object defined as below:
public class GenericEntity
{
public string PartitionKey { get; set; }
public string RowKey { get; set; }
Dictionary<string, object> properties = new Dictionary<string, object>();
/* "Property" property and indexer property omitted here */
}
I'm able to get this completely populated by utilizing the ReadingEntity event of the TableServiceContext object (called OnReadingGenericEvent). The following code is what actually pulls all the records and hopefully filter (once I get it working).
public IEnumerable<T> GetTableRecords(string tableName, int numRecords, string filter)
{
ServiceContext.IgnoreMissingProperties = true;
ServiceContext.ReadingEntity -= LogType.GenericEntity.OnReadingGenericEntity;
ServiceContext.ReadingEntity += LogType.GenericEntity.OnReadingGenericEntity;
var result = ServiceContext.CreateQuery<GenericEntity>(tableName).Select(c => c);
if (!string.IsNullOrEmpty(filter))
{
result = result.Where(filter);
}
var query = result.Take(numRecords).AsTableServiceQuery<GenericEntity>();
IEnumerable<GenericEntity> res = query.Execute().ToList();
return res;
}
I have TableServiceEntity derived types for all the tables that I have defined, so I can get all properties/types using Reflection. The problem with using the GenericEntity class in the Dynamic LINQ Query for filtering is that the GenericEntity object does NOT have any of the properties that I'm trying to filter by, as they're really just dictionary entries (dynamic query errors out). I can parse out the filter for all the property names of that particular type and wrap
"Property[" + propName + "]"
around each property (found by using a type resolver function and reflection). However, that seems a little... overkill. I'm trying to find a more elegant solution, but since I actually have to provide a type in ServiceContext.CreateQuery<>, it makes it somewhat difficult.
So I guess my ultimate question is this: How can I use dynamic classes or generic types with this construct to be able to utilize dynamic queries for filtering? That way I can just take in the filter from a textbox (such as "item_ID > 1023000") and just have the TableServiceEntity types dynamically generated.
There ARE other ways around this that I can utilize, but I figured since I started using Dynamic LINQ, might as well try Dynamic Classes as well.
Edit: So I've got the dynamic class being generated by the initial select using some reflection, but I'm hitting a roadblock in mapping the types of GenericEntity.Properties into the various associated table record classes (TableServiceEntity derived classes) and their property types. The primary issue is still that I have to initially use a specific datatype to even create the query, so I'm using the GenericEntity type which only contains KV pairs. This is ultimately preventing me from filtering, as I'm not able to do comparison operators (>, <, =, etc.) with object types.
Here's the code I have now to do the mapping into the dynamic class:
var properties = newType./* omitted */.GetProperties(
System.Reflection.BindingFlags.Instance |
System.Reflection.BindingFlags.Public);
string newSelect = "new(" + properties.Aggregate("", (seed, reflected) => seed += string.Format(", Properties[\"{0}\"] as {0}", reflected.Name)).Substring(2) + ")";
var result = ServiceContext.CreateQuery<GenericEntity>(tableName).Select(newSelect);
Maybe I should just modify the properties.Aggregate method to prefix the "Properties[...]" section with the reflected.PropertyType? So the new select string will be made like:
string newSelect = "new(" + properties.Aggregate("", (seed, reflected) => seed += string.Format(", ({1})Properties[\"{0}\"] as {0}", reflected.Name, reflected.PropertyType)).Substring(2) + ")";
Edit 2: So now I've hit quite the roadblock. I can generate the anonymous types for all tables to pull all values I need, but LINQ craps out on my no matter what I do for the filter. I've stated the reason above (no comparison operators on objects), but the issue I've been battling with now is trying to specify a type parameter to the Dynamic LINQ extension method to accept the schema of the new object type. Not much luck there, either... I'll keep you all posted.

I've created a simple System.Refection.Emit based solution to create the class you need at runtime.
http://blog.kloud.com.au/2012/09/30/a-better-dynamic-tableserviceentity/

I have run into exactly the same problem (with almost the same code :-)). I have a suspicion that the ADO.NET classes underneath somehow do not cooperate with dynamic types but haven't found exactly where yet.

So I've found a way to do this, but it's not very pretty...
Since I can't really do what I want within the framework itself, I utilized a concept used within the AzureTableQuery project. I pretty much just have a large C# code string that gets compiled on the fly with the exact object I need. If you look at the code of the AzureTableQuery project, you'll see that a separate library is compiled on the fly for whatever table we have, that goes through and builds all the properties and stuff we need as we query the table. Not the most elegant or lightweight solution, but it works, nevertheless.
Seriously wish there was a better way to do this, but unfortunately it's not as easy as I had hoped. Hopefully someone will be able to learn from this experience and possibly find a better solution, but I have what I need already so I'm done working on it (for now).

Set or change Attribute's properties or fields at runtime in C#. Possible?

I believe there is no human way to change any attribute or field inside an Attribute apart from doing it in the constructor. That is, short of redesigning and recompiling Visual Studio yourself. There is already a similar question posted here:
Change Attribute's parameter at runtime
but I believe the peculiarities of my problem are different enough to require a new post.
I use an enumeration to keep track of the different columns of a DataTable. I use attributes in each enumeration element to indicate the underlying type and the description -in case the .ToString() would give an "ugly" result due to the rigid set of characters that are allowed to name an enumeration element, such as "Tomato_Field" when you want "Tomato Field", and the like. This allows me to place all the related information in the same object, which is, I believe, what it should be. This way I can later create all the columns with a simple and clean foreach that cycles through the elements of the enumeration and extracts the metedata (description and type) to create each column.
Now, some of the columns are autocalculated, which means that during their creation -via DataTable Identifier.Columns.Add.(NameOfColumn,underlyingType,optional: autocalculatedString)- I need to specify a string that determines how it should be calculated. That string must use the names of other columns, which might be in the Description Attribute. The approach that looks logical is to use another attribute that holds the string, which should be built using the names of the other columns, requiring access to the metadata. Now that seems impossible in the constructor: you are forced to provide a constant string. You can't use a method or anything.
This problem could be solved if there were a way to change a property inside the attribute (lets call it AutocalculatedStringAttribute) at runtime. If you access the metadata you can retrieve the string you used at the constructor of the Attribute, and you can of course change that string. However, if you later access the metadata again that change is ignored, I believe the constructor is called every time the metadata is accessed at runtime, thus ignoring any changes.
There are, of course, dirty ways to achive what I am trying to do, but my question is specifically if there is a way to properly use attributes for this. Short of resorting to CodeDOM to recompile the whole assembly with the constructor of the AutocalculatedStringAttribute changed, a certain overkill.

Right, the metadata that's used to initialize the attribute is immutable. But you can add properties and methods to an attribute class that can run code and return relevant info after the attribute object is constructed. The data they rely on doesn't have to be stored in metadata, it can be persisted anywhere.
Of course, such code wouldn't have to be part of the attribute class implementation, it could just as well be part of the code that instantiates the attribute. Which is where it belongs.

It isn't entirely clear to me what code is consuming this attribute, and it matters...
You cannot change an attribute that is burned into the code - you can query it with reflection, but that is about it. However, in many cases you can still do interesting things - I don't know if they apply to your scenario, though:
you can subclass many attributes like [Description], [DisplayName], etc - and while you pass in a constant string (typically a key) to the .ctor, it can return (through regular C#) more flexible values - perhaps looking up the description from a resx to implement i18n
if the caller respects System.ComponentModel, you can attach attributes at runtime to types etc very easily - but much harder on individual properties, especially in the case of DataTable etc (since that has a custom descriptor model via DataView)
you can wrap things and provide your own model via ICustomTypeDescriptor / TypeDescriptionProvider / PropertyDescriptor - lots of work, but provides access to set your own attributes, or return a description (etc) outside of attributes
I don't know how much of this is suitable for your environment (perhaps show some code of what you have and what you want), but it highlights that (re the question title) yes: there are things you can do to tweak how attributes are perceived at runtime.

I wanted to post this as a comment but since I wanted to include some code I couldn't, given the 600 characters limit. This is the cleanest solution I have managed to find, although it does not include all the info to create the columns on the enum, which is my goal. I have translated every field to make it easier to follow. I am not showing some code which has an obvious use (in particular the implementations of the other custom attributes and their static methods to retrieve the metadata, assume that it works).
This gets the job done, but I would ideally like to include the information stored in the strings "instancesXExpString " and "totalInstancesString" in the Autocalculated attribute, which currently only marks the columns that have such a string. This is what I have been unable to do and what, I believe, cannot be easily accomplished via subclassing -although it is an ingenious approach, I must say.
Thanks for the two prompt replies, btw.
And without any further ado, lets get to the code:
// Form in which the DataGridView, its underlying DataTable and hence the enumeration are:
public partial class MainMenu : Form {
(...)
DataTable dt_expTable;
//Enum that should have all the info on its own... but does not:
public enum e_columns {
[TypeAttribute(typeof(int))]
Experiments = 0,
[TypeAttribute(typeof(decimal))]
Probability,
[DescriptionAttribute("Samples / Exp.")]
[TypeAttribute(typeof(int))]
SamplesXExperiment,
[DescriptionAttribute("Instances / Sample")]
[TypeAttribute(typeof(int))]
InstancesXSample,
[DescriptionAttribute("Instances / Exp.")]
[TypeAttribute(typeof(int))]
[Autocalculated()]
InstancesXExp,
[DescriptionAttribute("Total Instances")]
[TypeAttribute(typeof(long))]
[Autocalculated()]
Total_Instances
};
//These are the two strings
string instancesXExpString = "[" + DescriptionAttribute.obtain(e_columns.SamplesXExperiment) + "] * [" + DescriptionAttribute.obtain(e_columns.InstancesXMuestra) + "]";
string totalInstancesString = "[" + DescriptionAttribute.obtain(e_columns.InstancesXExp) + "] * [" + DescriptionAttribute.obtain(e_columns.Experiments) + "]";
public MainMenu() {
InitializeComponent();
(...)
}
private void MainMenu_Load(object sender, EventArgs e) {
(...)
// This is the neat foreach I refered to:
foreach (e_columns en in Enum.GetValues(typeof(e_columnas))) {
addColumnDT(en);
}
}
private void addColumnDT(Enum en) {
//*This is a custom static method for a custom attrib. that simply retrieves the description string or
//the standard .ToString() if there is no such attribute.*/
string s_columnName = DescriptionAttribute.obtain(en);
bool b_typeExists;
string s_calculusString;
Type TypeAttribute = TypeAttribute.obtain(en, out b_typeExists);
if (!b_typeExists) throw (new ArgumentNullException("Type has not been defined for one of the columns."));
if (isCalculatedColumn(DescriptionAttribute.obtain(en))) {
s_calculusString = calcString(en);
dt_expTable.Columns.Add(s_columnName, TypeAttribute, s_calculusString);
} else {
dt_expTable.Columns.Add(s_columnName, TypeAttribute);
}
}
private string calcString(Enum en) {
if (en.ToString() == e_columns.InstancessXExp.ToString()) {
return instancesXExpString;
} else if (en.ToString() == e_columns.Total_Samples.ToString()) {
return totalInstancesString;
} else throw (new ArgumentException("There is a column with the autocalculated attribute whose calculus string has not been considered."));
}
(...)
}
I hope this piece of code clarifies the situation and what I am trying to do.

Fastest way to use reflection for converting datareader to list

I am using reflection to convert datareader into the generic collection list. Can anybody
suggest me the best way to implement reflection for this? I want the fastestway?

I assume what you want to do is something like:
List<MyClass> list = LoadFromDataReader<MyClass>(dataReader);
with:
class MyClass
{
[DataField("FirstName")] public string FirstName { get; set; }
[DataField("LastName")] public string LastName { get; set; }
}
I do this by:
Using Type.GetProperties and PropertyInfo.GetCustomAttribute to put together a dictionary mapping field names to PropertyInfo objects
Calling PropertyInfo.SetValue on each field in each record
You can cache the results of step (1), since the field/property mapping isn't going to change during the life of the application.
If performance is a problem (i.e. if step (2) turns out to be a bottleneck), then you have to avoid using reflection and generate code to set the properties directly. A couple of alternative improvements:
Use System.CodeDom to generate a C# class containing code to set the properties according to the respective fields on the IDataReader. Note that System.CodeDom invokes the csc.exe compiler in the background, so you need to generate this code once at startup and re-use it on each call.
Use System.Reflection.Emit.DynamicMethod to generate IL code that sets properties. Less runtime overhead than System.CodeDom, but since you're generating raw IL, this is much harder to write and debug. Use as a last option.

This really depends on exactly what you are looking at doing. I implement a object/interface process where I create information objects that hold the data that is returned. I then use an interface IFillable or something similar that passes a DR to the object and the object does the hydration from the DR.
This way I avoid the need for reflection, and the performance is great. I then have a few generic helper methods for Fill and FillCollection.
I got the idea based on stuff inside the CBO object of the DotNetNuke framework. It also implements a reflection method as well, that is fairly decent in performance.