We Keep Coding

Convert string to lambda expression that contains variables from other classes

What I've been trying to do is convert a string of the form:
"StudentDatabase.avgHeight > 1.7"
to a lambda expression that looks like this:
() => StudentDatabase.avgHeight > 1.7;
I tried something in the lines of this:
/* String splitting and parsing occurs here */
var comparison = Expression.GreaterThan(
Type.GetType("MyNamespace.StudentDatabase").GetField("avgHeight"),
Expression.Constant(1.7)
);
var lambda = Expression.Lambda<Func<bool>>(comparison).Compile();
Of course something like this wouldn't work since the GetField() method returns type FieldInfo and not Expression.
Here's a list about useful stuff you might want to know about my sample code:
The StudentDatabase class is a static class that contains a static field avgHeight.
I have already done the part of the code that parses the string so there's no need to include it in any provided solutions.
This is just an example so you can change the string and variable/class names if you wish so.
This is not an assignment so feel free to post source code. In fact, that would be greately appreciated.
TL;DR; What I'm trying to do is use LINQ Expressions to access variables from other places of the code.

I disagree with the following comments, Linq expressions is a viable way to do this sort of thing. The below code accomplishes it. However, please consider the following code:
namespace MyNamespace
{
class Program
{
static void Main(string[] args)
{
/* String splitting and parsing occurs here */
var comparison = Expression.GreaterThan(
Expression.Field(null, Type.GetType("MyNamespace.StudentDatabase").GetField("avgHeight")),
Expression.Constant(1.7)
);
var lambda = Expression.Lambda<Func<bool>>(comparison).Compile();
StudentDatabase.avgHeight = 1.3;
var result1 = lambda(); //is true
StudentDatabase.avgHeight = 2.0;
var result2 = lambda(); //is false
}
}
class StudentDatabase
{
public static double avgHeight = 1.3;
}
}
Should result2 be true or false? If you want it to be true, then you have more work to do.

I've created this as a sort of framework you can work off of. It does not use LINQ but will output the value specified by the string.
var type = Type.GetType("MyNamespace.StudentDatabase");
if (type != null)
{
var field = type.GetField("avgHeight");
if (field != null)
{
Func<bool> lambda = () => (double)field.GetValue(type) > 1.7;
}
}
There is some error checking you could add/remove. The other areas such as the > and 1.7 can be parsed elsewhere and inserted but this is how you could get a value from the strings.

I need to call an EntityFunctions method in a dynamic EF Filter generation. How?

We have implemented a method of dynamically creating Lambda filters using some reflection and other methods. Most of what I want is working beautifully. The problem is when it comes to DateTime values and equality/greater than/less than filters.
Our SQL server is storing the items as DateTime objects and sometimes the Time is specified for specific items. However, when displaying a list on the web, we are only showing the date. So when the user (using Kendo Grid) tries to filter the data to dates greater than or something similar, it is using midnight the day of as it parameter. This means all items that happened after midnight on that day are included when they shouldn't be. It also means that when we use a equals statement, nothing is returned as there is very little that happens exactly at midnight.
My research (using Stack Overflow) led me to using EntityFunctions.TruncateTime as a method. I'm not sure if I am to do it to both the field and the filtering value or not, but I can't even get past the value for now.
My first try was to set the right side of my comparison (the value portion) to a call to EntityFunctions.TruncateTime(filter.Value). This gave me a This function can only be invoked by linq to entities error. Further research lead me to using the Expression.Call method, but currently I am getting an exception of No method 'TruncateTime' on type 'System.Data.Objects.EntityFunctions' is compatible with the supplied arguments.
Below is my code where I try to make the call. I provided the entire function for context. I created a GetPropertyType function to let me know if I am filtering on a DateTime field. I also added a property to my Filter object to tell me if I should ignore time or not. If both are true, only then do I try and apply the TruncateTime function.
I've also tried specifying the type of DateTime as a type argument to the Expression.Constant call in the method, just in case it needed to be typed, but that didn't help either.
public static Expression GetLambdaFilters(ICollection<Filter> filters)
{
if (filters.Count > 0)
{
Type entityType = filters.ElementAt(0).EntityType;
var item = Expression.Parameter(entityType, entityType.Name);
Expression leftSide = null;
Expression rightSide = null;
Expression filterExpression = null;
foreach (var filter in filters)
{
Expression left = GetPropertyExpression(filter, item);
Expression comparison = null;
if (left is MethodCallExpression)
{
comparison = left;
}
else
{
Expression right = null;
if (!filter.IsCollectionQuery)
{
if (GetPropertyType(filter) == typeof (DateTime) && filter.IgnoreTime)
{
right = Expression.Call(typeof (EntityFunctions), "TruncateTime", null,
Expression.Constant(filter.Value));
}
else
{
right = Expression.Constant(filter.Value);
}
}
else
{
Filter innerFilter = new Filter();
innerFilter.IsCollectionQuery = false;
innerFilter.Operator = filter.Operator;
innerFilter.PropertyName = GetCollectionPropertyName(filter, item);
innerFilter.Type = filter.CollectionType;
innerFilter.Value = filter.Value;
List<Filter> innerfilters = new List<Filter>(){
innerFilter
};
right = GetLambdaFilters(innerfilters);
filter.Operator = FilterOperator.Any;
}
comparison = GetExpression(left, right, filter);
}
if (leftSide == null)
{
leftSide = comparison;
filterExpression = leftSide;
continue;
}
if (rightSide == null)
{
rightSide = comparison;
filterExpression = Expression.AndAlso(leftSide, rightSide);
continue;
}
filterExpression = Expression.AndAlso(filterExpression, comparison);
}
var func = typeof(Func<,>);
func.MakeGenericType(entityType, typeof(bool));
return Expression.Lambda(func.MakeGenericType(entityType, typeof(bool)), filterExpression, item);
}
else
{
return GetLambdaFilter(filters.First());
}
}

If you change the line:
right = Expression.Call(typeof (EntityFunctions), "TruncateTime", null,
Expression.Constant(filter.Value));
to
right = Expression.Call(typeof (EntityFunctions), "TruncateTime", null,
Expression.Convert(Expression.Constant(DateTime.Parse(filter.Value)), typeof(DateTime?)));
this should work. Its the boxing conversion part that wont happen without the convert call that is causing the error.

Expression Tree ToString() method generates strange string WHY?

I need string conversion of an Expression Tree so
I create an Expression Tree and use ToString method like this
var exp = ((Expression<Func<UserDetailInfo, bool>>) (x => x.OperationID == operationId)).ToString();
but result is strange
x => (x.OperationID == value(TCS.Proxy.PermissionProxy+<>c__DisplayClass5).operationId)
TCS.Proxy.PermissionProxy is my class in WCF proxy project !!! (I send expression from client to proxy)
but when I create this Expression myself everything is good
var entity = Expression.Parameter(typeof(UserDetailInfo));
var constant = Expression.Constant(operationId);
var e = Expression.Equal(Expression.Property(entity, "OperationID"), constant);
var exp = Expression.Lambda<Func<UserDetailInfo, bool>>(e, entity).ToString();
and result is ok
Param_0 => (Param_0.OperationID == operationId) // I Need this
How I can use ToString() can generates result like above ?
Why two results is different ?
* I Convert Expression to String for transfer it from client to WCF service so I need correct string for convert in server side from string to Expression

This is because your right hand side member is not a constant, it is a captured variable. The TCS.Proxy.PermissionProxy+<>c__DisplayClass5 part means in the class TCS.Proxy.PermissionProxy it had to create 5 new classes that holds values that where passed in via variable capture and this specific lambda uses the 5th one it created.
Your code (You never show your function so I made some guesses)
namespace TCS.Proxy
{
class PermissionProxy
{
public void SomeFunction()
{
int operationId = 0;
var exp = ((Expression<Func<UserDetailInfo, bool>>) (x => x.OperationID == operationId)).ToString()
}
}
}
Is getting re-written to something similar (It's actually a lot different but this example gets the point across) to
namespace TCS.Proxy
{
public class PermissionProxy
{
private class c__DisplayClass5
{
public int operationId;
}
public void SomeFunction()
{
int operationId = 0;
var <>c__DisplayClass5 = new c__DisplayClass5();
<>c__DisplayClass5.operationId = operationId;
var exp = ((Expression<Func<UserDetailInfo, bool>>) (x => x.OperationID == <>c__DisplayClass5.operationId)).ToString()
}
}
}
Which is different than what you manually created. If you want to "unbox" these custom classes you will need to write up a ExpressionVisitor that will go through the expression and re-write it in to the form you want to go over the wire.

Checking for Nulls on DB Record Mapping

How can I check for db null values in the attached code? Please understand I am a new C# convert...
What this code does is takes a IDataReader object and converts and maps it to a strongly-typed list of objects. But what I am finding is it completely errors out when there are null columns returned in the reader.
Converter
internal class Converter<T> where T : new()
{
// Declare our _converter delegate
readonly Func<IDataReader, T> _converter;
// Declare our internal dataReader
readonly IDataReader dataReader;
// Build our mapping based on the properties in the class/type we've passed in to the class
private Func<IDataReader, T> GetMapFunc()
{
// declare our field count
int _fc = dataReader.FieldCount;
// declare our expression list
List<Expression> exps = new List<Expression>();
// build our parameters for the expression tree
ParameterExpression paramExp = Expression.Parameter(typeof(IDataRecord), "o7thDR");
ParameterExpression targetExp = Expression.Variable(typeof(T));
// Add our expression tree assignment to the exp list
exps.Add(Expression.Assign(targetExp, Expression.New(targetExp.Type)));
//does int based lookup
PropertyInfo indexerInfo = typeof(IDataRecord).GetProperty("Item", new[] { typeof(int) });
// grab a collection of column names from our data reader
var columnNames = Enumerable.Range(0, _fc).Select(i => new { i, name = dataReader.GetName(i) }).AsParallel();
// loop through all our columns and map them properly
foreach (var column in columnNames)
{
// grab our column property
PropertyInfo property = targetExp.Type.GetProperty(column.name, BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
// check if it's null or not
if (property != null)
{
// build our expression tree to map the column to the T
ConstantExpression columnNameExp = Expression.Constant(column.i);
IndexExpression propertyExp = Expression.MakeIndex(paramExp, indexerInfo, new[] { columnNameExp });
UnaryExpression convertExp = Expression.Convert(propertyExp, property.PropertyType);
BinaryExpression bindExp = Expression.Assign(Expression.Property(targetExp, property), convertExp);
// add it to our expression list
exps.Add(bindExp);
}
}
// add the originating map to our expression list
exps.Add(targetExp);
// return a compiled cached map
return Expression.Lambda<Func<IDataReader, T>>(Expression.Block(new[] { targetExp }, exps), paramExp).Compile();
}
// initialize
internal Converter(IDataReader dataReader)
{
// initialize the internal datareader
this.dataReader = dataReader;
// build our map
_converter = GetMapFunc();
}
// create and map each column to it's respective object
internal T CreateItemFromRow()
{
return _converter(dataReader);
}
}
Mapper
private static IList<T> Map<T>(DbDataReader dr) where T : new()
{
try
{
// initialize our returnable list
List<T> list = new List<T>();
// fire up the lamda mapping
var converter = new Converter<T>(dr);
while (dr.Read())
{
// read in each row, and properly map it to our T object
var obj = converter.CreateItemFromRow();
// add it to our list
list.Add(obj);
}
// reutrn it
return list;
}
catch (Exception ex)
{
// make sure this method returns a default List
return default(List<T>);
}
}
I just don't quite understand where the column to typed object happens in here, so I'd try to do it myself... but I just don;t know where it is.
I know this probably won't help much, but the error I am getting is:
Unable to cast object of type 'System.DBNull' to type 'System.String'.
and it happens on the
internal T CreateItemFromRow()
{
return _converter(dataReader); //<-- Here
}
Note
This does not happen if I wrap the columns in the query itself with an ISNULL(column, ''), but I am sure you can understand that this is surely not a solution

The problem lies in the line convertExp = Expression.Convert(propertyExp, property.PropertyType). You can't expect to convert DbNull value to its equivalent in framework type. This is especially nasty when your type is a value type. One option is to check if the read value from db is DbNull.Value and in case yes, you need to find a compatible value yourself. In some cases people are ok with default values of those types in C#. If you have to do this
property = value == DBNull.Value ? default(T): value;
a generic implementation would look like (as far as the foreach in your converter class goes):
foreach (var column in columns)
{
var property = targetExp.Type.GetProperty(
column.name,
BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
if (property == null)
continue;
var columnIndexExp = Expression.Constant(column.i);
var propertyExp = Expression.MakeIndex(
paramExp, indexerInfo, new[] { columnIndexExp });
var convertExp = Expression.Condition(
Expression.Equal(
propertyExp,
Expression.Constant(DBNull.Value)),
Expression.Default(property.PropertyType),
Expression.Convert(propertyExp, property.PropertyType));
var bindExp = Expression.Assign(
Expression.Property(targetExp, property), convertExp);
exps.Add(bindExp);
}
Now this does an equivalent of
property = reader[index] == DBNull.Value ? default(T): reader[index];
You could avoid the double lookup of the reader by assigning it to a variable and using its value in the conditional check. So this should be marginally better, but a lil' more complex:
foreach (var column in columns)
{
var property = targetExp.Type.GetProperty(
column.name,
BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
if (property == null)
continue;
var columnIndexExp = Expression.Constant(column.i);
var cellExp = Expression.MakeIndex(
paramExp, indexerInfo, new[] { columnIndexExp });
var cellValueExp = Expression.Variable(typeof(object), "o7thPropValue");
var convertExp = Expression.Condition(
Expression.Equal(
cellValueExp,
Expression.Constant(DBNull.Value)),
Expression.Default(property.PropertyType),
Expression.Convert(cellValueExp, property.PropertyType));
var cellValueReadExp = Expression.Block(new[] { cellValueExp },
Expression.Assign(cellValueExp, cellExp), convertExp);
var bindExp = Expression.Assign(
Expression.Property(targetExp, property), cellValueReadExp);
exps.Add(bindExp);
}
This does the conditional check this way:
value = reader[index];
property = value == DBNull.Value ? default(T): value;

This is one of the most annoying problems in dealing with datasets in general.
The way I normally get around it is to convert the DBNull value to something more useful, like an actual null or even a blank string in some cases. This can be done in a number of ways, but just recently I've taken to using extension methods.
public static T? GetValueOrNull<T>(this object value) where T : struct
{
return value == null || value == DBNull.Value ? (T?) null : (T) Convert.ChangeType(value, typeof (T));
}
A handy extension method for nullable types, so for example:
int? myInt = DataSet.Tables[0].Rows[0]["DBNullInt"].GetValueOrNull<int>();
Or a more generic one to just convert a DBNull in to a null:
public static object GetValueOrNull(this object value)
{
return value == DBNull.Value ? null : value;
}
string myString DataSet.Tables[0].Rows[0]["DBNullString"].GetValueOrNull();
You'll then get a null string, rather than trying to put a DBNull in to a string.
Hopefully that may help you a little.

As I come across this problem recently
both
Expression.TypeIs(propertyExp,typeof(DBNull));
and
Expression.Equal(propertyExp,Expression.Constant(DBNull.Value));
didn't work for me as they did increase memory allocation (which is my primary concern in this case)
here is the benchmark for both mapper approach compare to Dapper on 10K rows query.
TypeIs
and
Equal
so to fix this problem it came out that an IDataRecord is able to call "IsDBNull" to check whether the column in current reader is DBNull or not
and can be write as expression like
var isReaderDbNull = Expression.Call(paramExp, "IsDBNull", null, readerIndex);
finally, I end up with this solution
and now the performance is acceptable again.

Expression Tree with string assignment and getting value

I have built my own SQL Query builder that breaks apart an Expression, however, I'm having an issue trying to get the value of string defined in the same function as the lambda expression.
Here is what I am trying to do in console app:
private static void MyBuilderTest()
{
var sqlBuilder = new SqlBuilder();
// Doesn't work -- NEED GUIDANCE HERE
var testValue = "Test"; // Defined in the same function as the lambda below
sqlBuilder.Select<FooObject>(o => o.FooValue == testValue);
// Works
var someObject = new SomeObject { SomeValue = "classTest };
sqlBuilder.Select<FooObject>(o => o.FooValue == someObject.SomeValue);
}
In my builder it subclasses from ExpressionVisitor, and I override the VisitMember. I found that a string defined in at the base Console level will come back as:
Node.Expression.NodeType == ExpressionType.Constant
The Node.Expression passes back properties of:
CanReduce = false
DebugView = ".Constant<ConsoleApplication1.Program+<>c__DisplayClass1>(ConsoleApplication1.Program+<>c__DisplayClass1)"
NodeType = Constant
Type = System.Type {System.RunetimeType}
Value = {ConsoleApplication1.Program}
The Node.Expression.Value contains:
testValue = "Test" (Type: string)
How do I get this value? I've tried several things, like:
var memberType = node.Expression.Type.DeclaringType;
This passes back a ConsoleApplication1.Program type.
However, when I do:
memberType.GetProperty("testValue"); // Declaring Type from Expression
It passes back null.
The above methods work fine if I place the lambda "strings" in a class, but doesn't work if they string is defined in the console function.
Can anyone tell me how to get the string value if it's defined at the function level of the lambda?
EDITED: Added VisitMember
protected override Expression VisitMember(MemberExpression node)
{
if (node.NodeType == ExpressionType.Constant)
{
// Node.Expression is a ConstantExpression type.
// node.Expression contains properties above
// And Has Value of: {ConsoleApplication1.Program}
// Expanding Value in Watch window shows: testValue = "Test"
// How do I get this value, if the ConsoleApplication1.Program type doesn't
// even know about it? Looks like maybe a dynamic property?
}
}
EDITED
Added code to the console app example to show what works and what doesn't.

The lambda in your example has "closed over" the testValue variable, meaning the compiler has captured it as a field of the same name in an automatically generated class called ConsoleApplication1.Program+<>c__DisplayClass1>. You can use normal reflection to get the current value of that field by casting the right hand-side of the binary expression into a MemberExpression.
var testValue = "hello";
var expr = (Expression<Func<string, bool>>) (x => x == testValue);
var rhs = (MemberExpression) ((BinaryExpression) expr.Body).Right;
var obj = ((ConstantExpression) rhs.Expression).Value;
var field = (FieldInfo) rhs.Member;
var value = field.GetValue(obj);
Debug.Assert(Equals(value, "hello"));
testValue = "changed";
value = field.GetValue(obj);
Debug.Assert(Equals(value, "changed"));
Alternatively you can change your variable into a constant.
const string testValue = "hello";
var expr = (Expression<Func<string, bool>>) (x => x == testValue);
var value = ((ConstantExpression) ((BinaryExpression) expr.Body).Right).Value;
Debug.Assert(Equals(value, "hello"));

Instead of doing this by yourself, have a look at PartialEvaluator from Matt Warren. It replaces all references to constants with the constants themselves.