We Keep Coding

Utilizing Funcs within expressions?

Background
I have an example of a test that passes but an error that happens down the pipeline and I'm not sure why. I'd like to figure out what's going on but I'm new to Expression construction and don't want to make any assumptions.
This is for a search filtering mechanism. It uses ServiceStack's PredicateBuilder implementation. I essentially have a list of values that I pass in, and I want it to construct an expression tree. I had previously done this just with Func<T<bool>> but realized that I needed to wind up with Expression<Func<T<bool>>>. Bummer.
The Goal
Search filters built from Re-usable search filter types, which built out of Funcs and Expressions that allows me to pass in a field name from an object along with values I should match on and wind up with something that we can run a Where() statement against.
The Code / Issue
The generic "nullable bool" filter I'm trying -- sets up the acceptable items and returns a func that is meant to help filter:
public class NullableBoolFilter : IGenericSearchFilter<bool?>
{
public Func<bool?, bool> GetFilterFunc(string valuesToProcess)
{
var acceptableValues = new List<bool?>();
if (string.IsNullOrWhiteSpace(valuesToProcess))
{
// all values acceptable
acceptableValues = new List<bool?>{true, false, null};
}
else
{
if (!valuesToProcess.Contains("0") && !valuesToProcess.Contains("1"))
{
throw new ArgumentException("Invalid Nullable boolean filter attribute specified");
}
if (valuesToProcess.Contains("0"))
{
acceptableValues.Add(false);
}
if (valuesToProcess.Contains("1"))
{
acceptableValues.Add(true);
}
}
Func<bool?, bool> returnFunc = delegate(bool? item) { return acceptableValues.Any(x=>x == item); };
return returnFunc;
}
}
Then I have another filter, which inherits from the NullableBoolFilter and attempts to use the Func:
public class ClaimsReportIsMDLFilter : NullableBoolFilter, ISearchFilter<vSEARCH_ClaimsReport>
{
public Expression<Func<vSEARCH_ClaimsReport, bool>> GetExpression(string valuesToProcess)
{
var theFunc = base.GetFilterFunc(valuesToProcess);
Expression<Func<vSEARCH_ClaimsReport, bool>> mdlMatches = item => theFunc(item.IsMDL);
var predicate = PredicateBuilder.False<vSEARCH_ClaimsReport>();
predicate = predicate.Or(mdlMatches);
return predicate;
}
}
The following test passes:
public class ClaimsReportIsMDLFilterTests
{
// ReSharper disable InconsistentNaming
private readonly vSEARCH_ClaimsReport ItemWithMDL = new vSEARCH_ClaimsReport { IsMDL = true };
private readonly vSEARCH_ClaimsReport ItemWithoutMDL = new vSEARCH_ClaimsReport { IsMDL = false };
private readonly vSEARCH_ClaimsReport ItemWithNullMDL = new vSEARCH_ClaimsReport { IsMDL = null };
// ReSharper restore InconsistentNaming
[Fact]
public void WithSearchValueOf1_HidesNonMDLAndNull()
{
var sut = this.GetCompiledExpressionForValues("1");
sut.Invoke(ItemWithMDL).Should().BeTrue();
sut.Invoke(ItemWithoutMDL).Should().BeFalse();
sut.Invoke(ItemWithNullMDL).Should().BeFalse();
}
private Func<vSEARCH_ClaimsReport, bool> GetCompiledExpressionForValues(string searchValue)
{
return new ClaimsReportIsMDLFilter().GetExpression(searchValue).Compile();
}
}
The Problem
When I actually attempt to run this, I receive the error:
variable 'param' of type 'vSEARCH_ClaimsReport' referenced from scope '', but it is not defined
It makes sense to me why this might occur -- at the time it's evaluated, I don't have a real object to pass into the Func. However, I'm confused as to why my tests might pass but this doesn't in actual usage.
Questions
Why might my tests pass but I still receive this error?
How the heck should I begin trying to fix this?
Is there a remotely easy way to take that Func and turn it into an Expression that I can pass a field into?
Do I need to abandon the generic filter idea and have each class manually add expressions to the PredicateBuilder based on input passed in? That's doable, but it seems like the work could be reduced more.

Why might my tests pass [...]
Because your test is simply compiling the expression down into the code that it represents and invoking it. It doesn't need to actually parse the expression tree and look at what the code it represents is doing, it just runs it and ensures that the output is right.
Why might [...] I still receive this error?
Because when you're actually using it, it's not just executing the code; rather it is looking through the expression tree to try to determine what the code is doing so that it can be translated into something else, not so that it can be run as C# code.
Your expression is doing nothing but calling a delegate. There is no way for someone traversing the expression tree to see inside the delegate and know what it's doing. Knowing that you're calling another method isn't something that can be translated into another language.
How the heck should I begin trying to fix this?
You need to generate an Expression from the start, rather than generating a Func and then just creating an Expression that calls it.
Is there a remotely easy way to take that Func and turn it into an Expression that I can pass a field into?
No. You'd need to pull out the IL code of the function, decompile that into C# code, then build up Expression objects to represent that code. That's pretty much just not going to happen.
You're pretty much going to need to have GetFilterFunc return an Expression, to get this to work. Fortunately, this is quite easy to do, given what you have. You simply need to change the method signature and to replace the last two lines with the following:
return item => acceptableValues.Any(x => x == item);
And voila. The lambda can be compiled into an Expression object, rather than a delegate, based on context, so if the return type of the method is an Expression<Func<bool?,bool>> that's what you'll get.
Now, to use this in GetExpression. First off, the PredicateBuilder isn't really doing anything. Adding an OR FALSE to your expression changes nothing meaningful about it. All of that can go. All that leaves us with is using an Expression<Func<bool?,bool>> and changing it into an Expression<Func<vSEARCH_ClaimsReport, bool>> by pulling out a boolean property. To do this is a bit more work for expressions than for delegates. Rather than just invoking the expression, we need to do a tad more work to compose them. We'll want to write a method to do this operation:
public static Expression<Func<TFirstParam, TResult>>
Compose<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
And this relies on the use of the following method to replace all instances of one expression with another:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
What this is doing is replacing all instances of the second expression's parameter with the body of the first expression, effectively inlining that expression into the second. The rest is simply replacing all of the parameters with a new single parameter and wrapping it back up into a lambda.
Now that we have that, our method is quite easy:
public Expression<Func<vSEARCH_ClaimsReport, bool>> GetExpression(
string valuesToProcess)
{
Expression<Func<vSEARCH_ClaimsReport, bool?>> selector =
item => item.IsMDL;
return selector.Compose(base.GetFilterFunc(valuesToProcess));
}

SingleOrNewIdentified(o => o.prop, value), setting o.prop when new

I'd like a C# SingleOrNewIdentified(o => o.prop, value) function, responsible for returning a new o with o.prop = value preset when Any(o.prop == value) is not found. Unfortunately my understanding of expressions and lambdas is poor.
I frequently find myself needing to uniquely set just the one identifier (e.g. emailAddress, oauthId, GUID, natural key) before using a new EF object in a write-many manner. This write-once assignment often makes me shuffle some blocks around in a less-than-clear way.
It seems likely that it can be done since I'd be fine with reflection. I wouldn't want to defeat type checking in consumers of this function in any way, though.
Warnings about this idea are also welcome.

public static T SingleOrNewIdentified<T, TProp>(this IQueryable<T> queryable, Expression<Func<T, TProp>> propExpr, TProp value) where T : class, new()
{
var property = (PropertyInfo)((MemberExpression)propExpr.Body).Member;
var paramExpr = Expression.Parameter(typeof(T));
var eqExpr = Expression.Equal(Expression.Property(paramExpr, property), Expression.Constant(value));
var predicate = Expression.Lambda<Func<T, bool>>(eqExpr, paramExpr);
T maybe = queryable.SingleOrDefault(predicate);
if (maybe != null) return maybe;
T #default = new T();
property.SetValue(#default, value);
return #default;
}

Not sure if this is what you want but it seems you can do it this way: if your collection is null, assign a default:
var foo = /*db.FooTable.*/SingleOrDefault(x => /*your where clause*/) ?? new Foo
{
EmailMember = "me#foo.now"
};

It's not what I was looking for, but I thought I should note a related functionality for EF (update-or-create-by-indentity) for simplifying attaching at the end rather than beginning of your process, for entity keys only:
db.Attach(model);
db.SaveChanges();
It's been easy for me to forget that EF will do this work for you, but it maps especially well to UIs where create and update are very similar. You have to wait for SaveChanges() for a generated entity key of course.

Expression.Bind() - what does it actually do?

So I've been playing with dynamically building expression trees lately and came across this method, which seems kinda odd. At first I thought "oh cool this is exactly what I need" after constantly writing code along the lines of
var left = member is FieldInfo ? Expression.Field(instance, (FieldInfo)member) : Expression.Property(instance, (PropertyInfo)member);
var right = ...
var assign = Expression.Assign(left, right);
Yes, I know there is Expression.PropertyOrField() call, but it does roundtrip back to reflection to find member by name, where as I typically already have MemberInfo instance.
So anyway, I thought Expression.Bind() would be useful to me, but it does something I don't really understand. Given the following code:
void Main()
{
var m = typeof(Foo).GetField("Bar");
Expression.Bind(m, Expression.Constant("")).Dump();
}
public class Foo
{
public string Bar;
}
it produces MemberAssignment expression Bar = "". But there is no instance and no static reference. How would I ever apply this expression to and instance of Foo? I can't find any example of this method being used.

Object-Initializer expressions.
Let's say you had:
public class Foo
{
public int Property { get; set; }
}
Then you could do:
var parameter = Expression.Parameter(typeof(int), "i");
var newExpr = Expression.New(typeof(Foo));
var bindExprs = new[]
{
Expression.Bind(typeof(Foo).GetProperty("Property"), parameter)
// You can bind more properties here if you like.
};
var body = Expression.MemberInit(newExpr, bindExprs);
var lambda = Expression.Lambda<Func<int, Foo>>(body, parameter);
which is something like:
i => new Foo { Property = i }
Old:
I can't help you solve the "performance issue" you are determined to solve (will using Expression.PropertyOrField really introduce a bottleneck in your application? I'm somewhat skeptical. You should determine this before optimizing prematurely) (EDIT: Apologies for incorrectly assuming that this was a perf optimization, and as you've found out yourself, Expression.MakeMemberAccess is what you need), but I can tell you what Expression.Bind is useful for.

Using an expression tree to read the name and value of a property. Is there an alternative?

Assumptions
Suppose I have a class with a property:
class ClassWithProperty
{
public string Prop { get; private set; }
public ClassWithProperty(string prop)
{
this.Prop = prop;
}
}
And now suppose I have created an instance of that class:
var test = new ClassWithProperty("test value");
What I want
I want this printed out to the console:
Prop = 'test value'
Piece of cake! I use this code to produce the wanted output:
Console.WriteLine("Prop = '{1}'", test.Prop);
Problem
I am violating DRY, because "Prop" appears twice in the code above. If I refactor the class and change the name of the property, I also have to change the string literal. There is also a lot of boilerplate code, if I had a class with many properties.
Proposed solution
string result = buildString(() => c.Prop);
Console.WriteLine(result);
Where the buildString method looks like this:
private static string buildString(Expression<Func<string>> expression)
{
MemberExpression memberExpression = (MemberExpression)expression.Body;
string propertyName = memberExpression.Member.Name;
Func<string> compiledFunction = expression.Compile();
string propertyValue = compiledFunction.Invoke();
return string.Format("{0} = '{1}'", propertyName, propertyValue);
}
Question
The above solution works fine and I am happy with it, but if there is some more easier and less "scary" way to solve this, that would make me much happier. Is there some easier alternative to achieve the same result with less and simpler code? Maybe something without expression trees?
Edit
Based on Manu's fine idea (see below) I could use this extension method:
static public IEnumerable<string> ListProperties<T>(this T instance)
{
return instance.GetType().GetProperties()
.Select(p => string.Format("{0} = '{1}'",
p.Name, p.GetValue(instance, null)));
}
It's great for getting a string representation for all properties for an instance.
But: From this enumeration, how could I pick type-safely a specific property? Again I would use expression trees ... or am I not seeing the wood for the trees?
Edit 2
Using reflection or expression trees here is a matter of taste.
Luke's idea using projection initializers is just brilliant. From his answer I finally build this extension method (which is basically just a LINQ'd version of his answer):
public static IEnumerable<string> BuildString(this object source)
{
return from p in source.GetType().GetProperties()
select string.Format("{0} = '{1}'", p.Name, p.GetValue(source, null));
}
Now a call will look like this:
new { c.Prop }.BuildString().First()
Which looks a bit nicer than the my original call with a lambda (but this is also a matter of taste I guess). Luke's suggestion is however superior to my solution, since it allows specifying as many properties as you like (see below).

You could pass an anonymous type to a BuildStrings method, taking advantage of projection initializers to automatically create the names and values of the anonymous type's properties. Inside the method you would use reflection to interrogate those properties.
This would also allow you to pass multiple items if you wished. You could also pass fields and locals as well as properties, because they'd all be projected as properties of the anonymous type that can then be interrogated with GetProperties etc. (Of course, all that the method is actually doing is enumerating all properties of the object that's passed-in. You could pass any type.)
string result = BuildStrings(new { test.Prop }).First();
Console.WriteLine(result);
// or
string foo = "Test";
int bar = 42;
string results = BuildStrings(new { foo, bar, test.Prop });
foreach (string r in results)
{
Console.WriteLine(r);
}
// ...
public static IEnumerable<string> BuildStrings(object source)
{
return source.GetType().GetProperties().Select(
p => string.Format("{0} = '{1}'", p.Name, p.GetValue(source, null)));
}

I actually like your original idea of using an expression tree. It's a good use case for ET. But you make it look a little bit scary because you compile and execute an expression tree to get the value of the property, while all you need is just the name. Make the method return only the name and then use it as you did in your first "non-DRY" attempt.
private static string buildString(Expression<Func<string>> expression)
{
MemberExpression memberExpression = (MemberExpression)expression.Body;
return memberExpression.Member.Name;
}
And then
Console.WriteLine("{0} = {1}", buildString(() => c.Prop), c.Prop);
It doesn't look that scary. Yes, you use c.Prop twice here, but I think you get a significant performance improvement, because you don't need expression tree compilation. And you still don't use any string literals.

var listOfPropertyNamesAndValues = this.GetType().GetProperties()
.Select(prop => string.Format("{0} = '{1}'",
prop.Name, prop.GetValue(this,null)));
If you want to fetch a specific Property, you'll have to pass it's name as a string and get it via reflection (just add a where clause to above query). The good news are that you are not violating DRY anymore, the bad news are that it's not typesafe.

Reflection?

I've seen it done with a delegate and IL interrogation, but that isn't exactly simpler. In short, no: there is no infoof in C#. Here's Eric Lippert's take on this: In Foof We Trust: A Dialogue

What's the best way to define & access selected properties in C#?

From my recent question, I try to centralize the domain model by including some silly logic in domain interface. However, I found some problem that need to include or exclude some properties from validating.
Basically, I can use expression tree like the following code. Nevertheless, I do not like it because I need to define local variable ("u") each time when I create lambda expression. Do you have any source code that is shorter than me? Moreover, I need some method to quickly access selected properties.
public void IncludeProperties<T>(params Expression<Func<IUser,object>>[] selectedProperties)
{
// some logic to store parameter
}
IncludeProperties<IUser>
(
u => u.ID,
u => u.LogOnName,
u => u.HashedPassword
);
Thanks,

Lambdas are great for many scenarios - but if you don't want them, perhaps simply don't use them? I hate to say it, but simple strings are tried and tested, especially for scenarios like data binding. If you want fast access, you could look at HyperDescriptor, or there are ways of compiling a delegate to the property accessors, or you can build an Expression from the string and compile it (including a cast to object if you want a known signature, rather than calling the (much slower) DynamicInvoke).
Of course, in most cases even crude reflection is fast enough, and isn't the bottleneck.
I suggest starting with the simplest code, and check it is actually too slow before worrying about it being fast. If it isn't too slow, don't change it. Any of the above options would work otherwise.
Another thought; if you are using Expression, you could do something like:
public void IncludeProperties<T>(
Expression<Func<T,object>> selectedProperties)
{
// some logic to store parameter
}
IncludeProperties<IUser>( u => new { u.ID, u.LogOnName, u.HashedPassword });
and then take the expression apart? A bit tidier, at least... here's some sample code showing the deconstruction:
public static void IncludeProperties<T>(
Expression<Func<T, object>> selectedProperties)
{
NewExpression ne = selectedProperties.Body as NewExpression;
if (ne == null) throw new InvalidOperationException(
"Object constructor expected");
foreach (Expression arg in ne.Arguments)
{
MemberExpression me = arg as MemberExpression;
if (me == null || me.Expression != selectedProperties.Parameters[0])
throw new InvalidOperationException(
"Object constructor argument should be a direct member");
Console.WriteLine("Accessing: " + me.Member.Name);
}
}
static void Main()
{
IncludeProperties<IUser>(u => new { u.ID, u.LogOnName, u.HashedPassword });
}
Once you know the MemberInfos (me.Member in the above), building your own lambdas for individual access should be trivial. For example (including a cast to object to get a single signature):
var param = Expression.Parameter(typeof(T), "x");
var memberAccess = Expression.MakeMemberAccess(param, me.Member);
var body = Expression.Convert(memberAccess, typeof(object));
var lambda = Expression.Lambda<Func<T, object>>(body, param);
var func = lambda.Compile();

Here's the shortest expression I can come up with:
public static void IncludeProperties(Expression<Action<IUser>> selectedProperties)
{
// some logic to store parameter
}
public static void S(params object[] props)
{
// dummy method to get to the params syntax
}
[Test]
public void ParamsTest()
{
IncludeProperties(u => S(
u.Id,
u.Name
));
}