Related
Background
My client would like to have a method of sending over an array of field (string), value (string), and comparison (enum) values in order to retrieve their data.
public class QueryableFilter {
public string Name { get; set; }
public string Value { get; set; }
public QueryableFilterCompareEnum? Compare { get; set; }
}
My company and I have never attempted to do anything like this before, so it is up to my team to come up with a viable solution. This is the result of working on a solution with a week or so of research.
What Works: Part 1
I have created a service that is able to retrieve the data from our table Classroom. Retrieval of the data is done in Entity Framework Core by way of LINQ-to-SQL. The way I have written below works if one of the fields that are supplied in the filter doesn't exist for Classroom but does exist for its related Organization (the client wanted to be able to search among organization addresses as well) and has a navigatable property.
public async Task<IEnumerable<IExportClassroom>> GetClassroomsAsync(
IEnumerable<QueryableFilter> queryableFilters = null) {
var filters = queryableFilters?.ToList();
IQueryable<ClassroomEntity> classroomQuery = ClassroomEntity.All().AsNoTracking();
// The organization table may have filters searched against it
// If any are, the organization table should be inner joined to all filters are used
IQueryable<OrganizationEntity> organizationQuery = OrganizationEntity.All().AsNoTracking();
var joinOrganizationQuery = false;
// Loop through the supplied queryable filters (if any) to construct a dynamic LINQ-to-SQL queryable
if (filters?.Count > 0) {
foreach (var filter in filters) {
try {
classroomQuery = classroomQuery.BuildExpression(filter.Name, filter.Value, filter.Compare);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
organizationQuery = organizationQuery.BuildExpression(filter.Name, filter.Value, filter.Compare);
joinOrganizationQuery = true;
} else {
throw new ArgumentException(ex.Message);
}
}
}
}
// Inner join the classroom and organization queriables (if necessary)
var query = joinOrganizationQuery
? classroomQuery.Join(organizationQuery, classroom => classroom.OrgId, org => org.OrgId, (classroom, org) => classroom)
: classroomQuery;
query = query.OrderBy(x => x.ClassroomId);
IEnumerable<IExportClassroom> results = await query.Select(ClassroomMapper).ToListAsync();
return results;
}
What Works: Part 2
The BuildExpression that exists in code is something that I created as such (with room for expansion).
public static IQueryable<T> BuildExpression<T>(this IQueryable<T> source, string columnName, string value, QueryableFilterCompareEnum? compare = QueryableFilterCompareEnum.Equal) {
var param = Expression.Parameter(typeof(T));
// Get the field/column from the Entity that matches the supplied columnName value
// If the field/column does not exists on the Entity, throw an exception; There is nothing more that can be done
MemberExpression dataField;
try {
dataField = Expression.Property(param, propertyName);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
throw new ArgumentException($"Queryable selection does not have a \"{propertyName}\" field.", ex.ParamName);
} else {
throw new ArgumentException(ex.Message);
}
}
ConstantExpression constant = !string.IsNullOrWhiteSpace(value)
? Expression.Constant(value.Trim(), typeof(string))
: Expression.Constant(value, typeof(string));
BinaryExpression binary = GetBinaryExpression(dataField, constant, compare);
Expression<Func<T, bool>> lambda = (Expression<Func<T, bool>>)Expression.Lambda(binary, param)
return source.Where(lambda);
}
private static Expression GetBinaryExpression(MemberExpression member, ConstantExpression constant, QueryableFilterCompareEnum? comparisonOperation) {
switch (comparisonOperation) {
case QueryableFilterCompareEnum.NotEqual:
return Expression.Equal(member, constant);
case QueryableFilterCompareEnum.GreaterThan:
return Expression.GreaterThan(member, constant);
case QueryableFilterCompareEnum.GreaterThanOrEqual:
return Expression.GreaterThanOrEqual(member, constant);
case QueryableFilterCompareEnum.LessThan:
return Expression.LessThan(member, constant);
case QueryableFilterCompareEnum.LessThanOrEqual:
return Expression.LessThanOrEqual(member, constant);
case QueryableFilterCompareEnum.Equal:
default:
return Expression.Equal(member, constant);
}
}
}
The Problem / Getting Around to My Question
While the inner join on the Classroom and Organization works, I'd rather not have to pull in a second entity set for checking values that are navigatable. If I typed in a City as my filter name, normally I would do this:
classroomQuery = classroomQuery.Where(x => x.Organization.City == "Atlanta");
That doesn't really work here.
I have tried a couple of different methods in order to get me what I'm looking for:
A compiled function that would return Func<T, bool>, but when put through LINQ-to-SQL, the query did not include it.
I changed it to an Expression<Func<T, bool>>, but my return didn't return a bool in the way I attempted to implement it, so that didn't work.
I switched the way that I was implementing the navigation property, but none of my functions would read the value properly.
Basically, is there some way that I can implement the following in a way that LINQ-to-SQL from Entity Framework Core will work? Other options are welcome as well.
classroomQuery = classroomQuery.Where(x => x.Organization.BuildExpression(filter.Name, filter.Value, filter.Compare));
Edit 01:
When using the expression without the dynamic builder like so:
IQueryable<ClassroomEntity>classroomQuery = ClassroomEntity.Where(x => x.ClassroomId.HasValue).Where(x => x.Organization.City == "Atlanta").AsNoTracking();
The debug reads:
.Call Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.AsNoTracking(.Call System.Linq.Queryable.Where(
.Call System.Linq.Queryable.Where(
.Constant<Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[ClassroomEntity]>(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[ClassroomEntity]),
'(.Lambda #Lambda1<System.Func`2[ClassroomEntity,System.Boolean]>)),
'(.Lambda #Lambda2<System.Func`2[ClassroomEntity,System.Boolean]>)))
.Lambda #Lambda1<System.Func`2[ClassroomEntity,System.Boolean]>(ClassroomEntity $x)
{
($x.ClassroomId).HasValue
}
.Lambda #Lambda2<System.Func`2[ClassroomEntity,System.Boolean]>(ClassroomEntity $x)
{
($x.Organization).City == "Bronx"
}
I tried with the dynamic builder to get the Classroom teacher, which gave me a debug of:
.Lambda #Lambda3<System.Func`2[ClassroomEntity,System.Boolean]>(ClassroomEntity $var1)
{
$var1.LeadTeacherName == "Sharon Candelariatest"
}
Still cannot figure out how to get ($var1.Organization) as the entity I'm reading from.
If you can ask the client to supply the full dot notation expression for the property. eg "Organization.City";
dataField = (MemberExpression)propertyName.split(".")
.Aggregate(
(Expression)param,
(result,name) => Expression.Property(result, name));
If I am getting your problem statement, you want to be able to travel up the navigation property chain.
If that is indeed the case the real challenge is getting the navigation relationships from EF. And this is where EntityTypeExtensions comes in handy. GetNavigations() in particular.
You could recursively travel up your navigation properties and build property accessor expressions as you go:
private static IEnumerable<Tuple<IProperty, Expression>> GetPropertyAccessors(this IEntityType model, Expression param)
{
var result = new List<Tuple<IProperty, Expression>>();
result.AddRange(model.GetProperties()
.Where(p => !p.IsShadowProperty()) // this is your chance to ensure property is actually declared on the type before you attempt building Expression
.Select(p => new Tuple<IProperty, Expression>(p, Expression.Property(param, p.Name)))); // Tuple is a bit clunky but hopefully conveys the idea
foreach (var nav in model.GetNavigations().Where(p => p is Navigation))
{
var parentAccessor = Expression.Property(param, nav.Name); // define a starting point so following properties would hang off there
result.AddRange(GetPropertyAccessors(nav.ForeignKey.PrincipalEntityType, parentAccessor)); //recursively call ourselves to travel up the navigation hierarchy
}
return result;
}
then your BuildExpression method can probably be a bit simplified. Notice, I added DbContext as parameter:
public static IQueryable<T> BuildExpression<T>(this IQueryable<T> source, DbContext context, string columnName, string value, QueryableFilterCompareEnum? compare = QueryableFilterCompareEnum.Equal)
{
var param = Expression.Parameter(typeof(T));
// Get the field/column from the Entity that matches the supplied columnName value
// If the field/column does not exists on the Entity, throw an exception; There is nothing more that can be done
MemberExpression dataField;
try
{
var model = context.Model.FindEntityType(typeof(T)); // start with our own entity
var props = model.GetPropertyAccessors(param); // get all available field names including navigations
var reference = props.FirstOrDefault(p => RelationalPropertyExtensions.GetColumnName(p.Item1) == columnName); // find the filtered column - you might need to handle cases where column does not exist
dataField = reference.Item2 as MemberExpression; // we happen to already have correct property accessors in our Tuples
}
catch (ArgumentException)
{
throw new NotImplementedException("I think you shouldn't be getting these anymore");
}
ConstantExpression constant = !string.IsNullOrWhiteSpace(value)
? Expression.Constant(value.Trim(), typeof(string))
: Expression.Constant(value, typeof(string));
BinaryExpression binary = GetBinaryExpression(dataField, constant, compare);
Expression<Func<T, bool>> lambda = (Expression<Func<T, bool>>)Expression.Lambda(binary, param);
return source.Where(lambda);
}
and GetClassroomsAsync would look something like this:
public async Task<IEnumerable<IExportClassroom>> GetClassroomsAsync(IEnumerable<QueryableFilter> queryableFilters = null)
{
IQueryable<ClassroomEntity> classroomQuery = ClassroomEntity.All().AsNoTracking();
// Loop through the supplied queryable filters (if any) to construct a dynamic LINQ-to-SQL queryable
foreach (var filter in queryableFilters ?? new List<QueryableFilter>())
{
try
{
classroomQuery = classroomQuery.BuildExpression(_context, filter.Name, filter.Value, filter.Compare);
}
catch (ArgumentException ex)
{
// you probably should look at catching different exceptions now as joining is not required
}
}
query = classroomQuery.OrderBy(x => x.ClassroomId);
IEnumerable<IExportClassroom> results = await query.Select(ClassroomMapper).ToListAsync();
return results;
}
Testing it out
Since you didn't supply entity hierarchy, I experimented on one of my own:
public class Entity
{
public int Id { get; set; }
}
class Company: Entity
{
public string CompanyName { get; set; }
}
class Team: Entity
{
public string TeamName { get; set; }
public Company Company { get; set; }
}
class Employee: Entity
{
public string EmployeeName { get; set; }
public Team Team { get; set; }
}
// then i've got a test harness method as GetClassroomsAsync won't compile wothout your entities
class DynamicFilters<T> where T : Entity
{
private readonly DbContext _context;
public DynamicFilters(DbContext context)
{
_context = context;
}
public IEnumerable<T> Filter(IEnumerable<QueryableFilter> queryableFilters = null)
{
IQueryable<T> mainQuery = _context.Set<T>().AsQueryable().AsNoTracking();
// Loop through the supplied queryable filters (if any) to construct a dynamic LINQ-to-SQL queryable
foreach (var filter in queryableFilters ?? new List<QueryableFilter>())
{
mainQuery = mainQuery.BuildExpression(_context, filter.Name, filter.Value, filter.Compare);
}
mainQuery = mainQuery.OrderBy(x => x.Id);
return mainQuery.ToList();
}
}
// --- DbContext
class MyDbContext : DbContext
{
public DbSet<Company> Companies{ get; set; }
public DbSet<Team> Teams { get; set; }
public DbSet<Employee> Employees { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer("Server=.\\SQLEXPRESS;Database=test;Trusted_Connection=true");
base.OnConfiguring(optionsBuilder);
}
}
// ---
static void Main(string[] args)
{
var context = new MyDbContext();
var someTableData = new DynamicFilters<Employee>(context).Filter(new
List<QueryableFilter> {new QueryableFilter {Name = "CompanyName", Value = "Microsoft" }});
}
With the above, and a filter CompanyName = "Microsoft" EF Core 3.1 generated me the following SQL:
SELECT [e].[Id], [e].[EmployeeName], [e].[TeamId]
FROM [Employees] AS [e]
LEFT JOIN [Teams] AS [t] ON [e].[TeamId] = [t].[Id]
LEFT JOIN [Companies] AS [c] ON [t].[CompanyId] = [c].[Id]
WHERE [c].[CompanyName] = N'Microsoft'
ORDER BY [e].[Id]
This approach seems to produce desired result but has one issue: column names must be unique across all your entities. This likely can be dealt with but since I don't know much specifics of your data model I'd defer it to you.
(Disclaimer: I've written code similar to this, but I haven't actually tested the code in this answer.)
Your BuildExpression takes one query (in the form of an IQueryable<T>) and returns another query. This constrains all your filters to be applied to the property of the parameter -- x.ClassroomId -- when you actually want to apply some of them to a property of a property of the parameter -- x.Organization.City.
I would suggest a GetFilterExpression method, which produces the filter expression off of some arbitrary base expression:
private static Expression GetFilterExpression(Expression baseExpr, string columnName, string value, QueryableFilterCompareEnum? compare = QueryableFilterCompareEnum.Equal) {
MemberExpression dataField;
try {
dataField = Expression.Property(baseExpr, columnName);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
throw new ArgumentException($"Base expression type does not have a \"{propertyName}\" field.", ex.ParamName);
} else {
throw new ArgumentException(ex.Message);
}
}
if (!string.IsNullOrWhiteSpace(value)) {
value = value.Trim();
}
ConstantExpression constant = Expression.Constant(value, typeof(string));
BinaryExpression binary = GetBinaryExpression(dataField, constant, compare);
return binary;
}
Within GetClassroomsAsync, you can either build the filter expression against the original ClassroomEntity parameter, or against the returned value of the Organization property on the parameter, by passing in a different expression:
public async Task<IEnumerable<IExportClassroom>> GetClassroomsAsync(IEnumerable<QueryableFilter> queryableFilters = null) {
var filters = queryableFilters?.ToList();
var param = Expression.Parameter(typeof(ClassroomEntity));
var orgExpr = Expression.Property(param, "Organization"); // equivalent of x.Organization
IQueryable<ClassroomEntity> query = ClassroomEntity.All().AsNoTracking();
if (filters is {}) {
// Map the filters to expressions, applied to the `x` or to the `x.Organization` as appropriate
var filterExpressions = filters.Select(filter => {
try {
return GetFilterExpression(param, filter.Name, filter.Value, filter.Compare);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
return GetFilterExpression(orgExpr, filter.Name, filter.Value, filter.Compare);
} else {
throw new ArgumentException(ex.Message);
}
}
});
// LogicalCombined is shown later in the answer
query = query.Where(
Expression.Lambda<Func<ClassroomEntity, bool>>(LogicalCombined(filters))
);
}
query = query.OrderBy(x => x.ClassroomId);
IEnumerable<IExportClassroom> results = await query.Select(ClassroomMapper).ToListAsync();
return results;
}
LogicalCombined takes multiple bool-returning expressions and combines them into a single expression:
private static Expression LogicalCombined(IEnumerable<Expression> exprs, ExpressionType expressionType = ExpressionType.AndAlso) {
// ensure the expression type is a boolean operator
switch (expressionType) {
case ExpressionType.And:
case ExpressionType.AndAlso:
case ExpressionType.Or:
case ExpressionType.OrElse:
case ExpressionType.ExclusiveOr:
break;
default:
throw new ArgumentException("Invalid expression type for logically combining expressions.");
}
Expression? final = null;
foreach (var expr in exprs) {
if (final is null) {
final = expr;
continue;
}
final = Expression.MakeBinary(expressionType, final, expr);
}
return final;
}
Some suggestions:
As I've written it, GetFilterExpression is a static method. Since all the arguments (except the base expression) come from QueryableFilter, you might consider making it an instance method off of QueryableFilter.
I would also suggest changing GetBinaryExpression to use a dictionary to map from QueryableFilterCompareEnum to the built-in ExpressionType. Then, the implementation of GetBinaryExpression is just a wrapper for the built-in Expression.MakeBinary method:
private static Dictionary<QueryableFilterCompareEnum, ExpressionType> comparisonMapping = new Dictionary<QueryableFilterCompareEnum, ExpressionType> {
[QueryableFilterCompareEnum.NotEqual] = ExpressionType.NotEqual,
[QueryableFilterCompareEnum.GreaterThan] = ExpressionType.GreaterThan,
[QueryableFilterCompareEnum.GreaterThanOrEqual] = ExpressionType.GreaterThanOrEqual,
[QueryableFilterCompareEnum.LessThan] = ExpressionType.LessThan,
[QueryableFilterCompareEnum.LessThanOrEqual] = ExpressionType.LessThanOrEqual,
[QueryableFilterCompareEnum.Equal] = ExpressionType.Equal
}
private static Expression GetBinaryExpression(MemberExpression member, ConstantExpression constant, QueryableFilterCompareEnum? comparisonOperation) {
comparisonOperation = comparisonOperation ?? QueryableFilterCompareEnum.Equal;
var expressionType = comparisonMapping[comparisonOperation];
return Expression.MakeBinary(
expressionType,
member,
constant
);
}
Both GetFilterExpression and GetClassroomsAsync handle the possibility that the specified property doesn't exist on either ClassroomEntity or OrganizationEntity, by trying to construct the member-access expression and handling the thrown exception.
It might be clearer to use reflection to test if the property exists on either type or not.
More, you might consider storing a static HashSet<string> with all the valid fieldnames, and check against that.
I found an example in the VS2008 Examples for Dynamic LINQ that allows you to use a SQL-like string (e.g. OrderBy("Name, Age DESC")) for ordering. Unfortunately, the method included only works on IQueryable<T>. Is there any way to get this functionality on IEnumerable<T>?
Just stumbled into this oldie...
To do this without the dynamic LINQ library, you just need the code as below. This covers most common scenarios including nested properties.
To get it working with IEnumerable<T> you could add some wrapper methods that go via AsQueryable - but the code below is the core Expression logic needed.
public static IOrderedQueryable<T> OrderBy<T>(
this IQueryable<T> source,
string property)
{
return ApplyOrder<T>(source, property, "OrderBy");
}
public static IOrderedQueryable<T> OrderByDescending<T>(
this IQueryable<T> source,
string property)
{
return ApplyOrder<T>(source, property, "OrderByDescending");
}
public static IOrderedQueryable<T> ThenBy<T>(
this IOrderedQueryable<T> source,
string property)
{
return ApplyOrder<T>(source, property, "ThenBy");
}
public static IOrderedQueryable<T> ThenByDescending<T>(
this IOrderedQueryable<T> source,
string property)
{
return ApplyOrder<T>(source, property, "ThenByDescending");
}
static IOrderedQueryable<T> ApplyOrder<T>(
IQueryable<T> source,
string property,
string methodName)
{
string[] props = property.Split('.');
Type type = typeof(T);
ParameterExpression arg = Expression.Parameter(type, "x");
Expression expr = arg;
foreach(string prop in props) {
// use reflection (not ComponentModel) to mirror LINQ
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
Type delegateType = typeof(Func<,>).MakeGenericType(typeof(T), type);
LambdaExpression lambda = Expression.Lambda(delegateType, expr, arg);
object result = typeof(Queryable).GetMethods().Single(
method => method.Name == methodName
&& method.IsGenericMethodDefinition
&& method.GetGenericArguments().Length == 2
&& method.GetParameters().Length == 2)
.MakeGenericMethod(typeof(T), type)
.Invoke(null, new object[] {source, lambda});
return (IOrderedQueryable<T>)result;
}
Edit: it gets more fun if you want to mix that with dynamic - although note that dynamic only applies to LINQ-to-Objects (expression-trees for ORMs etc can't really represent dynamic queries - MemberExpression doesn't support it). But here's a way to do it with LINQ-to-Objects. Note that the choice of Hashtable is due to favorable locking semantics:
using Microsoft.CSharp.RuntimeBinder;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Dynamic;
using System.Linq;
using System.Runtime.CompilerServices;
static class Program
{
private static class AccessorCache
{
private static readonly Hashtable accessors = new Hashtable();
private static readonly Hashtable callSites = new Hashtable();
private static CallSite<Func<CallSite, object, object>> GetCallSiteLocked(
string name)
{
var callSite = (CallSite<Func<CallSite, object, object>>)callSites[name];
if(callSite == null)
{
callSites[name] = callSite = CallSite<Func<CallSite, object, object>>
.Create(Binder.GetMember(
CSharpBinderFlags.None,
name,
typeof(AccessorCache),
new CSharpArgumentInfo[] {
CSharpArgumentInfo.Create(
CSharpArgumentInfoFlags.None,
null)
}));
}
return callSite;
}
internal static Func<dynamic,object> GetAccessor(string name)
{
Func<dynamic, object> accessor = (Func<dynamic, object>)accessors[name];
if (accessor == null)
{
lock (accessors )
{
accessor = (Func<dynamic, object>)accessors[name];
if (accessor == null)
{
if(name.IndexOf('.') >= 0) {
string[] props = name.Split('.');
CallSite<Func<CallSite, object, object>>[] arr
= Array.ConvertAll(props, GetCallSiteLocked);
accessor = target =>
{
object val = (object)target;
for (int i = 0; i < arr.Length; i++)
{
var cs = arr[i];
val = cs.Target(cs, val);
}
return val;
};
} else {
var callSite = GetCallSiteLocked(name);
accessor = target =>
{
return callSite.Target(callSite, (object)target);
};
}
accessors[name] = accessor;
}
}
}
return accessor;
}
}
public static IOrderedEnumerable<dynamic> OrderBy(
this IEnumerable<dynamic> source,
string property)
{
return Enumerable.OrderBy<dynamic, object>(
source,
AccessorCache.GetAccessor(property),
Comparer<object>.Default);
}
public static IOrderedEnumerable<dynamic> OrderByDescending(
this IEnumerable<dynamic> source,
string property)
{
return Enumerable.OrderByDescending<dynamic, object>(
source,
AccessorCache.GetAccessor(property),
Comparer<object>.Default);
}
public static IOrderedEnumerable<dynamic> ThenBy(
this IOrderedEnumerable<dynamic> source,
string property)
{
return Enumerable.ThenBy<dynamic, object>(
source,
AccessorCache.GetAccessor(property),
Comparer<object>.Default);
}
public static IOrderedEnumerable<dynamic> ThenByDescending(
this IOrderedEnumerable<dynamic> source,
string property)
{
return Enumerable.ThenByDescending<dynamic, object>(
source,
AccessorCache.GetAccessor(property),
Comparer<object>.Default);
}
static void Main()
{
dynamic a = new ExpandoObject(),
b = new ExpandoObject(),
c = new ExpandoObject();
a.X = "abc";
b.X = "ghi";
c.X = "def";
dynamic[] data = new[] {
new { Y = a },
new { Y = b },
new { Y = c }
};
var ordered = data.OrderByDescending("Y.X").ToArray();
foreach (var obj in ordered)
{
Console.WriteLine(obj.Y.X);
}
}
}
Too easy without any complication:
Add using System.Linq.Dynamic; at the top.
Use vehicles = vehicles.AsQueryable().OrderBy("Make ASC, Year DESC").ToList();
Edit: to save some time, the System.Linq.Dynamic.Core (System.Linq.Dynamic is deprecated) assembly is not part of the framework, but can be installed from nuget: System.Linq.Dynamic.Core
Just stumbled across this question.
Using Marc's ApplyOrder implementation from above, I slapped together an Extension method that handles SQL-like strings like:
list.OrderBy("MyProperty DESC, MyOtherProperty ASC");
Details can be found here: http://aonnull.blogspot.com/2010/08/dynamic-sql-like-linq-orderby-extension.html
I guess it would work to use reflection to get whatever property you want to sort on:
IEnumerable<T> myEnumerables
var query=from enumerable in myenumerables
where some criteria
orderby GetPropertyValue(enumerable,"SomeProperty")
select enumerable
private static object GetPropertyValue(object obj, string property)
{
System.Reflection.PropertyInfo propertyInfo=obj.GetType().GetProperty(property);
return propertyInfo.GetValue(obj, null);
}
Note that using reflection is considerably slower than accessing the property directly, so the performance would have to be investigated.
Just building on what others have said. I found that the following works quite well.
public static IEnumerable<T> OrderBy<T>(this IEnumerable<T> input, string queryString)
{
if (string.IsNullOrEmpty(queryString))
return input;
int i = 0;
foreach (string propname in queryString.Split(','))
{
var subContent = propname.Split('|');
if (Convert.ToInt32(subContent[1].Trim()) == 0)
{
if (i == 0)
input = input.OrderBy(x => GetPropertyValue(x, subContent[0].Trim()));
else
input = ((IOrderedEnumerable<T>)input).ThenBy(x => GetPropertyValue(x, subContent[0].Trim()));
}
else
{
if (i == 0)
input = input.OrderByDescending(x => GetPropertyValue(x, subContent[0].Trim()));
else
input = ((IOrderedEnumerable<T>)input).ThenByDescending(x => GetPropertyValue(x, subContent[0].Trim()));
}
i++;
}
return input;
}
I was trying to do this but having problems with Kjetil Watnedal's solution because I don't use the inline linq syntax - I prefer method-style syntax. My specific problem was in trying to do dynamic sorting using a custom IComparer.
My solution ended up like this:
Given an IQueryable query like so:
List<DATA__Security__Team> teams = TeamManager.GetTeams();
var query = teams.Where(team => team.ID < 10).AsQueryable();
And given a run-time sort field argument:
string SortField; // Set at run-time to "Name"
The dynamic OrderBy looks like so:
query = query.OrderBy(item => item.GetReflectedPropertyValue(SortField));
And that's using a little helper method called GetReflectedPropertyValue():
public static string GetReflectedPropertyValue(this object subject, string field)
{
object reflectedValue = subject.GetType().GetProperty(field).GetValue(subject, null);
return reflectedValue != null ? reflectedValue.ToString() : "";
}
One last thing - I mentioned that I wanted the OrderBy to use custom IComparer - because I wanted to do Natural sorting.
To do that, I just alter the OrderBy to:
query = query.OrderBy(item => item.GetReflectedPropertyValue(SortField), new NaturalSortComparer<string>());
See this post for the code for NaturalSortComparer().
I've stumble this question looking for Linq multiple orderby clauses
and maybe this was what the author was looking for
Here's how to do that:
var query = pets.OrderBy(pet => pet.Name).ThenByDescending(pet => pet.Age);
Use dynamic linq
just add using System.Linq.Dynamic;
And use it like this to order all your columns:
string sortTypeStr = "ASC"; // or DESC
string SortColumnName = "Age"; // Your column name
query = query.OrderBy($"{SortColumnName} {sortTypeStr}");
After a lot of searching this worked for me:
public static IEnumerable<TEntity> OrderBy<TEntity>(this IEnumerable<TEntity> source,
string orderByProperty, bool desc)
{
string command = desc ? "OrderByDescending" : "OrderBy";
var type = typeof(TEntity);
var property = type.GetProperty(orderByProperty);
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var orderByExpression = Expression.Lambda(propertyAccess, parameter);
var resultExpression = Expression.Call(typeof(Queryable), command,
new[] { type, property.PropertyType },
source.AsQueryable().Expression,
Expression.Quote(orderByExpression));
return source.AsQueryable().Provider.CreateQuery<TEntity>(resultExpression);
}
First Install Dynamic
Tools --> NuGet Package Manager --> Package Manager Console
install-package System.Linq.Dynamic
Add Namespace using System.Linq.Dynamic;
Now you can use OrderBy("Name, Age DESC")
You could add it:
public static IEnumerable<T> OrderBy( this IEnumerable<T> input, string queryString) {
//parse the string into property names
//Use reflection to get and sort by properties
//something like
foreach( string propname in queryString.Split(','))
input.OrderBy( x => GetPropertyValue( x, propname ) );
// I used Kjetil Watnedal's reflection example
}
The GetPropertyValue function is from Kjetil Watnedal's answer
The issue would be why? Any such sort would throw exceptions at run-time, rather than compile time (like D2VIANT's answer).
If you're dealing with Linq to Sql and the orderby is an expression tree it will be converted into SQL for execution anyway.
Here's something else I found interesting.
If your source is a DataTable, you can use dynamic sorting without using Dynamic Linq
DataTable orders = dataSet.Tables["SalesOrderHeader"];
EnumerableRowCollection<DataRow> query = from order in orders.AsEnumerable()
orderby order.Field<DateTime>("OrderDate")
select order;
DataView view = query.AsDataView();
bindingSource1.DataSource = view;
reference: http://msdn.microsoft.com/en-us/library/bb669083.aspx (Using DataSetExtensions)
Here is one more way to do it by converting it to a DataView:
DataTable contacts = dataSet.Tables["Contact"];
DataView view = contacts.AsDataView();
view.Sort = "LastName desc, FirstName asc";
bindingSource1.DataSource = view;
dataGridView1.AutoResizeColumns();
You can convert the IEnumerable to IQueryable.
items = items.AsQueryable().OrderBy("Name ASC");
An alternate solution uses the following class/interface. It's not truly dynamic, but it works.
public interface IID
{
int ID
{
get; set;
}
}
public static class Utils
{
public static int GetID<T>(ObjectQuery<T> items) where T:EntityObject, IID
{
if (items.Count() == 0) return 1;
return items.OrderByDescending(u => u.ID).FirstOrDefault().ID + 1;
}
}
Thanks to Maarten (Query a collection using PropertyInfo object in LINQ) I got this solution:
myList.OrderByDescending(x => myPropertyInfo.GetValue(x, null)).ToList();
In my case I was working on a "ColumnHeaderMouseClick" (WindowsForm) so just found the specific Column pressed and its correspondent PropertyInfo:
foreach (PropertyInfo column in (new Process()).GetType().GetProperties())
{
if (column.Name == dgvProcessList.Columns[e.ColumnIndex].Name)
{}
}
OR
PropertyInfo column = (new Process()).GetType().GetProperties().Where(x => x.Name == dgvProcessList.Columns[e.ColumnIndex].Name).First();
(be sure to have your column Names matching the object Properties)
Cheers
You can use this:
public List<Book> Books(string orderField, bool desc, int skip, int take)
{
var propertyInfo = typeof(Book).GetProperty(orderField);
return _context.Books
.Where(...)
.OrderBy(p => !desc ? propertyInfo.GetValue(p, null) : 0)
.ThenByDescending(p => desc ? propertyInfo.GetValue(p, null) : 0)
.Skip(skip)
.Take(take)
.ToList();
}
This answer is a response to the comments that need an example for the solution provided by #John Sheehan - Runscope
Please provide an example for the rest of us.
in DAL (Data Access Layer),
The IEnumerable version:
public IEnumerable<Order> GetOrders()
{
// i use Dapper to return IEnumerable<T> using Query<T>
//.. do stuff
return orders // IEnumerable<Order>
}
The IQueryable version
public IQueryable<Order> GetOrdersAsQuerable()
{
IEnumerable<Order> qry= GetOrders();
// use the built-in extension method AsQueryable in System.Linq namespace
return qry.AsQueryable();
}
Now you can use the IQueryable version to bind, for example GridView in Asp.net and benefit for sorting (you can't sort using IEnumerable version)
I used Dapper as ORM and build IQueryable version and utilized sorting in GridView in asp.net so easy.
You can define a dictionary from string to Func<> like this :
Dictionary<string, Func<Item, object>> SortParameters = new Dictionary<string, Func<Item, object>>()
{
{"Rank", x => x.Rank}
};
And use it like this :
yourList.OrderBy(SortParameters["Rank"]);
In this case you can dynamically sort by string.
you can do it like this for multiple order by
IOrderedEnumerable<JToken> sort;
if (query.OrderBys[0].IsDESC)
{
sort = jarry.OrderByDescending(r => (string)r[query.OrderBys[0].Key]);
}
else
{
sort = jarry.OrderBy(r =>
(string) r[query.OrderBys[0].Key]);
}
foreach (var item in query.OrderBys.Skip(1))
{
if (item.IsDESC)
{
sort = sort.ThenByDescending(r => (string)r[item.Key]);
}
else
{
sort = sort.ThenBy(r => (string)r[item.Key]);
}
}
Convert List to IEnumerable or Iquerable, add using System.LINQ.Dynamic namespace, then u can mention the property names in comma seperated string to OrderBy Method which comes by default from System.LINQ.Dynamic.
I am able to do this with the code below. No need write long and complex code.
protected void sort_array(string field_name, string asc_desc)
{
objArrayList= Sort(objArrayList, field_name, asc_desc);
}
protected List<ArrayType> Sort(List<ArrayType> input, string property, string asc_desc)
{
if (asc_desc == "ASC")
{
return input.OrderBy(p => p.GetType()
.GetProperty(property)
.GetValue(p, null)).ToList();
}
else
{
return input.OrderByDescending(p => p.GetType()
.GetProperty(property)
.GetValue(p, null)).ToList();
}
}
If you are using Specification (such as Ardalis Specification)
using Microsoft.EntityFrameworkCore;
namespace TestExtensions;
public static class IQueryableExtensions
{
public static void ApplyOrder<T>(ISpecificationBuilder<T> query, string propertyName, bool ascendingOrder)
{
if (ascendingOrder)
query.OrderBy(T => EF.Property<object>(T!, propertyName));
else
query.OrderByDescending(T => EF.Property<object>(T!, propertyName));
}
}
With Net6 and EF
.AsQueryable().OrderBy((ColumnOrder.Column, ColumnOrder.Dir));
var result1 = lst.OrderBy(a=>a.Name);// for ascending order.
var result1 = lst.OrderByDescending(a=>a.Name);// for desc order.
This question is related to this other question.
I have the following method:
public static T GetNewData<T>(params Action<dynamic>[] actions) where T : class, new()
{
dynamic dynamicData = new DeepObject();
foreach (var action in actions)
{
action(dynamicData);
}
return Converter.Convert<T>(dynamicData);
}
The users of this method will include less technical people, even non-developers and as such the easier writing calls to this method is the better. My sticking point right now is that by using Action<dynamic> as the parameter type there is no intellisense provided to the user. In the context I know that the intellisense should be acting as if the dynamic was in fact T.
So is their a way I could either: Tell Visual Studio to use type T for the intellisense or change the parameter to be Action<T> and somehow programmatically change it to be Action<dynamic> or Action<DeepObject> so that the call to it will succeed?
EDIT: To clarify, the types that I am using for T are not of type DeepObject and they do not inherit any standard interface, the use of DeepObject is to allow setting up nested types without the user needing to explicitly instantiate at each level. This was the original usage before adding the dynamic and DeepObject code:
ExampleDataFactory.GetNewData<ServicesAndFeaturesInfo>(
x => x.Property1 = ExampleDataFactory.GetNewData<Property1Type>(),
x => x.Property1.Property2 = ExampleDataFactory.GetNewData<Property2Type>(),
x => x.Property1.Property2.Property3 = ExampleDataFactory.GetNewData<Property3Type>(),
x => x.Property1.Property2.Property3.Property4 = true);
Here is what it looks like now:
ExampleDataFactory.GetNewData<ServicesAndFeaturesInfo>(
x => x.Property1.Property2.Property3.Property4 = true);
EDIT: Here is the fully implemented solution based on nmclean's answer
public static DataBuilder<T> GetNewData<T>() where T : class, new()
{
return new DataBuilder<T>();
}
The DataBuilder Class:
public class DataBuilder<T>
{
public readonly T data;
public DataBuilder()
{
data = Activator.CreateInstance<T>();
}
public DataBuilder(T data)
{
this.data = data;
}
public DataBuilder<T> SetValue<T2>(Expression<Func<T, T2>> expression, T2 value)
{
var mExpr = GetMemberExpression(expression);
var obj = Recurse(mExpr);
var p = (PropertyInfo)mExpr.Member;
p.SetValue(obj, value);
return this;
}
public T Build()
{
return data;
}
public object Recurse(MemberExpression expr)
{
if (expr.Expression.Type != typeof(T))
{
var pExpr = GetMemberExpression(expr.Expression);
var parent = Recurse(pExpr);
var pInfo = (PropertyInfo) pExpr.Member;
var obj = pInfo.GetValue(parent);
if (obj == null)
{
obj = Activator.CreateInstance(pInfo.PropertyType);
pInfo.SetValue(parent, obj);
}
return obj;
}
return data;
}
private static MemberExpression GetMemberExpression(Expression expr)
{
var member = expr as MemberExpression;
var unary = expr as UnaryExpression;
return member ?? (unary != null ? unary.Operand as MemberExpression : null);
}
private static MemberExpression GetMemberExpression<T2>(Expression<Func<T, T2>> expr)
{
return GetMemberExpression(expr.Body);
}
}
The Usage:
ExampleDataFactory.GetNewData<ServicesAndFeaturesInfo>()
.SetValue(x=> x.Property1.EnumProperty, EnumType.Own)
.SetValue(x=> x.Property2.Property3.Property4.BoolProperty, true)
.Build();
Do not use Action<dynamic>, use Action<T>with method's constraint where T:DeepObject. Users will get intellisence and ability to use strongly typed objects:
public static DeepObject GetNewData<T>(params Action<T>[] actions)
where T : DeepObject, //restrict user only inheritors of DeepObject
new() //and require constructor
{
var data = new T();
foreach (var action in actions)
{
action(data);
}
return data;
}
Does the user need to access unknown properties or add new ones? If not, using dynamic objects seems like a step backwards. If your desired syntax does compile as an Action<T>, I think you should just declare it that way and then go with your first instinct of using the LINQ Expression API to decide how to interpret the code.
Unfortunately, although statements, such as an assignment, are part of the API, C# doesn't support converting them to expression trees. This is not allowed:
public static T GetNewData<T>(params Expression<Action<T>>[] actions)
where T : class, new() {
...
}
...
ExampleDataFactory.GetNewData<ServicesAndFeaturesInfo>(
x => x.Property1.Property2.Property3.Property4 = true);
Only single-line expressions that would have a return a value are supported. So I think the best you could do is something like this:
public class Assignment<T> {
public readonly Expression Expression;
public readonly object Value;
public Assignment(Expression<Func<T, object>> expression, object value) {
Expression = expression;
Value = value;
}
}
...
public static T GetNewData<T>(params Assignment<T>[] assignments)
where T : class, new() {
var data = Activator.CreateInstance<T>();
foreach (var assignment in assignments) {
// todo:
// - pull property names from assignment.Expression
// - initialize nested properties / assign to assignment.Value
}
return data;
}
...
ExampleDataFactory.GetNewData<ServicesAndFeaturesInfo>(
new Assignment<ServicesAndFeaturesInfo>(
x => x.Property1.Property2.Property3.Property4, true));
Getting the property names from an expression tree of chained property access is not too complicated. Here is one implementation.
Of course, the new Assignment<ServicesAndFeaturesInfo>(...) is ugly and repetitive, so maybe it could be restructured to something like this:
var newData = ExampleDataFactory.NewData<ServicesAndFeaturesInfo>();
newData.Add(x => x.Property1.Property2.Property3.Property4, true);
newData.Add(...);
...
newData.Get();
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
LINQ To SQL exception: Local sequence cannot be used in LINQ to SQL implementation of query operators except the Contains operator
I am trying the following query:
var data = (from bk in DataContext.Book
where ((searchArray.Count() == 0 || searchArray.ToList().Any(x => bk.Name.Contains(x))) ||
(searchArray.Count() == 0 || searchArray.ToList().Any(x => bk.Genre.Contains(x)))))
where searchArray is a Array containing the individual words that I want to search for, I split the string the user entered and put the results in this array. Whenever I try to run this I get the following error:
"Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator."
Can anyone tell me what I am doing wrong and what is the correct way to perform this search?
In a nutshell, I am trying to allow a user to enter a string like "Hello World" and for a query to be generated that will look for either hello or world or both. But, a user can enter any number of words.
The simplest option is probably to build the lambda expression by hand:
static class ContainsAny
{
private static readonly MethodInfo StringContains
= typeof(string).GetMethod("Contains", new[] { typeof(string) });
public static Builder<T> Words<T>(IEnumerable<string> words)
{
return new Builder<T>(words);
}
public static Builder<T> Words<T>(params string[] words)
{
return new Builder<T>(words);
}
public sealed class Builder<T>
{
private static readonly ParameterExpression Parameter
= Expression.Parameter(typeof(T), "obj");
private readonly List<Expression> _properties = new List<Expression>();
private readonly List<ConstantExpression> _words;
internal Builder(IEnumerable<string> words)
{
_words = words
.Where(word => !string.IsNullOrEmpty(word))
.Select(word => Expression.Constant(word))
.ToList();
}
public Builder<T> WithProperty(Expression<Func<T, string>> property)
{
if (_words.Count != 0)
{
_properties.Add(ReplacementVisitor.Transform(
property, property.Parameters[0], Parameter));
}
return this;
}
private Expression BuildProperty(Expression prop)
{
return _words
.Select(w => (Expression)Expression.Call(prop, StringContains, w))
.Aggregate(Expression.OrElse);
}
public Expression<Func<T, bool>> Build()
{
if (_words.Count == 0) return (T obj) => true;
var body = _properties
.Select(BuildProperty)
.Aggregate(Expression.OrElse);
return Expression.Lambda<Func<T, bool>>(body, Parameter);
}
}
private sealed class ReplacementVisitor : ExpressionVisitor
{
private ICollection<ParameterExpression> Parameters { get; set; }
private Expression Find { get; set; }
private Expression Replace { get; set; }
public static Expression Transform(
LambdaExpression source,
Expression find,
Expression replace)
{
var visitor = new ReplacementVisitor
{
Parameters = source.Parameters,
Find = find,
Replace = replace,
};
return visitor.Visit(source.Body);
}
private Expression ReplaceNode(Expression node)
{
return (node == Find) ? Replace : node;
}
protected override Expression VisitConstant(ConstantExpression node)
{
return ReplaceNode(node);
}
protected override Expression VisitBinary(BinaryExpression node)
{
var result = ReplaceNode(node);
if (result == node) result = base.VisitBinary(node);
return result;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (Parameters.Contains(node)) return ReplaceNode(node);
return Parameters.FirstOrDefault(p => p.Name == node.Name) ?? node;
}
}
}
With this code in place, you can call:
Expression<Func<Book, bool>> filter = ContainsAny
.Words<Book>(searchArray)
.WithProperty(book => book.Name)
.WithProperty(book => book.Genre)
.Build();
var data = DataContext.Book.Where(filter);
For example, if the searchArray contains { "Hello", "World" }, the generated lambda will be:
obj => (obj.Name.Contains("Hello") || obj.Name.Contains("World"))
|| (obj.Genre.Contains("Hello") || obj.Genre.Contains("World")))
If I understand what you're trying to do correctly, you should be able to condense your query down to:
from bk in DataContext.Book
where searchArray.Contains(bk.Name) || searchArray.Contains(bk.Genre)
select bk
This is basically equivalent to the SQL:
select bk.*
from Book bk
where bk.Name in (...) or bk.Genre in (...)
In your case you must combine interpreted and local queries which can hurt performance or use SQL CLR integration by creating CLR function on database.
I have a db table that stores the following:
RuleID objectProperty ComparisonOperator TargetValue
1 age 'greater_than' 15
2 username 'equal' 'some_name'
3 tags 'hasAtLeastOne' 'some_tag some_tag2'
Now say I have a collection of these rules:
List<Rule> rules = db.GetRules();
Now I have an instance of a user also:
User user = db.GetUser(....);
How would I loop through these rules, and apply the logic and perform the comparisons etc?
if(user.age > 15)
if(user.username == "some_name")
Since the object's property like 'age' or 'user_name' is stored in the table, along with the comparison operater 'great_than' and 'equal', how could I possible do this?
C# is a statically typed language, so not sure how to go forward.
This snippet compiles the Rules into fast executable code (using Expression trees) and does not need any complicated switch statements:
(Edit : full working example with generic method)
public Func<User, bool> CompileRule(Rule r)
{
var paramUser = Expression.Parameter(typeof(User));
Expression expr = BuildExpr(r, paramUser);
// build a lambda function User->bool and compile it
return Expression.Lambda<Func<User, bool>>(expr, paramUser).Compile();
}
You can then write:
List<Rule> rules = new List<Rule> {
new Rule ("Age", "GreaterThan", "21"),
new Rule ( "Name", "Equal", "John"),
new Rule ( "Tags", "Contains", "C#" )
};
// compile the rules once
var compiledRules = rules.Select(r => CompileRule(r)).ToList();
public bool MatchesAllRules(User user)
{
return compiledRules.All(rule => rule(user));
}
Here is the implementation of BuildExpr:
Expression BuildExpr(Rule r, ParameterExpression param)
{
var left = MemberExpression.Property(param, r.MemberName);
var tProp = typeof(User).GetProperty(r.MemberName).PropertyType;
ExpressionType tBinary;
// is the operator a known .NET operator?
if (ExpressionType.TryParse(r.Operator, out tBinary)) {
var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tProp));
// use a binary operation, e.g. 'Equal' -> 'u.Age == 21'
return Expression.MakeBinary(tBinary, left, right);
} else {
var method = tProp.GetMethod(r.Operator);
var tParam = method.GetParameters()[0].ParameterType;
var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tParam));
// use a method call, e.g. 'Contains' -> 'u.Tags.Contains(some_tag)'
return Expression.Call(left, method, right);
}
}
Note that I used 'GreaterThan' instead of 'greater_than' etc. - this is because 'GreaterThan' is the .NET name for the operator, therefore we don't need any extra mapping.
If you need custom names you can build a very simple dictionary and just translate all operators before compiling the rules:
var nameMap = new Dictionary<string, string> {
{ "greater_than", "GreaterThan" },
{ "hasAtLeastOne", "Contains" }
};
The code uses the type User for simplicity. You can replace User with a generic type T to have a generic Rule compiler for any types of objects. Also, the code should handle errors, like unknown operator name.
Note that generating code on the fly was possible even before the Expression trees API was introduced, using Reflection.Emit. The method LambdaExpression.Compile() uses Reflection.Emit under the covers (you can see this using ILSpy).
Here is some code that compiles as is and does the job.
Basically use two dictionaries, one containing a mapping from operator names to boolean functions, and another containing a map from the property names of the User type to PropertyInfos used to invoke the property getter (if public).
You pass the User instance, and the three values from your table to the static Apply method.
class User
{
public int Age { get; set; }
public string UserName { get; set; }
}
class Operator
{
private static Dictionary<string, Func<object, object, bool>> s_operators;
private static Dictionary<string, PropertyInfo> s_properties;
static Operator()
{
s_operators = new Dictionary<string, Func<object, object, bool>>();
s_operators["greater_than"] = new Func<object, object, bool>(s_opGreaterThan);
s_operators["equal"] = new Func<object, object, bool>(s_opEqual);
s_properties = typeof(User).GetProperties().ToDictionary(propInfo => propInfo.Name);
}
public static bool Apply(User user, string op, string prop, object target)
{
return s_operators[op](GetPropValue(user, prop), target);
}
private static object GetPropValue(User user, string prop)
{
PropertyInfo propInfo = s_properties[prop];
return propInfo.GetGetMethod(false).Invoke(user, null);
}
#region Operators
static bool s_opGreaterThan(object o1, object o2)
{
if (o1 == null || o2 == null || o1.GetType() != o2.GetType() || !(o1 is IComparable))
return false;
return (o1 as IComparable).CompareTo(o2) > 0;
}
static bool s_opEqual(object o1, object o2)
{
return o1 == o2;
}
//etc.
#endregion
public static void Main(string[] args)
{
User user = new User() { Age = 16, UserName = "John" };
Console.WriteLine(Operator.Apply(user, "greater_than", "Age", 15));
Console.WriteLine(Operator.Apply(user, "greater_than", "Age", 17));
Console.WriteLine(Operator.Apply(user, "equal", "UserName", "John"));
Console.WriteLine(Operator.Apply(user, "equal", "UserName", "Bob"));
}
}
I built a rule engine that takes a different approach than you outlined in your question, but I think you will find it to be much more flexible than your current approach.
Your current approach seems to be focused on a single entity, "User", and your persistent rules identify "propertyname", "operator" and "value". My pattern, instead stores the C# code for a predicate (Func<T, bool>) in an "Expression" column in my database. In the current design, using code generation I am querying the "rules" from my database and compiling an assembly with "Rule" types, each with a "Test" method. Here is the signature for the interface that is implemented each Rule:
public interface IDataRule<TEntity>
{
/// <summary>
/// Evaluates the validity of a rule given an instance of an entity
/// </summary>
/// <param name="entity">Entity to evaluate</param>
/// <returns>result of the evaluation</returns>
bool Test(TEntity entity);
/// <summary>
/// The unique indentifier for a rule.
/// </summary>
int RuleId { get; set; }
/// <summary>
/// Common name of the rule, not unique
/// </summary>
string RuleName { get; set; }
/// <summary>
/// Indicates the message used to notify the user if the rule fails
/// </summary>
string ValidationMessage { get; set; }
/// <summary>
/// indicator of whether the rule is enabled or not
/// </summary>
bool IsEnabled { get; set; }
/// <summary>
/// Represents the order in which a rule should be executed relative to other rules
/// </summary>
int SortOrder { get; set; }
}
The "Expression" is compiled as the body of the "Test" method when the application first executes. As you can see the other columns in the table are also surfaced as first-class properties on the rule so that a developer has flexibility to create an experience for how the user gets notified of failure or success.
Generating an in-memory assembly is a 1-time occurrence during your application and you get a performance gain by not having to use reflection when evaluating your rules. Your expressions are checked at runtime as the assembly will not generate correctly if a property name is misspelled, etc.
The mechanics of creating an in-memory assembly are as follows:
Load your rules from the DB
iterate over the rules and for-each, using a StringBuilder and some string concatenation write the Text representing a class that inherits from IDataRule
compile using CodeDOM -- more info
This is actually quite simple because for the majority this code is property implementations and value initialization in the constructor. Besides that, the only other code is the Expression.
NOTE: there is a limitation that your expression must be .NET 2.0 (no lambdas or other C# 3.0 features) due to a limitation in CodeDOM.
Here is some sample code for that.
sb.AppendLine(string.Format("\tpublic class {0} : SomeCompany.ComponentModel.IDataRule<{1}>", className, typeName));
sb.AppendLine("\t{");
sb.AppendLine("\t\tprivate int _ruleId = -1;");
sb.AppendLine("\t\tprivate string _ruleName = \"\";");
sb.AppendLine("\t\tprivate string _ruleType = \"\";");
sb.AppendLine("\t\tprivate string _validationMessage = \"\";");
/// ...
sb.AppendLine("\t\tprivate bool _isenabled= false;");
// constructor
sb.AppendLine(string.Format("\t\tpublic {0}()", className));
sb.AppendLine("\t\t{");
sb.AppendLine(string.Format("\t\t\tRuleId = {0};", ruleId));
sb.AppendLine(string.Format("\t\t\tRuleName = \"{0}\";", ruleName.TrimEnd()));
sb.AppendLine(string.Format("\t\t\tRuleType = \"{0}\";", ruleType.TrimEnd()));
sb.AppendLine(string.Format("\t\t\tValidationMessage = \"{0}\";", validationMessage.TrimEnd()));
// ...
sb.AppendLine(string.Format("\t\t\tSortOrder = {0};", sortOrder));
sb.AppendLine("\t\t}");
// properties
sb.AppendLine("\t\tpublic int RuleId { get { return _ruleId; } set { _ruleId = value; } }");
sb.AppendLine("\t\tpublic string RuleName { get { return _ruleName; } set { _ruleName = value; } }");
sb.AppendLine("\t\tpublic string RuleType { get { return _ruleType; } set { _ruleType = value; } }");
/// ... more properties -- omitted
sb.AppendLine(string.Format("\t\tpublic bool Test({0} entity) ", typeName));
sb.AppendLine("\t\t{");
// #############################################################
// NOTE: This is where the expression from the DB Column becomes
// the body of the Test Method, such as: return "entity.Prop1 < 5"
// #############################################################
sb.AppendLine(string.Format("\t\t\treturn {0};", expressionText.TrimEnd()));
sb.AppendLine("\t\t}"); // close method
sb.AppendLine("\t}"); // close Class
Beyond this I did make a class I called "DataRuleCollection", which implemented ICollection>. This enabled me to create a "TestAll" capability and an indexer for executing a specific rule by name. Here are the implementations for those two methods.
/// <summary>
/// Indexer which enables accessing rules in the collection by name
/// </summary>
/// <param name="ruleName">a rule name</param>
/// <returns>an instance of a data rule or null if the rule was not found.</returns>
public IDataRule<TEntity, bool> this[string ruleName]
{
get { return Contains(ruleName) ? list[ruleName] : null; }
}
// in this case the implementation of the Rules Collection is:
// DataRulesCollection<IDataRule<User>> and that generic flows through to the rule.
// there are also some supporting concepts here not otherwise outlined, such as a "FailedRules" IList
public bool TestAllRules(User target)
{
rules.FailedRules.Clear();
var result = true;
foreach (var rule in rules.Where(x => x.IsEnabled))
{
result = rule.Test(target);
if (!result)
{
rules.FailedRules.Add(rule);
}
}
return (rules.FailedRules.Count == 0);
}
MORE CODE: There was a request for the code related to the Code Generation. I encapsulated the functionality in a class called 'RulesAssemblyGenerator' which I have included below.
namespace Xxx.Services.Utils
{
public static class RulesAssemblyGenerator
{
static List<string> EntityTypesLoaded = new List<string>();
public static void Execute(string typeName, string scriptCode)
{
if (EntityTypesLoaded.Contains(typeName)) { return; }
// only allow the assembly to load once per entityType per execution session
Compile(new CSharpCodeProvider(), scriptCode);
EntityTypesLoaded.Add(typeName);
}
private static void Compile(CodeDom.CodeDomProvider provider, string source)
{
var param = new CodeDom.CompilerParameters()
{
GenerateExecutable = false,
IncludeDebugInformation = false,
GenerateInMemory = true
};
var path = System.Reflection.Assembly.GetExecutingAssembly().Location;
var root_Dir = System.IO.Path.Combine(System.AppDomain.CurrentDomain.BaseDirectory, "Bin");
param.ReferencedAssemblies.Add(path);
// Note: This dependencies list are included as assembly reference and they should list out all dependencies
// That you may reference in your Rules or that your entity depends on.
// some assembly names were changed... clearly.
var dependencies = new string[] { "yyyyyy.dll", "xxxxxx.dll", "NHibernate.dll", "ABC.Helper.Rules.dll" };
foreach (var dependency in dependencies)
{
var assemblypath = System.IO.Path.Combine(root_Dir, dependency);
param.ReferencedAssemblies.Add(assemblypath);
}
// reference .NET basics for C# 2.0 and C#3.0
param.ReferencedAssemblies.Add(#"C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.dll");
param.ReferencedAssemblies.Add(#"C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.5\System.Core.dll");
var compileResults = provider.CompileAssemblyFromSource(param, source);
var output = compileResults.Output;
if (compileResults.Errors.Count != 0)
{
CodeDom.CompilerErrorCollection es = compileResults.Errors;
var edList = new List<DataRuleLoadExceptionDetails>();
foreach (CodeDom.CompilerError s in es)
edList.Add(new DataRuleLoadExceptionDetails() { Message = s.ErrorText, LineNumber = s.Line });
var rde = new RuleDefinitionException(source, edList.ToArray());
throw rde;
}
}
}
}
If there are any other questions or comments or requests for further code samples, let me know.
Reflection is your most versatile answer. You have three columns of data, and they need to be treated in different ways:
Your field name. Reflection is the way to get the value from a coded field name.
Your comparison operator. There should be a limited number of these, so a case statement should handle them most easily. Especially as some of them ( has one or more of ) is slightly more complex.
Your comparison value. If these are all straight values then this is easy, although you will have divide the multiple entries up. However, you could also use reflection if they are field names too.
I would take an approach more like:
var value = user.GetType().GetProperty("age").GetValue(user, null);
//Thank you Rick! Saves me remembering it;
switch(rule.ComparisonOperator)
case "equals":
return EqualComparison(value, rule.CompareTo)
case "is_one_or_more_of"
return IsInComparison(value, rule.CompareTo)
etc. etc.
It gives you flexibility for adding more options for comparison. It also means that you can code within the Comparison methods any type validation that you might want, and make them as complex as you want. There is also the option here for the CompareTo to be evaluated as a recursive call back to another line, or as a field value, which could be done like:
return IsInComparison(value, EvaluateComparison(rule.CompareTo))
It all depends on the possibilities for the future....
If you only have a handful of properties and operators, the path of least of resistance is to just code up all the checks as special cases like this:
public bool ApplyRules(List<Rule> rules, User user)
{
foreach (var rule in rules)
{
IComparable value = null;
object limit = null;
if (rule.objectProperty == "age")
{
value = user.age;
limit = Convert.ToInt32(rule.TargetValue);
}
else if (rule.objectProperty == "username")
{
value = user.username;
limit = rule.TargetValue;
}
else
throw new InvalidOperationException("invalid property");
int result = value.CompareTo(limit);
if (rule.ComparisonOperator == "equal")
{
if (!(result == 0)) return false;
}
else if (rule.ComparisonOperator == "greater_than")
{
if (!(result > 0)) return false;
}
else
throw new InvalidOperationException("invalid operator");
}
return true;
}
If you have a lot of properties, you may find a table-driven approach more palatable. In that case you would create a static Dictionary that maps property names to delegates matching, say, Func<User, object>.
If you don't know the names of the properties at compile time, or you want to avoid special-cases for each property and don't want to use the table approach, you can use reflection to get properties. For example:
var value = user.GetType().GetProperty("age").GetValue(user, null);
But since TargetValue is probably a string, you'll need to take care to do type conversion from the rules table if necessary.
What about a data type orientated approach with an extention method:
public static class RoleExtension
{
public static bool Match(this Role role, object obj )
{
var property = obj.GetType().GetProperty(role.objectProperty);
if (property.PropertyType == typeof(int))
{
return ApplyIntOperation(role, (int)property.GetValue(obj, null));
}
if (property.PropertyType == typeof(string))
{
return ApplyStringOperation(role, (string)property.GetValue(obj, null));
}
if (property.PropertyType.GetInterface("IEnumerable<string>",false) != null)
{
return ApplyListOperation(role, (IEnumerable<string>)property.GetValue(obj, null));
}
throw new InvalidOperationException("Unknown PropertyType");
}
private static bool ApplyIntOperation(Role role, int value)
{
var targetValue = Convert.ToInt32(role.TargetValue);
switch (role.ComparisonOperator)
{
case "greater_than":
return value > targetValue;
case "equal":
return value == targetValue;
//...
default:
throw new InvalidOperationException("Unknown ComparisonOperator");
}
}
private static bool ApplyStringOperation(Role role, string value)
{
//...
throw new InvalidOperationException("Unknown ComparisonOperator");
}
private static bool ApplyListOperation(Role role, IEnumerable<string> value)
{
var targetValues = role.TargetValue.Split(' ');
switch (role.ComparisonOperator)
{
case "hasAtLeastOne":
return value.Any(v => targetValues.Contains(v));
//...
}
throw new InvalidOperationException("Unknown ComparisonOperator");
}
}
Than you can evaulate like this:
var myResults = users.Where(u => roles.All(r => r.Match(u)));
Although the most obvious way to answer the "How to implement a rule engine? (in C#)" question is to execute a given set of rules in sequence, this is in general considered as a naïve implementation (does not mean it does not work :-)
It seems it's "good enough" in your case because your problem seems more to be "how to run a set of rules in sequence", and the lambda/expression tree (Martin's answer) is certainly the most elegant way in that matter if you are equiped with recent C# versions.
However for more advanced scenarios, here is a link to the Rete Algorithm that is in fact implemented in many commercial rule engine systems, and another link to NRuler, an implementation of that algorithm in C#.
Martin's answer was quite good. I actually made a rules engine that has the same idea as his. And I was surprised that it's almost the same. I've included some of his code to somewhat improve it. Although I've made it to handle more complex rules.
You can look at Yare.NET
Or download it in Nuget
How about using the workflow rules engine?
You can execute Windows Workflow Rules without Workflow
see Guy Burstein's Blog: http://blogs.microsoft.co.il/blogs/bursteg/archive/2006/10/11/RuleExecutionWithoutWorkflow.aspx
and to programatically create your rules, see Stephen Kaufman's WebLog
http://blogs.msdn.com/b/skaufman/archive/2006/05/15/programmatically-create-windows-workflow-rules.aspx
I added implementation for and,or between rules
i added class RuleExpression that represent the root of a tree that can be leaf the is simple rule or can be and,or binary expressions there for they dont have rule and have expressions:
public class RuleExpression
{
public NodeOperator NodeOperator { get; set; }
public List<RuleExpression> Expressions { get; set; }
public Rule Rule { get; set; }
public RuleExpression()
{
}
public RuleExpression(Rule rule)
{
NodeOperator = NodeOperator.Leaf;
Rule = rule;
}
public RuleExpression(NodeOperator nodeOperator, List<RuleExpression> expressions, Rule rule)
{
this.NodeOperator = nodeOperator;
this.Expressions = expressions;
this.Rule = rule;
}
}
public enum NodeOperator
{
And,
Or,
Leaf
}
I have another class that compile the ruleExpression to one Func<T, bool>:
public static Func<T, bool> CompileRuleExpression<T>(RuleExpression ruleExpression)
{
//Input parameter
var genericType = Expression.Parameter(typeof(T));
var binaryExpression = RuleExpressionToOneExpression<T>(ruleExpression, genericType);
var lambdaFunc = Expression.Lambda<Func<T, bool>>(binaryExpression, genericType);
return lambdaFunc.Compile();
}
private static Expression RuleExpressionToOneExpression<T>(RuleExpression ruleExpression, ParameterExpression genericType)
{
if (ruleExpression == null)
{
throw new ArgumentNullException();
}
Expression finalExpression;
//check if node is leaf
if (ruleExpression.NodeOperator == NodeOperator.Leaf)
{
return RuleToExpression<T>(ruleExpression.Rule, genericType);
}
//check if node is NodeOperator.And
if (ruleExpression.NodeOperator.Equals(NodeOperator.And))
{
finalExpression = Expression.Constant(true);
ruleExpression.Expressions.ForEach(expression =>
{
finalExpression = Expression.AndAlso(finalExpression, expression.NodeOperator.Equals(NodeOperator.Leaf) ?
RuleToExpression<T>(expression.Rule, genericType) :
RuleExpressionToOneExpression<T>(expression, genericType));
});
return finalExpression;
}
//check if node is NodeOperator.Or
else
{
finalExpression = Expression.Constant(false);
ruleExpression.Expressions.ForEach(expression =>
{
finalExpression = Expression.Or(finalExpression, expression.NodeOperator.Equals(NodeOperator.Leaf) ?
RuleToExpression<T>(expression.Rule, genericType) :
RuleExpressionToOneExpression<T>(expression, genericType));
});
return finalExpression;
}
}
public static BinaryExpression RuleToExpression<T>(Rule rule, ParameterExpression genericType)
{
try
{
Expression value = null;
//Get Comparison property
var key = Expression.Property(genericType, rule.ComparisonPredicate);
Type propertyType = typeof(T).GetProperty(rule.ComparisonPredicate).PropertyType;
//convert case is it DateTimeOffset property
if (propertyType == typeof(DateTimeOffset))
{
var converter = TypeDescriptor.GetConverter(propertyType);
value = Expression.Constant((DateTimeOffset)converter.ConvertFromString(rule.ComparisonValue));
}
else
{
value = Expression.Constant(Convert.ChangeType(rule.ComparisonValue, propertyType));
}
BinaryExpression binaryExpression = Expression.MakeBinary(rule.ComparisonOperator, key, value);
return binaryExpression;
}
catch (FormatException)
{
throw new Exception("Exception in RuleToExpression trying to convert rule Comparison Value");
}
catch (Exception e)
{
throw new Exception(e.Message);
}
}
I have created a package for a rich and high performance rule engine written in dotnet, check out this repo for more info.
Once installed, you can use it as simple as:
var engine = new RulesService<TestModel>(new RulesCompiler(), new LazyCache.Mocks.MockCachingService());
var matchingRules = engine.GetMatchingRules(
new TestModel { NumericField = 5 },
new[] {
new RulesConfig {
Id = Guid.NewGuid(),
RulesOperator = Rule.InterRuleOperatorType.And,
RulesGroups = new RulesGroup[] {
new RulesGroup {
RulesOperator = Rule.InterRuleOperatorType.And,
Rules = new[] {
new Rule {
ComparisonOperator = Rule.ComparisonOperatorType.Equal,
ComparisonValue = 5.ToString(),
ComparisonPredicate = nameof(TestModel.NumericField)
}
}
}
}
}
});
I want to add Microsoft's Rules Engine library for future visits;
https://github.com/microsoft/RulesEngine