I am doing a query using linq to entities and I want to reuse the same code but with a slightly different select statement. This is the query:
return from f in GetFields<T>()
where f.Id == Id
select new Prop<V>
{
Name = f.Name,
Value = toValue(f.value)
};
where toValue is a Func that creates a new Value object. I want to use several toValue function which have this kind of body:
var toValue = f => new DerivedValueClass {
FirstField = f.FirstField,
SecondField = f.SecondField
}
So it is just simple assignment, which will be easily translated by sql to entities. However when I execute the code, linq to entities states:
The LINQ expression node type 'Invoke' is not supported in LINQ to Entities
I guess it is not possible to call the toValue function, since it can't be translated. How can I create an expression from the function toValue in order to use it inside the linq query?
Eventually I find out that Linqkit allows to do this kind of things very easily:
Expression<Func<T, V>> toValue = a => new DerivedObject() {
// perform assignment here
}
return from f in GetFields<T>().AsExpandable() // <- LinqKit AsExpandable() extension method
where f.Id == Id
select new Prop<V>
{
Name = f.Name,
Value = toValue.Invoke(f.value) // <- LinqKit Invoke() extension method
};
It is really important that toValue is an Expression, since the compiler will create an ExpressionTree instead of simply generating a lambda expression. Linqkit use some trick in order to replace the toValue invocation with an expression tree. More info on the LinqKit site and here
Related
I am trying to refactor the following line of an Entity Framework query into a generalized static extension method:
dbContext.Employees
.Where(e => permissionResolver.AuthorizedUsers.Select(p => p.Id).Contains(e.Id))
.OrderBy(...)
PermissionResolver is just an instance I receive a list of IDs from to compare against a user ID stored in the current record. It compiles perfectly to a SQL statement WHERE Id IN (....).
Now I am trying to create an extension method for IQueryable<T> that I can use for any type of record, I just want to pass in a property where the owner's ID is stored in.
So that is what I came up with:
public static IQueryable<T> AuthorizedRecords<T>(this IQueryable<T> query, Expression<Func<T, Int32>> property, IPermissionResolver permissionResolver)
{
Expression<Func<T, Boolean>> idIsAuthorized = entity => permissionResolver.AuthorizedUsers.Select(e => e.Id).ToList().Contains(property.Compile()(entity));
return query.Where(idIsAuthorized);
}
I'm getting a runtime error that this expression cannot be translated into SQL.
How can I combine the property expression to the main query expression that it can be translated into SQL correctly? Is there a better way to rewrite the query expression?
property.Compile() converts the expression tree into a delegate, this delegate cannot be properly translated back to an expression tree/SQL.
You need to construct expression tree like that:
var ids = permissionResolver.AuthorizedUsers.Select(e => e.Id).ToList().AsEnumerable();
// method Enumerable.Contains<int>()
var methodContains = typeof(System.Linq.Enumerable).GetMethods()
.Where(m => m.Name == "Contains" && m.GetParameters().Length == 2)
.First()
.MakeGenericMethod(typeof(int));
var lambdaParam = property.Parameters.Single();
var lambda = Expression.Lambda(
Expression.Call(
methodContains,
Expression.Constant(ids),
property.Body),
lambdaParam
);
var predicate = (Expression<Func<T, bool>>)lambda;
return query.Where(predicate);
The short answer is you cannot use custom extension methods in entity framework queries. Under the hood entity framework parses expression tree Expression<Func<>> into sql query. It seems kind of impossible to translate any possible extension method to sql so they support limited set of Linq methods to properly translate it. Some useful information about expression trees: https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/expression-trees.
Some refactoring you can do to simplify your query is to use Any method like this:
dbContext.Employees
.Where(e => permissionResolver.AuthorizedUsers.Any(u => u.Id == e.Id))
.OrderBy(...
I have a large select expression to reuse in several classes. For the DRY principle I have chosen to create a property that returns the Expression to the caller code
protected virtual Expression<Func<SezioneJoin, QueryRow>> Select
{
get
{
return sj => new QueryRow
{
A01 = sj.A.A01,
A01a = sj.A.A01a,
A01b = sj.A.A01b,
A02 = sj.A.A02,
A03 = sj.A.A03,
A11 = sj.A.A11,
A12 = sj.A.A12,
A12a = sj.A.A12a,
A12b = sj.A.A12b,
A12c = sj.A.A12c,
A21 = sj.A.A21,
A22 = sj.A.A22,
..............
Lots of assignements
};
}
}
Now I can successfully use the property if I do
var query = dataContext.entity.Join(...).Where(x => ...).Select(Select);
But the following will not compile:
from SezioneJoin sj in (
from A a in ...
join D d in ... on new { ... } equals new { ... }
where
d.D13 == "086" &&
!String.IsNullOrEmpty(a.A32) && a.A32 != "086"
orderby a.A21
orderby a.prog
select new SezioneJoin{...})
select Select
Error is
Unable to cast 'System.Linq.IQueryable<System.Linq.Expressions.Expression<System.Func<DiagnosticoSite.Data.Query.SezioneJoin,DiagnosticoSite.Data.Query.QueryRow>>>' into 'System.Linq.IQueryable<DiagnosticoSite.Data.Query.QueryRow>'
I can understand that the LINQ syntax requires the body of the select statement to be the inner type of the IQueryable that it returns, so the compiler is fooled into returning a list of expressions. With the Lambda syntax, the expression is a parameter that is either compiled in-line or returned by some other method (even dynamically!).
I would like to ask if there is any way to circumvent this and avoid defining large select expressions inline
protected virtual Expression> Select
I'd avoid using the names of any of the Linq-mapped methods (Select, Where, GroupBy, OrderBy, OrderByDescending) as member names. It works in this case, but when it causes problems by matching the definitions for those it can be confusing if you aren't in the habit of just not using those names unless you deliberately want to override Linq.
On a related note. Consider that:
from var item in source select item.Something
is equivalent to:
source.Select(item => item.Something);
Therefore:
from SezioneJoin sj in (/*…*/) select Select;
is equivalent to:
(/*…*/).Select(sj => Select);
That is, you arent' creating a query that executes the expression in Select, but one that returns the expression itself.
You should either just use the form .Select(Select) or use select sj => (Select)(sj) but that second one will (if I even have the parentheses correct to stop it clashing with Queryable.Select, I haven't tested that) call the Select property every time so is at best wasteful and at worse not going to be something a query provider can make use of, so it will fail with most linq-providers. In all, use the .Select(Select) form (and change the name).
(On a separate note, if you're going to buffer an expression, actually buffer it; create a private Expression<Func<SezioneJoin, QueryRow>> once and return it in the property's getter, rather than creating it every time).
Simply use extension method in place of last LINQ select statement:
var query = from SezioneJoin sj ... select new SezioneJoin{...});
var projection = query.Select(Select);
I am writing a dynamic linq query to realize paging, and I am now facing a problem, I need the System.Linq.Expressions.Expression.Like function, but it does not exist in System.Linq.Expressions.Expression, here is my code.
Expression mWhereFunc; // Filter clause
ParameterExpression mLinqParam; // Linq param
// Get current request page
string mCurPage = this.Request.QueryString["page"];
if (String.IsNullOrEmpty(mCurPage))
{
mCurPage = "1";
}
mLinqParam = Expression.Parameter(typeof(ORD_Order), "p");
mWhereFunc = Expression.Equal(Expression.Property(mLinqParam,
typeof(ORD_Order).GetProperty("ItemIsValid")),
Expression.Constant(true));
string mOrderSN = this.Request.QueryString["txtOrderSN"];
if (!String.IsNullOrEmpty(mOrderSN))
{
mWhereFunc = Expression.And(mWhereFunc,
**Expression.Equal**(Expression.Property(mLinqParam,
typeof(ORD_Order).GetProperty("OrderSN")),
Expression.Constant(mOrderSN)));
}
var mLambdaWhere = (Expression<Func<ORD_Order,
bool>>)Expression.Lambda<Func<ORD_Order, bool>>(mWhereFunc,
new ParameterExpression[] { mLinqParam });
Func<ORD_Order, Int32> mLambdaOrder = p => p.OrderID;
IORD_OrderRepository rptOrder = new ORD_OrderRepository();
ICTM_CustomerRepository rptCtm = new CTM_CustomerRepository();
var list = from o in rptOrder.GetAll()
.Where(mLambdaWhere)
.OrderBy(mLambdaOrder)
.Skip((int.Parse(mCurPage) - 1) * mPageSize).Take(mPageSize)
join c in rptCtm.GetAll()
on o.CustomerID equals c.CustomerID
select new
{
o.OrderID,
o.OrderSN,
o.CustomerID,
c.ContactName,
o.Status,
o.CreateDate,
o.Description
};
Expression.Equal is the place where I want to change it to Expression.Like
Any help will be appreciated.
I modified my code like this
mWhereFunc = Expression.And(mWhereFunc, Expression.Call(
Expression.Property(mLinqParam, typeof(ORD_Order).GetProperty("OrderSN")),
typeof(String).GetMethod("Contains"),
new Expression[] { Expression.Constant(mOrderSN) }));
and it works, many thanks to all of you.
Like equivalents for linq are String.Contains, String.StartsWith etc. http://www.simonrhart.com/2008/06/using-like-in-linq-to-sql-under-c.html
if you are using NHibernate you can create a Like method and use a Expression to call that
You can check this post on how to do the "Like" extension here: NHibernate Linq Provider Extension
And then you build the "Like" Expression like this:
Expression.Call(typeof(MyLinqExtensions).GetMethod("IsLike"), Expression.Property(mLinqParam,
typeof(ORD_Order).GetProperty("OrderSN")),
Expression.Constant(mOrderSN)));
EDIT: Just as an additional comment, you should use Expression.AndAlso instead of Expression.And since the first one is the && operator and the last one is the & operator
EDIT 2: For Entity Framework check this post (Linq To Entities scenario): SQL User-Defined Functions in Entity Framework 4, I have no experience with it but it seems the same as a NH provider, after doing that then building the Linq Expression should be the same i posted before
I'm building a dynamic query that can have n of Where method calls and n of SelectMany calls dependent upon user input. For example I may have:
var qZ = entityContext.TableA
.SelectMany(a=>a.TableB, (a,t)=>new{a,t} )
.Where(a=>a.t.FieldID==21)
.Where(a=> EntityFunctions.Left(a.t.Value,1)=="p")
.SelectMany(a=>a.a.TableC, (a,t)=>new{a,t} )
.Where(a=>a.t.FieldID==22)
.Where(a=> a.a.t.Value=="Peter" && a.t.Value=="Pan")
.Where(a=> a.a.a.TypeID==3)
.Select(a=> new{ a.a.a.ItemID }
).Distinct();
In the method I'm writing, I use helper methods that return an IQueryable as seen in the return line below.
return query.Provider.CreateQuery(
Expression.Call(typeof(Queryable),
"Where",
new Type[] {query.ElementType},
query.Expression, predicateLambda)
);
I'm able to create LambdaExpressions for all of the various query attribute-value pairs required, but I am unable to create one for the resultSelector of Queryable.SelectMany.
How can we create (a,t) => new{a=a, t=t} in an expression tree? Or How do we accomplish the same result as the .SelectMany above using Expression.Call like below?
Expression.Call(typeof(Queryable),
"SelectMany",
????????,
????????
);
I've tried using the SelectMany overload that doesn't require the resultSelector which works to some degree, however, I don't know how to reference the properties of t in subsequent method calls.
I've found this lambda expression ((a,t) => new{a=a, t=t}) associated with SelectMany all over the web, but I can't find any example of how to convert it to an expression tree.
UPDATE:
Let's reframe the question. I can pass the lambda like this
var q = entityContext.TableA.AsQueryable();
var q1 = Queryable.SelectMany(q, a => a.TableB, (a, t) => new { a = a, t = t });
var q2 = Queryable.Where(q1,a=>a.t.FieldID==22);
That works, however, since I don't know ahead of time how many SelectMany need to be called and since each call changes to anonymous type of the IQueriable, is there a way to cast (and re-cast) the anonymous type to a single variable? This way I can loop through and apply whatever method necessary to the variable and then enumerate to get the results once the query is built. Something like:
var q = entityContext.TableA..AsQueryable();
q = Queryable.SelectMany(q, a => a.TableB, (a, t) => new { a = a, t = t });
q = Queryable.Where(q,a=>a.t.FieldID==22);
(BTW: This doesn't work)
The way that I ended up resolving this required a paradigm shift. The first query above was based upon the fact that I learned to write queries by joining all the tables I needed together to give me access to filter on and select fields in those tables.
SelectMany() creates joins and the box around my thinking at the time required that if I need to filter on a specific column in a table, I had to join that table to my query. This in turn changed the type of my IQueryable resulting in my not being able to predict the Type of the IQueryable at design time.
Answer:
Step 1: Set the type of IQueryable to the output type it needs to return. In the case above, the result was always IQueryable.
Step 2: Utilize Expressions to dynamically create the WHERE predicate, including any and all tables necessary to create the proper filter. This always returns Expression> and all of the other variables we easily accounted for. And rememeber, in EF it isn't necessary to join table outside of Where() if they are only needed in Where().
I would like to create a repository model that could take an Expression and use Linq-To-Sql to generate the required SQL statement.
For example, I have a function such as this:
// Possible criteria
Expression<Func<Purchase,bool>> criteria1 = p => p.Price > 1000;
// Function that should take that criteria and convert to SQL statement
static IEnumerable<Customer> GetCustomers (Expression<Func<Purchase,bool>> criteria)
{
// ...
}
Inside the function, I would like to convert criteria to a SQL statement using Linq-To-Sql.
I am aware that you can use DataContext.Log to see the executed queries and DataContext.GetCommand(query).CommandText to see the full query before it is executed. However, I would like just a part of the entire expression generated.
What I am hoping to accomplish is to make my repository abstract the underlying technology (Linq-to-Sql, Dapper, etc). That way I could pass the Expression to the repository, have it generate the right statement and use the right technology to execute it.
You could do something like this:
string sql = DataContext.GetTable<Customer>().Where(criteria).ToString();
ToString() gives you the SQL expression. You could then use regex to pull out the WHERE clause.
This is a code excerpt that I use to build my own predicate to use in the Where function. The compiler can't cope with ienumerables of complex objects, so you have to do it yourself.
Essentially, the code gets passed an ienumerable of (string code, string exchange) tuples, and then builds an expression to retrieve all Security objects that have Security.Code == tuple.Code AND (Security.MasterExchangeForStocksId == tuple.exchange OR SecurityExchangeId == tuple.exchange).
CreateTrEntitiesAsync() simply returns a Entity Framework context, which has a DbSet Security property.
public async Task<Security[]> GetSecurities(IEnumerable<(string code, string exchange)> tickers)
{
using (var ctx = await CreateTrEntitiesAsync())
{
var securityExpr = Expression.Parameter(typeof(Security), "security");
Expression expr = null;
Expression exprToadd;
foreach (var item in tickers)
{
exprToadd = Expression.And(
Expression.Equal(Expression.Property(securityExpr, nameof(Security.Code)), Expression.Constant(item.code)),
Expression.Or(
Expression.Equal(Expression.Property(Expression.Property(securityExpr, nameof(Security.Exchange)), nameof(Exchange.MasterExchangeForStocksId)), Expression.Constant(item.exchange)),
Expression.Equal(Expression.Property(securityExpr, nameof(Security.ExchangeId)), Expression.Constant(item.exchange))
)
);
if (expr == null)
expr = exprToadd;
else
expr = Expression.Or(expr, exprToadd);
}
var criteria = Expression.Lambda<Func<Security, bool>>(expr, new ParameterExpression[] { securityExpr });
var items = ctx.Securities.Where(criteria);
return await items.ToArrayAsync();
}
}