Select only specific fields with Linq (EF core) - c#

I have a DbContext where I would like to run a query to return only specific columns, to avoid fetching all the data.
The problem is that I would like to specify the column names with a set of strings, and I would like to obtain an IQueryable of the original type, i.e. without constructing an anonymous type.
Here is an example:
// Install-Package Microsoft.AspNetCore.All
// Install-Package Microsoft.EntityFrameworkCore
using Microsoft.EntityFrameworkCore;
using System;
using System.Linq;
public class Person {
public int Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
public class TestContext : DbContext {
public virtual DbSet<Person> Persons { get; set; }
public TestContext(DbContextOptions<TestContext> options) : base(options) {
}
}
class Program {
static void Main(string[] args) {
var builder = new DbContextOptionsBuilder<TestContext>();
builder.UseInMemoryDatabase(Guid.NewGuid().ToString());
var context = new TestContext(builder.Options);
context.Persons.Add(new Person { FirstName = "John", LastName = "Doe" });
context.SaveChanges();
// How can I express this selecting columns with a set of strings?
IQueryable<Person> query = from p in context.Persons select new Person { FirstName = p.FirstName };
}
}
I would like to have something like this method:
static IQueryable<Person> GetPersons(TestContext context, params string[] fieldsToSelect) {
// ...
}
Is there a way I can do this?

Since you are projecting (selecting) the members of the type T to the same type T, the required Expression<Func<T, T>> can relatively easy be created with Expression class methods like this:
public static partial class QueryableExtensions
{
public static IQueryable<T> SelectMembers<T>(this IQueryable<T> source, params string[] memberNames)
{
var parameter = Expression.Parameter(typeof(T), "e");
var bindings = memberNames
.Select(name => Expression.PropertyOrField(parameter, name))
.Select(member => Expression.Bind(member.Member, member));
var body = Expression.MemberInit(Expression.New(typeof(T)), bindings);
var selector = Expression.Lambda<Func<T, T>>(body, parameter);
return source.Select(selector);
}
}
Expression.MemberInit is the expression equivalent of the new T { Member1 = x.Member1, Member2 = x.Member2, ... } C# construct.
The sample usage would be:
return context.Set<Person>().SelectMembers(fieldsToSelect);

This can be achieved by using Dynamic Linq.
and for .Net Core - System.Linq.Dynamic.Core
With Dynamic Linq you can pass in your SELECT and WHERE as a string.
Using your example, you could then do something like:
IQueryable<Person> query = context.Persons
.Select("new Person { FirstName = p.FirstName }");

Based on answer of Ivan I made crude version of caching function to eliminate the toll layed on us by using of reflexion. It allow as to lower this toll from milliseconds to microseconds on repeated requests (typical for DbAccess API, for example).
public static class QueryableExtensions
{
public static IQueryable<T> SelectMembers<T>(this IQueryable<T> source, IEnumerable<string> memberNames)
{
var result = QueryableGenericExtensions<T>.SelectMembers(source, memberNames);
return result;
}
}
public static class QueryableGenericExtensions<T>
{
private static readonly ConcurrentDictionary<string, ParameterExpression> _parameters = new();
private static readonly ConcurrentDictionary<string, MemberAssignment> _bindings = new();
private static readonly ConcurrentDictionary<string, Expression<Func<T, T>>> _selectors = new();
public static IQueryable<T> SelectMembers(IQueryable<T> source, IEnumerable<string> memberNames)
{
var parameterName = typeof(T).FullName;
var requestName = $"{parameterName}:{string.Join(",", memberNames.OrderBy(x => x))}";
if (!_selectors.TryGetValue(requestName, out var selector))
{
if (!_parameters.TryGetValue(parameterName, out var parameter))
{
parameter = Expression.Parameter(typeof(T), typeof(T).Name.ToLowerInvariant());
_ = _parameters.TryAdd(parameterName, parameter);
}
var bindings = memberNames
.Select(name =>
{
var memberName = $"{parameterName}:{name}";
if (!_bindings.TryGetValue(memberName, out var binding))
{
var member = Expression.PropertyOrField(parameter, name);
binding = Expression.Bind(member.Member, member);
_ = _bindings.TryAdd(memberName, binding);
}
return binding;
});
var body = Expression.MemberInit(Expression.New(typeof(T)), bindings);
selector = Expression.Lambda<Func<T, T>>(body, parameter);
_selectors.TryAdd(requestName, selector);
}
return source.Select(selector);
}
}
Example of results after sequential run with same params (please note that this is NANOseconds):
SelectMembers time ... 3092214 ns
SelectMembers time ... 145724 ns
SelectMembers time ... 38613 ns
SelectMembers time ... 1969 ns
I have no idea why the time decreases gradually, not from "without cache" to "with cache", may be it is because of my environment with loop of questioning 4 servers with same request and some deep-level magic with asyncs. Repeating request produces consistent results similar to the last one +/- 1-2 microseconds.

Try this code:
string fieldsToSelect = "new Person { FirstName = p.FirstName }"; //Pass this as parameter.
public static IQueryable<Person> GetPersons(TestContext context, string fieldsToSelect)
{
IQueryable<Person> query = context.Persons.Select(fieldsToSelect);
}

I was able to do this with the package https://github.com/StefH/System.Linq.Dynamic.Core so easily.
Here is an example code.
use namespacing, using System.Linq.Dynamic.Core;
//var selectQuery = "new(Name, Id, PresentDetails.RollNo)";
var selectQuery = "new(Name, Id, PresentDetails.GuardianDetails.Name as GuardianName)";
var students = dbContext.Students
.Include(s => s.PresentDetails)
.Include(s => s.PresentDetails.GuardianDetails)
.Where(s => s.StudentStatus == "Admitted")
.Select(selectQuery);

var students = dbContext.Students
.Include(s => s.PresentDetails)
.Where(s => s.StudentStatus == "Admitted")
.Select(p => new Person()
{
Id = p.Id,
Name = p.Name
});
Why not minimize the selected columns in the regular way? this is way cleaner.

Related

Custom expression for RavenDB where clause: field contains any item from provided array

Could you please help reviewing the expressions which neither seem to work?
Trying to find records where a record property #In a provided array of values.
RavenDB 3.5 throwing exception that it did not manage to understand specified expression.
I am trying to avoid using the client sdk and resolve this with custom expressions, either with a similar sdk .In extension or Enumerable.Any.
Any help is appreciated
System.NotSupportedException: 'Could not understand expression
void Main()
{
var exp = ByPersonNameIn(new[] { "Walter" });
//var exp = ByPersonNameAny(new[] { "Walter" });
var persons = new List<Person>()
{
new Person { Name = "Walter" },
new Person { Name = "Jesse" },
new Person { Name = "Walter" },
};
// Expression works here, but not at IRavenQueryable.Where
var res = persons.Where(exp.Compile()).ToList();
res.Dump(nameof(res));
}
private Expression<Func<Person, bool>> ByPersonNameIn(string[] names)
{
var person = Expression.Parameter(typeof(Person), "person");
var personProp = Expression.PropertyOrField(person, nameof(Person.Name));
var namesParam = Expression.Constant(names, typeof(string[]));
var #in = typeof(StringExt).GetMethod("In", new[] { typeof(string), typeof(IEnumerable<string>) });
var inExp = Expression.Call(#in, personProp, namesParam);
return Expression.Lambda<Func<Person, bool>>(inExp, person);
}
private Expression<Func<Person, bool>> ByPersonNameAny(string[] names)
{
var person = Expression.Parameter(typeof(Person), "person");
var name = Expression.Parameter(typeof(string), "name");
var namesParam = Expression.Constant(names, typeof(string[]));
var #eq = Expression.Equal(name, (Expression.PropertyOrField(person, nameof(Person.Name))));
var #any = Expression.Call(typeof(Enumerable), "Any", new[] { typeof(string) }, namesParam, Expression.Lambda(#eq, name));
return Expression.Lambda<Func<Person, bool>>(#any, person);
}
public class Person
{
public string Name {get;set;}
}
public static class StringExt
{
public static bool In(this string field, IEnumerable<string> values)
{
return values.Any(value => field.Equals(value));
}
public static bool In(this string field, params string[] values)
{
return values.Any(value => field.Equals(value));
}
}
The RavenDB library has code in its LINQ provider to translate queries on a IRavenQueryable to queries RavenDB can understand (RQL). This translation is limited to the methods (and operators, etc) it is specifically written to handle, and when you bring it something it doesn't understand, like your own extension method, then it throws the exception to indicate that you have "fallen off the edge".
Most LINQ providers understand the pattern var personNames = new[]{ "Walter" }; something .Where(person => personNames.Contains(person)) to do what you want - ie where Enumerable.Contains is used to do an IN/any query. I think constructing the expression tree to use this method will work. Just plain using it in the LINQ query directly instead of the custom method would also work in your example. There doesn't seem to be a way to extend the LINQ provider to teach it how your method would be translated.
I had some "luck" mimicking #In as OR, which is not ideal, but should work for the time being until I find something better.
private static Expression<Func<TDocument, bool>> ByPropertyViaEqOr<TDocument>(string propName, string[] values)
{
var doc = Expression.Parameter(typeof(TDocument), "doc");
var eq = values.Select(value => ByPropertyViaEq<TDocument>(propName, value));
var orElse = eq.Aggregate((l, r) => Expression.OrElse(l, r));
return Expression.Lambda<Func<TDocument, bool>>(orElse, doc);
}
private static Expression ByPropertyViaEq<TDocument>(string propName, string value)
{
var doc = Expression.Parameter(typeof(TDocument), "doc");
var docProp = Expression.PropertyOrField(doc, propName);
var propValue = Expression.Constant(value, typeof(string));
return Expression.Equal(docProp, propValue);
}

LINQ select property by name [duplicate]

This question already has answers here:
Dynamic LINQ OrderBy on IEnumerable<T> / IQueryable<T>
(24 answers)
Closed 2 years ago.
The community reviewed whether to reopen this question last year and left it closed:
Original close reason(s) were not resolved
I'm attempting to use a variable inside of a LINQ select statement.
Here is an example of what I'm doing now.
using System;
using System.Collections.Generic;
using System.Linq;
using Faker;
namespace ConsoleTesting
{
internal class Program
{
private static void Main(string[] args)
{
List<Person> listOfPersons = new List<Person>
{
new Person(),
new Person(),
new Person(),
new Person(),
new Person(),
new Person(),
new Person(),
new Person(),
new Person(),
new Person(),
new Person()
};
var firstNames = Person.GetListOfAFirstNames(listOfPersons);
foreach (var item in listOfPersons)
{
Console.WriteLine(item);
}
Console.WriteLine();
Console.ReadKey();
}
public class Person
{
public string City { get; set; }
public string CountryName { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public Person()
{
FirstName = NameFaker.Name();
LastName = NameFaker.LastName();
City = LocationFaker.City();
CountryName = LocationFaker.Country();
}
public static List<string> GetListOfAFirstNames(IEnumerable<Person> listOfPersons)
{
return listOfPersons.Select(x => x.FirstName).Distinct().OrderBy(x => x).ToList();
}
public static List<string> GetListOfCities(IEnumerable<Person> listOfPersons)
{
return listOfPersons.Select(x => x.FirstName).Distinct().OrderBy(x => x).ToList();
}
public static List<string> GetListOfCountries(IEnumerable<Person> listOfPersons)
{
return listOfPersons.Select(x => x.FirstName).Distinct().OrderBy(x => x).ToList();
}
public static List<string> GetListOfLastNames(IEnumerable<Person> listOfPersons)
{
return listOfPersons.Select(x => x.FirstName).Distinct().OrderBy(x => x).ToList();
}
}
}
}
I have a Some very not DRY code with the GetListOf... Methods
i feel like i should be able to do something like this
public static List<string> GetListOfProperty(
IEnumerable<Person> listOfPersons, string property)
{
return listOfPersons.Select(x =>x.property).Distinct().OrderBy(x=> x).ToList();
}
but that is not vaild code. I think the key Might Relate to Creating a Func
if That is the answer how do I do that?
Here is a second attempt using refelection But this is also a no go.
public static List<string> GetListOfProperty(IEnumerable<Person>
listOfPersons, string property)
{
Person person = new Person();
Type t = person.GetType();
PropertyInfo prop = t.GetProperty(property);
return listOfPersons.Select(prop).Distinct().OrderBy(x =>
x).ToList();
}
I think the refection might be a DeadEnd/red herring but i thought i would show my work anyway.
Note Sample Code is simplified in reality this is used to populate a datalist via AJAX to Create an autocomplete experience. That object has 20+ properties and I can complete by writing 20+ methods but I feel there should be a DRY way to complete this. Also making this one method also would clean up my controller action a bunch also.
Question:
Given the first section of code is there a way to abstract those similar methods into a single method buy passing some object into the select Statement???
Thank you for your time.
You would have to build the select
.Select(x =>x.property).
by hand. Fortunately, it isn't a tricky one since you expect it to always be the same type (string), so:
var x = Expression.Parameter(typeof(Person), "x");
var body = Expression.PropertyOrField(x, property);
var lambda = Expression.Lambda<Func<Person,string>>(body, x);
Then the Select above becomes:
.Select(lambda).
(for LINQ based on IQueryable<T>) or
.Select(lambda.Compile()).
(for LINQ based on IEnumerable<T>).
Note that anything you can do to cache the final form by property would be good.
From your examples, I think what you want is this:
public static List<string> GetListOfProperty(IEnumerable<Person>
listOfPersons, string property)
{
Type t = typeof(Person);
PropertyInfo prop = t.GetProperty(property);
return listOfPersons
.Select(person => (string)prop.GetValue(person))
.Distinct()
.OrderBy(x => x)
.ToList();
}
typeof is a built-in operator in C# that you can "pass" the name of a type to and it will return the corresponding instance of Type. It works at compile-time, not runtime, so it doesn't work like normal functions.
PropertyInfo has a GetValue method that takes an object parameter. The object is which instance of the type to get the property value from. If you are trying to target a static property, use null for that parameter.
GetValue returns an object, which you must cast to the actual type.
person => (string)prop.GetValue(person) is a lamba expression that has a signature like this:
string Foo(Person person) { ... }
If you want this to work with any type of property, make it generic instead of hardcoding string.
public static List<T> GetListOfProperty<T>(IEnumerable<Person>
listOfPersons, string property)
{
Type t = typeof(Person);
PropertyInfo prop = t.GetProperty(property);
return listOfPersons
.Select(person => (T)prop.GetValue(person))
.Distinct()
.OrderBy(x => x)
.ToList();
}
I would stay away from reflection and hard coded strings where possible...
How about defining an extension method that accepts a function selector of T, so that you can handle other types beside string properties
public static List<T> Query<T>(this IEnumerable<Person> instance, Func<Person, T> selector)
{
return instance
.Select(selector)
.Distinct()
.OrderBy(x => x)
.ToList();
}
and imagine that you have a person class that has an id property of type int besides those you already expose
public class Person
{
public int Id { get; set; }
public string City { get; set; }
public string CountryName { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
all you need to do is fetch the results with type safe lambda selectors
var ids = listOfPersons.Query(p => p.Id);
var firstNames = listOfPersons.Query(p => p.FirstName);
var lastNames = listOfPersons.Query(p => p.LastName);
var cityNames = listOfPersons.Query(p => p.City);
var countryNames = listOfPersons.Query(p => p.CountryName);
Edit
As it seems you really need hardcoded strings as the property inputs, how about leaving out some dynamism and use a bit of determinism
public static List<string> Query(this IEnumerable<Person> instance, string property)
{
switch (property)
{
case "ids": return instance.Query(p => p.Id.ToString());
case "firstName": return instance.Query(p => p.FirstName);
case "lastName": return instance.Query(p => p.LastName);
case "countryName": return instance.Query(p => p.CountryName);
case "cityName": return instance.Query(p => p.City);
default: throw new Exception($"{property} is not supported");
}
}
and access the desired results as such
var cityNames = listOfPersons.Query("cityName");
You should be able to do it with Reflection. I use it something similar.
Just change your reflection try to this:
public static List<string> GetListOfValues(IEnumerable<Person> listOfPersons, string propertyName)
{
var ret = new List<string>();
PropertyInfo prop = typeof(Person).GetProperty(propertyName);
if (prop != null)
ret = listOfPersons.Select(p => prop.GetValue(p).ToString()).Distinct().OrderBy(x => x).ToList();
return ret;
}
I hope it helps.
It's based on C# 6
You can also use this. works for me.
public static class ObjectReflectionExtensions
{
public static object GetValueByName<T>(this T thisObject, string propertyName)
{
PropertyInfo prop = typeof(T).GetProperty(propertyName);
return prop.GetValue(thisObject);
}
}
And call like this.
public static List<string> GetListOfProperty(IEnumerable<Person> listOfPersons, string propertyName)
{
return listOfPersons.Select(x =>(string)x.GetValueByName(propertyName)).Distinct().OrderBy(x=> x).ToList();
}
If you want to select all the values:
object[] foos = objects.Select(o => o.GetType().GetProperty("PropertyName").GetValue(o)).ToArray();

How to solve LINQ to Entity query duplication when the queries only differ by a property?

I have two DbSets, Foo and Bar. Foo has an identifying string property, FooName, and Bar has an identifying string property, BarName.
I am designing a very simple search feature, where a user's query term can either be equal to, or contained in the identifying name.
So I have two methods (heavily simplified):
public ActionView SearchFoo(string query)
{
var equalsQuery = db.Foo.Where(f => f.FooName.Equals(query));
var containsQuery = db.Foo.Where(f => f.FooName.Contains(query)).Take(10); // Don't want too many or else a search for "a" would yield too many results
var result = equalsQuery.Union(containsQuery).ToList();
... // go on to return a view
}
public ActionView SearchBar(string query)
{
var equalsQuery = db.Bar.Where(f => f.BarName.Equals(query));
var containsQuery = db.Bar.Where(f => f.BarName.Contains(query)).Take(10); // Don't want too many or else a search for "a" would yield too many results
var result = equalsQuery.Union(containsQuery).ToList();
... // go on to return a view
}
Clearly I want some helper method like so:
public IList<T> Search<T>(string query, DbSet<T> set)
{
var equalsQuery = set.Where(f => ???.Equals(query));
var containsQuery = set.Where(f => ???.Contains(query)).Take(10); // Don't want too many or else a search for "a" would yield too many results
var result = equalsQuery.Union(containsQuery).ToList();
... // go on to return a view
}
I originally tried to add a Func<T, string> to the Search parameters, where I could use f => f.FooName and b => b.BarName respectively, but LINQ to Entities doesn't support a lambda expression during the execution of the query.
I've been scratching my head as to how I can extract this duplication.
You can achieve this with Expression<Funt<T,string>>
public IList<T> Search<T>(string query, DbSet<T> set, Expression<Func<T, string>> propExp)
{
MethodInfo method = typeof(string).GetMethod("Contains", new[] { typeof(string) });
ConstantExpression someValue = Expression.Constant(query, typeof(string));
MethodCallExpression containsMethodExp =
Expression.Call(propExp.Body, method, someValue);
var e = (Expression<Func<T, bool>>)
Expression.Lambda(containsMethodExp, propExp.Parameters.ToArray());
var containsQuery = set.Where(e).Take(10);
BinaryExpression equalExpression = Expression.Equal(propExp.Body, someValue);
e = (Expression<Func<T, bool>>)
Expression.Lambda(equalExpression, propExp.Parameters.ToArray());
var equalsQuery = set.Where(e);
var result = equalsQuery.Union(containsQuery).ToList();
}
Then you'll call it:
Search ("myValue", fooSet, foo=>foo.FooName);
if you can have a static method, then you could have it as an extension method:
public static IList<T> Search<T>(this DbSet<T> set,
string query, Expression<Func<T, string>> propExp)
And call it:
FooSet.Search ("myValue", foo=>foo.FooName);
Here's one way to do it.
First you need a helper method to generate the Expression for you:
private Expression<Func<T, bool>> GetExpression<T>(string propertyName, string propertyValue, string operatorName)
{
var parameterExp = Expression.Parameter(typeof(T));
var propertyExp = Expression.Property(parameterExp, propertyName);
MethodInfo method = typeof(string).GetMethod(operatorName, new[] { typeof(string) });
var someValue = Expression.Constant(propertyValue, typeof(string));
var methodExp = Expression.Call(propertyExp, method, someValue);
return Expression.Lambda<Func<T, bool>>(methodExp, parameterExp);
}
This is how you can use this method, propertyName would be FooName and BarName:
public IList<T> Search<T>(string propertyName, string query, DbSet<T> set)
{
var equalsQuery = set.Where(GetExpression<T>(propertyName, query, "Equals"));
var containsQuery = set.Where(GetExpression<T>(propertyName, query, "Contains")).Take(10); // Don't want too many or else a search for "a" would yield too many results
var result = equalsQuery.Union(containsQuery).ToList();
return result;
}
You can create interface:
public interface IName
{
string Name { get; set; }
}
Then implicitly implement IName interface in both entities.
public class Bar : IName { ... }
public class Foo : IName { ... }
And then change your method as:
public IList<T> SearchByName<T>(string query, DbSet<T> set)
where T: class, IName
{
var equalsQuery = set.Where(f => f.Name.Equals(query));
var containsQuery = set.Where(f => f.Name.Contains(query)).Take(10); // Don't want too many or else a search for "a" would yield too many results
var result = equalsQuery.Union(containsQuery).ToList();
... // go on to return a view
}
You could overide your ToString() method and use that in the query
public class foo
{
public string FooName
{
get;
set;
}
public override string ToString()
{
return FooName;
}
}
public class Bar
{
public string BarName
{
get;
set;
}
public override string ToString()
{
return BarName;
}
}
public IList<T> Search<T>(string query, DbSet<T> set)
{
var equalsQuery = set.AsEnumerable().Where(f => f.ToString().Equals(query));
var containsQuery = set.AsEnumerable().Where(f => f.ToString().Contains(query)).Take(10);
var result = equalsQuery.Union(containsQuery).ToList(); . . . // go on to return a view
}

Can a string-based Include alternative be created in Entity Framework Core?

On an API I need dynamic include, but EF Core does not support string-based include.
Because of this, I created a mapper which maps strings to lambda expressions added to a list as:
List<List<Expression>> expressions = new List<List<Expression>>();
Consider the following specific types:
public class EFContext
{
public DbSet<P1> P1s { get; set; }
public DbSet<P1> P2s { get; set; }
public DbSet<P1> P3s { get; set; }
}
public class P1
{
public P2 P2 { get; set; }
public P3 P3 { get; set; }
}
public class P2
{
public P3 P3 { get; set; }
}
public class P3 { }
Include and ThenInclude are normally used as follows:
EFContext efcontext = new EFContext();
IQueryable<P1> result = efcontext.P1s
.Include(p1 => p1.P2)
.ThenInclude(p2 => p2.P3)
.Include(p1 => p1.P3);
They can also be used the following way:
Expression<Func<P1, P2>> p1p2 = p1 => p1.P2;
Expression<Func<P1, P3>> p1p3 = p1 => p1.P3;
Expression<Func<P2, P3>> p2p3 = p2 => p2.P3;
List<List<Expression>> expressions = new List<List<Expression>>
{
new List<Expression> { p1p2, p1p3 },
new List<Expression> { p2p3 }
};
EFContext efcontext = new EFContext();
IIncludableQueryable<P1, P2> q1 = EntityFrameworkQueryableExtensions
.Include(efcontext.P1s, p1p2);
IIncludableQueryable<P1, P3> q2 = EntityFrameworkQueryableExtensions
.ThenInclude(q1, p2p3);
IIncludableQueryable<P1, P3> q3 = EntityFrameworkQueryableExtensions
.Include(q2, p1p3);
result = q3.AsQueryable();
The problem is that my method receives a list of Expressions and I only have the base type in T:
public static class IncludeExtensions<T>
{
public static IQueryable<T> IncludeAll(this IQueryable<T> collection, List<List<Expression>> expressions)
{
MethodInfo include = typeof(EntityFrameworkQueryableExtensions)
.GetTypeInfo()
.GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.Include))
.Single(mi => mi.GetParameters()
.Any(pi => pi.Name == "navigationPropertyPath"));
MethodInfo includeAfterCollection = typeof(EntityFrameworkQueryableExtensions)
.GetTypeInfo()
.GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude))
.Single(mi =>
!mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);
MethodInfo includeAfterReference = typeof(EntityFrameworkQueryableExtensions)
.GetTypeInfo()
.GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude))
.Single(mi => mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);
foreach (List<Expression> path in expressions)
{
bool start = true;
foreach (Expression expression in path)
{
if (start)
{
MethodInfo method = include.MakeGenericMethod(typeof(T), ((LambdaExpression)expression).ReturnType);
IIncludableQueryable<T,?> result = method.Invoke(null, new Object[] { collection, expression });
start = false;
}
else
{
MethodInfo method = includeAfterReference.MakeGenericMethod(typeof(T), typeof(?), ((LambdaExpression)expression).ReturnType);
IIncludableQueryable <T,?> result = method.Invoke(null, new Object[] { collection, expression });
}
}
}
return collection; // (to be replaced by final as Queryable)
}
}
The main problem has been resolving the correct types for each Include and ThenInclude step and also which ThenInclude to use.
Is this even possible with the current EF7 Core? Did someone find a solution for dynamic Include?
The Include and ThenIncludeAfterReference and ThenIncludeAfterCollection methods are part of EntityFrameworkQueryableExtensions class in EntityFramework Github's repository.
Update:
Starting with v1.1.0, the string based include is now part of EF Core, so the issue and the below solution are obsolete.
Original answer:
Interesting exercise for the weekend.
Solution:
I've ended up with the following extension method:
public static class IncludeExtensions
{
private static readonly MethodInfo IncludeMethodInfo = typeof(EntityFrameworkQueryableExtensions).GetTypeInfo()
.GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.Include)).Single(mi => mi.GetParameters().Any(pi => pi.Name == "navigationPropertyPath"));
private static readonly MethodInfo IncludeAfterCollectionMethodInfo = typeof(EntityFrameworkQueryableExtensions).GetTypeInfo()
.GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude)).Single(mi => !mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);
private static readonly MethodInfo IncludeAfterReferenceMethodInfo = typeof(EntityFrameworkQueryableExtensions).GetTypeInfo()
.GetDeclaredMethods(nameof(EntityFrameworkQueryableExtensions.ThenInclude)).Single(mi => mi.GetParameters()[0].ParameterType.GenericTypeArguments[1].IsGenericParameter);
public static IQueryable<TEntity> Include<TEntity>(this IQueryable<TEntity> source, params string[] propertyPaths)
where TEntity : class
{
var entityType = typeof(TEntity);
object query = source;
foreach (var propertyPath in propertyPaths)
{
Type prevPropertyType = null;
foreach (var propertyName in propertyPath.Split('.'))
{
Type parameterType;
MethodInfo method;
if (prevPropertyType == null)
{
parameterType = entityType;
method = IncludeMethodInfo;
}
else
{
parameterType = prevPropertyType;
method = IncludeAfterReferenceMethodInfo;
if (parameterType.IsConstructedGenericType && parameterType.GenericTypeArguments.Length == 1)
{
var elementType = parameterType.GenericTypeArguments[0];
var collectionType = typeof(ICollection<>).MakeGenericType(elementType);
if (collectionType.IsAssignableFrom(parameterType))
{
parameterType = elementType;
method = IncludeAfterCollectionMethodInfo;
}
}
}
var parameter = Expression.Parameter(parameterType, "e");
var property = Expression.PropertyOrField(parameter, propertyName);
if (prevPropertyType == null)
method = method.MakeGenericMethod(entityType, property.Type);
else
method = method.MakeGenericMethod(entityType, parameter.Type, property.Type);
query = method.Invoke(null, new object[] { query, Expression.Lambda(property, parameter) });
prevPropertyType = property.Type;
}
}
return (IQueryable<TEntity>)query;
}
}
Test:
Model:
public class P
{
public int Id { get; set; }
public string Info { get; set; }
}
public class P1 : P
{
public P2 P2 { get; set; }
public P3 P3 { get; set; }
}
public class P2 : P
{
public P4 P4 { get; set; }
public ICollection<P1> P1s { get; set; }
}
public class P3 : P
{
public ICollection<P1> P1s { get; set; }
}
public class P4 : P
{
public ICollection<P2> P2s { get; set; }
}
public class MyDbContext : DbContext
{
public DbSet<P1> P1s { get; set; }
public DbSet<P2> P2s { get; set; }
public DbSet<P3> P3s { get; set; }
public DbSet<P4> P4s { get; set; }
// ...
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<P1>().HasOne(e => e.P2).WithMany(e => e.P1s).HasForeignKey("P2Id").IsRequired();
modelBuilder.Entity<P1>().HasOne(e => e.P3).WithMany(e => e.P1s).HasForeignKey("P3Id").IsRequired();
modelBuilder.Entity<P2>().HasOne(e => e.P4).WithMany(e => e.P2s).HasForeignKey("P4Id").IsRequired();
base.OnModelCreating(modelBuilder);
}
}
Usage:
var db = new MyDbContext();
// Sample query using Include/ThenInclude
var queryA = db.P3s
.Include(e => e.P1s)
.ThenInclude(e => e.P2)
.ThenInclude(e => e.P4)
.Include(e => e.P1s)
.ThenInclude(e => e.P3);
// The same query using string Includes
var queryB = db.P3s
.Include("P1s.P2.P4", "P1s.P3");
How it works:
Given a type TEntity and a string property path of the form Prop1.Prop2...PropN, we split the path and do the following:
For the first property we just call via reflection the EntityFrameworkQueryableExtensions.Include method:
public static IIncludableQueryable<TEntity, TProperty>
Include<TEntity, TProperty>
(
this IQueryable<TEntity> source,
Expression<Func<TEntity, TProperty>> navigationPropertyPath
)
and store the result. We know TEntity and TProperty is the type of the property.
For the next properties it's a bit more complex. We need to call one of the following ThenInclude overloads:
public static IIncludableQueryable<TEntity, TProperty>
ThenInclude<TEntity, TPreviousProperty, TProperty>
(
this IIncludableQueryable<TEntity, ICollection<TPreviousProperty>> source,
Expression<Func<TPreviousProperty, TProperty>> navigationPropertyPath
)
and
public static IIncludableQueryable<TEntity, TProperty>
ThenInclude<TEntity, TPreviousProperty, TProperty>
(
this IIncludableQueryable<TEntity, TPreviousProperty> source,
Expression<Func<TPreviousProperty, TProperty>> navigationPropertyPath
)
source is the current result. TEntity is one and the same for all calls. But what is TPreviousProperty and how we decide which method to call.
Well, first we use a variable to remember what was the TProperty in the previous call. Then we check if it is a collection property type, and if yes, we call the first overload with TPreviousProperty type extracted from the generic arguments of the collection type, otherwise simply call the second overload with that type.
And that's all. Nothing fancy, just emulating an explicit Include / ThenInclude call chains via reflection.
String-based Include() shipped in EF Core 1.1. I would suggest you try upgrading and removing any workarounds you had to add to your code to address this limitation.
Creating an "IncludeAll" extension on query will require a different approach from what you have initially done.
EF Core does expression interpretation. When it sees the .Include method, it interprets this expression into creating additional queries. (See RelationalQueryModelVisitor.cs and IncludeExpressionVisitor.cs).
One approach would be to add an additional expression visitor that handles your IncludeAll extension. Another (and probably better) approach would be to interpret the expression tree from .IncludeAll to the appropriate .Includes and then let EF handle the includes normally. An implementation of either is non-trivial and beyond the scope of a SO answer.
String-based Include() shipped in EF Core 1.1. If you keep this extension you will get error "Ambiguous match found". I spent half day to search solution for this error. Finally i removed above extension and error was resolved.

How to Convert Lambda Expression To Sql?

I am developing a small framework to access the database. I want to add a feature that makes a query using a lambda expression. How do I do this?
public class TestModel
{
public int Id {get;set;}
public string Name {get;set;}
}
public class Repository<T>
{
// do something.
}
For example:
var repo = new Repository<TestModel>();
var query = repo.AsQueryable().Where(x => x.Name == "test");
// This query must be like this:
// SELECT * FROM testmodel WHERE name = 'test'
var list = query.ToDataSet();
// When I call ToDataSet(), it will get the dataset after running the made query.
Go on and create a LINQ Provider (I am sure you don't want to do this, anyway).
It's a lot of work, so maybe you just want to use NHibernate or Entity Framework or something like that.
If your queries are rather simple, maybe you don't need a full blown LINQ Provider. Have a look at Expression Trees (which are used by LINQ Providers).
You can hack something like this:
public static class QueryExtensions
{
public static IEnumerable<TSource> Where<TSource>(this Repo<TSource> source, Expression<Func<TSource, bool>> predicate)
{
// hacks all the way
dynamic operation = predicate.Body;
dynamic left = operation.Left;
dynamic right = operation.Right;
var ops = new Dictionary<ExpressionType, String>();
ops.Add(ExpressionType.Equal, "=");
ops.Add(ExpressionType.GreaterThan, ">");
// add all required operations here
// Instead of SELECT *, select all required fields, since you know the type
var q = String.Format("SELECT * FROM {0} WHERE {1} {2} {3}", typeof(TSource), left.Member.Name, ops[operation.NodeType], right.Value);
return source.RunQuery(q);
}
}
public class Repo<T>
{
internal IEnumerable<T> RunQuery(string query)
{
return new List<T>(); // run query here...
}
}
public class TestModel
{
public int Id { get; set; }
public string Name { get; set; }
}
class Program
{
static void Main(string[] args)
{
var repo = new Repo<TestModel>();
var result = repo.Where(e => e.Name == "test");
var result2 = repo.Where(e => e.Id > 200);
}
}
Please, don't use this as it is. This is just a quick and dirty example how expression trees can be analyzed to create SQL statements.
Why not just use Linq2Sql, NHibernate or EntityFramework...
if you want to do things like
db.Employee
.Where(e => e.Title == "Spectre")
.Set(e => e.Title, "Commander")
.Update();
or
db
.Into(db.Employee)
.Value(e => e.FirstName, "John")
.Value(e => e.LastName, "Shepard")
.Value(e => e.Title, "Spectre")
.Value(e => e.HireDate, () => Sql.CurrentTimestamp)
.Insert();
or
db.Employee
.Where(e => e.Title == "Spectre")
.Delete();
Then check out this, BLToolkit
You might want to look at http://iqtoolkit.codeplex.com/ Which is very complex and i dont recommend you to build something from scratch.
I just wrote something close to dkons's answer I will add it anyway. Just using fluent interface nothing more.
public class Query<T> where T : class
{
private Dictionary<string, string> _dictionary;
public Query()
{
_dictionary = new Dictionary<string, string>();
}
public Query<T> Eq(Expression<Func<T, string>> property)
{
AddOperator("Eq", property.Name);
return this;
}
public Query<T> StartsWith(Expression<Func<T, string>> property)
{
AddOperator("Sw", property.Name);
return this;
}
public Query<T> Like(Expression<Func<T, string>> property)
{
AddOperator("Like", property.Name);
return this;
}
private void AddOperator(string opName, string prop)
{
_dictionary.Add(opName,prop);
}
public void Run(T t )
{
//Extract props of T by reflection and Build query
}
}
Lets say you have a model like
class Model
{
public string Surname{ get; set; }
public string Name{ get; set; }
}
You can use this as :
static void Main(string[] args)
{
Model m = new Model() {Name = "n", Surname = "s"};
var q = new Query<Model>();
q.Eq(x => x.Name).Like(x=>x.Surname).Run(m);
}

Categories