I have a code that fetching a table. Here's my sample code:
public IQueryable<MyItem> MyItems(){
return context.MyItem;
}
Here are the sample properties of MyItem
public int Id {get;set;}
public byte[] Image {get;set;}
public string Name {get;set;}
Since, byte[] can have multiple characters, I don't want to include then in searching because it will take so long if I have a records like 10,000 items.
Typically, I would Select like this:
public IQueryable<MyItem> MyItems(){
return context.MyItem.Select(item=>new MyItem{
Id=item.Id,
Name=item.Name
});
}
This is okay for few properties, but what I have a 10-20 properties, it would be hassle to write them one by one.
Is there any way like, I just Except the property Image in lambda for shorter code?
Create your own extension method SelectExcept:
public static IQueryable<T> SelectExcept<T, TKey>(this IQueryable<T> sequence,
Expression<Func<T, TKey>> excluder)
{
List<string> excludedProperties = new List<string>();
if (excluder.Body is MemberExpression memberExpression)
{
excludedProperties.Add(memberExpression.Member.Name);
}
else if (excluder.Body is NewExpression anonymousExpression)
{
excludedProperties.AddRange(anonymousExpression.Members.Select(m => m.Name));
}
var includedProperties = typeof(T).GetProperties()
.Where(p => !excludedProperties.Contains(p.Name));
return sequence.Select(x => Selector(x, includedProperties));
}
private static T Selector<T>(T obj, IEnumerable<PropertyInfo> properties)
{
var instance = Activator.CreateInstance<T>();
foreach (var property in properties)
property.SetValue(instance, property.GetValue(obj), null);
return instance;
}
Usage:
var result = context.MyItem.SelectExcept(x => x.Image);
You can exclude more than one property:
var result = context.MyItem.SelectExcept(x => new { x.Image, x.Name });
You can use AutoMapper for this. Just create a DTO class with all the properties excluding those which you don't want to query:
public class MyItemDTO
{
public int Id { get; set; }
public string Name { get; set; }
}
Then add some mapping:
Mapper.Initialize(cfg =>
{
cfg.CreateMap<MyItem, MyItemDTO>();
});
Now you can query your entities like this:
context.MyItem
.OrderBy(m => m.Name)
.ProjectTo<MyItemDTO>();
This extension is available in AutoMapper.QueryableExtensions.
You can also create a DTO class with all the properties from an entity and then ignore some of them in mapping:
cfg.CreateMap<MyItem, MyItemDTO>().ForMember(d => d.Image, m => m.Ignore());
If you use EF Core you can instantiate entity class inside LINQ query so it's possible to map entity to itself and ignore some properties from it without creating additional DTO classes:
cfg.CreateMap<MyItem, MyItem>().ForMember(d => d.Image, m => m.Ignore());
Then you just use ProjectTo with the entity type which excludes unwanted properties from SQL query:
context.MyItem
.OrderBy(m => m.Name)
.ProjectTo<MyItem>();
Note that you may also need to create mappings for each navigation property of an entity.
Related
I am trying to build generic method which would look simething like this:
public static IQueryable<TModel> IncludeByUserCondition<TModel, TIncludable>(this IQueryable<TModel> query, Func<TModel, IQueryable<TIncludable>> includes, List<int> userIDs)
where TModel : class
where TIncludable: class
{
Expression<Func<TModel, object>> result = x => includes(x);
if(typeof(ASqlBase).IsAssignableFrom(typeof(TIncludable)))
{
result = x =>
includes(x)
.Select(prop => prop as ASqlBase)
.Where(prop =>
prop.DeleteDate == null
)
.Where(prop =>
userIDs != null && userIDs.Count > 0 ? userIDs.Contains(prop.IdentityUnitID) : true
)
.Select(prop => prop as TIncludable);
}
query = query.Include(result);
return query;
}
This method would allow me to centrally check if user can read navigation property's value and, if so, include it in result. My applications read rights are conceived in hierarchical way: logged user can read his records and records of all users he had added to the system. Because of that, I cannot determine all visible records in compile-time and, thus, cannot use different database contexts for different groups of users. Also, since this is only one of many ways for filtering data, unfortunately I cannot make use of Global Filters.
I am trying to call the above method like this:
qry = qry.IncludeByUserCondition<AllocatedFund, AllocatedFundDetailPaymentMade>(p => p.AllocatedFundDetailPaymentsMade.AsQueryable(), allowedUserIDs);
However, when I try to invoke it in run-time, I get the following exception:
The expression 'Invoke(__includes_0, x).Select(prop => (prop As ASqlBase)).Where(prop => ((prop.DeleteDate == null))).Where(prop => True).Select(prop => (prop As AllocatedFundDetailPaymentMade))' is invalid inside an 'Include' operation, since it does not represent a property access: 't => t.MyProperty'. To target navigations declared on derived types, use casting ('t => ((Derived)t).MyProperty') or the 'as' operator ('t => (t as Derived).MyProperty'). Collection navigation access can be filtered by composing Where, OrderBy(Descending), ThenBy(Descending), Skip or Take operations.
On another hand, when I try to run query manually, everything works fine:
qry = qry.Include(p =>
p.AllocatedFundDetailPaymentsMade
.AsQueryable()
.Where(prop =>
prop.DeleteDate == null
)
.Where(prop =>
userIDs != null && userIDs.Count > 0 ? userIDs.Contains(prop.IdentityUnitID) : true
)
Since I have more than just one navigation property to include in this manner (and I have to perform query in similar manner for all 30+ other models I use in my application), I wouldn't want to manually write those where clauses in every query.
Does anybody know the solution for this problem? Any help would be kindly appreciated.
EDIT:
ASqlBase is just base, abstract class from which some other models interit (although not all of them - ie. User model does not inherit from ASqlBase).
ASqlBase looks like this:
public abstract class ASqlBase
{
[Key]
public int ID { get; set; }
[Required]
public int UserID { get; set; }
[ForeignKey("UserID")]
public virtual User User { get; set; }
public DateTime? DeleteDate { get; set; }
}
I plan to use that function to get data and then display it in report. I'm giving example for Person, then the method call would look something like this:
var qry = dbContext.Person.IncludeByUserCondition<Person, Athlete>(p => p.Athletes.AsQueryable(), athleteAllowedUserIDs);
qry = qry.IncludeByUserCondition<Person, Employee>(p => p.Employees.AsQueryable(), employeeAllowedUserIDs);
qry = qry.IncludeByUserCondition<Person, Student>(p => p.Students.AsQueryable(), studentAllowedUserIDs);
Person model looks something like this:
public class Person : ASqlBase
{
...
public virtual ICollection<Athlete> Athletes { get; set; }
public virtual ICollection<Employee> Employees { get; set; }
public virtual ICollection<Student> Students { get; set; }
}
All of the above models: Athlete, Employee and Student inherit from ASqlBase
EDIT 2:
Sorry for bad method naming, method should be called IncludeByUserCondition, not IncludeMultiple (as it was named before).
A little bit simplified usage. You do not need to call AsQueryable() and explicitly specify generic parameters:
var qry = dbContext.Person.IncludeByUserCondition(p => p.Athletes, athleteAllowedUserIDs);
qry = qry.IncludeByUserCondition(p => p.Employees, employeeAllowedUserIDs);
qry = qry.IncludeByUserCondition(p => p.Students, studentAllowedUserIDs);
And realization:
public static class IncludeExtensions
{
public static IQueryable<TModel> IncludeByUserCondition<TModel, TRelated>(this IQueryable<TModel> query,
Expression<Func<TModel, IEnumerable<TRelated>>> collectionProp, List<int> userIDs)
where TModel : class
where TRelated : ASqlBase
{
var relatedParam = Expression.Parameter(typeof(TRelated), "r");
// r.DeleteDate == null
var filterPredicate = (Expression)Expression.Equal(
Expression.PropertyOrField(relatedParam, nameof(ASqlBase.DeleteDate)),
Expression.Constant(null, typeof(DateTime?)));
if (userIDs?.Count > 0)
{
// r.DeleteDate == null && userIDs.Contains(r.UserID)
filterPredicate = Expression.AndAlso(filterPredicate,
Expression.Call(typeof(Enumerable), nameof(Enumerable.Contains), new[] { typeof(int) },
Expression.Constant(userIDs),
Expression.PropertyOrField(relatedParam, nameof(ASqlBase.UserID))));
}
// r => r.DeleteDate == null && userIDs.Contains(r.UserID)
var filterLambda = Expression.Lambda(filterPredicate, relatedParam);
// p => p.Navigation.Where(r => r.DeleteDate == null && userIDs.Contains(r.UserID))
var transformedProp = Expression.Lambda(Expression.Call(typeof(Enumerable), nameof(Enumerable.Where),
new[] { typeof(TRelated) }, collectionProp.Body, filterLambda), collectionProp.Parameters);
// query.Include(p => p.Navigation.Where(r => r.DeleteDate == null && userIDs.Contains(r.UserID)))
var includeExpression = Expression.Call(typeof(EntityFrameworkQueryableExtensions),
nameof(EntityFrameworkQueryableExtensions.Include),
new[] { typeof(TModel), typeof(IEnumerable<TRelated>) },
query.Expression,
Expression.Quote(transformedProp));
// instantiate new IQueryable<TModel>
var resultQuery = query.Provider.CreateQuery<TModel>(includeExpression);
return resultQuery;
}
}
I'm trying to project from my Order model to my OrderDTO model. Order has an enum. The problem is that projection doesn't work if I try to to get the Description attribute from the Enum. Here it's my code:
OrderStatus.cs:
public enum OrderStatus {
[Description("Paid")]
Paid,
[Description("Processing")]
InProcess,
[Description("Delivered")]
Sent
}
Order.cs:
public class Order {
public int Id { get; set; }
public List<OrderLine> OrderLines { get; set; }
public OrderStatus Status { get; set; }
}
OrderDTO.cs:
public class OrderDTO {
public int Id { get; set; }
public List<OrderLineDTO> OrderLines { get; set; }
public string Status { get; set; }
}
With this following configuration in my AutoMapper.cs:
cfg.CreateMap<Order, OrderDTO>().ForMember(
dest => dest.Status,
opt => opt.MapFrom(src => src.Status.ToString())
);
Projection works, but I get an OrderDTO object like this:
- Id: 1
- OrderLines: List<OrderLines>
- Sent //I want "Delivered"!
I don't want Status property to be "Sent", I want it to be as its associated Description attribute, in this case, "Delivered".
I have tried two solutions and none of them have worked:
Using ResolveUsing AutoMapper function as explained here, but, as it's stated here:
ResolveUsing is not supported for projections, see the wiki on LINQ projections for supported operations.
Using a static method to return the Description attribute in String by Reflection.
cfg.CreateMap<Order, OrderDTO>().ForMember(
dest => dest.Status,
opt => opt.MapFrom(src => EnumHelper<OrderStatus>.GetEnumDescription(src.Status.ToString()))
);
But this gives me the following error:
LINQ to Entities does not recognize the method 'System.String
GetEnumDescription(System.String)' method, and this method cannot be
translated into a store expression.
Then, how can I achieve this?
You can add an extension method like this one (borrowed the logic from this post):
public static class ExtensionMethods
{
static public string GetDescription(this OrderStatus This)
{
var type = typeof(OrderStatus);
var memInfo = type.GetMember(This.ToString());
var attributes = memInfo[0].GetCustomAttributes(typeof(DescriptionAttribute), false);
return ((DescriptionAttribute)attributes[0]).Description;
}
}
Then access it in your map:
cfg =>
{
cfg.CreateMap<Order, OrderDTO>()
.ForMember
(
dest => dest.Status,
opt => opt.MapFrom
(
src => src.Status.GetDescription()
)
);
}
This results in what you are asking for:
Console.WriteLine(dto.Status); //"Delivered", not "sent"
See a working example on DotNetFiddle
Edit1: Don’t think you can add a local look up function like that to LINQ to entities. It would only work in LINQ to objects. The solution you should pursue perhaps is a domain table in the database that allows you to join to it and return the column that you want so that you don’t have to do anything with AutoMapper.
You can achieve an expression based enum description mapping by building up an expression that evaluates to a string containing a condition statement (such as switch/if/case depending on how the provider implements it) with the enum descriptions as the results.
Because the enum descriptions can be extracted ahead of time we can obtain them and use them as a constant for the result of the condition expression.
Note: I've used the above extension method GetDescription() but you can use whatever flavour of attribute extraction you need.
public static Expression<Func<TEntity, string>> CreateEnumDescriptionExpression<TEntity, TEnum>(
Expression<Func<TEntity, TEnum>> propertyExpression)
where TEntity : class
where TEnum : struct
{
// Get all of the possible enum values for the given enum type
var enumValues = Enum.GetValues(typeof(TEnum)).Cast<Enum>();
// Build up a condition expression based on each enum value
Expression resultExpression = Expression.Constant(string.Empty);
foreach (var enumValue in enumValues)
{
resultExpression = Expression.Condition(
Expression.Equal(propertyExpression.Body, Expression.Constant(enumValue)),
// GetDescription() can be replaced with whatever extension
// to get you the needed enum attribute.
Expression.Constant(enumValue.GetDescription()),
resultExpression);
}
return Expression.Lambda<Func<TEntity, string>>(
resultExpression, propertyExpression.Parameters);
}
Then your Automapper mapping becomes:
cfg.CreateMap<Order, OrderDTO>().ForMember(
dest => dest.Status, opts => opts.MapFrom(
CreateEnumDescriptionExpression<Order, OrderStatus>(src => src.Status)));
When this is evaluated at runtime using Entity Framework with SQL server provider, the resulting SQL will be something like:
SELECT
-- various fields such as Id
CASE WHEN (2 = [Extent1].[Status]) THEN N'Delivered'
WHEN (1 = [Extent1].[Status]) THEN N'Processing'
WHEN (0 = [Extent1].[Status]) THEN N'Paid' ELSE N'' END AS [C1]
FROM [Orders] as [Extent1]
This should also work for other Entity Framework DB providers.
I'm currently using the simple model below. It's pretty straightforward: we have resources and they can be Room, EmptyOffice (...) or Service.
Room and EmptyOffice can have a capacity, but NOT Service.
public abstract class Resource : Entity
{
public string Name { get; set; }
}
public class Room : Resource
{
public int Capacity { get; set; }
}
public class EmptyOffice : Resource
{
public int Capacity { get; set; }
}
public class Service : Resource
{ }
To get the data from my SQL view, I use the mappings:
builder.Entity<Resource>(m =>
{
m.ToTable("resource", "facility");
m.HasKey(x => x.Id);
m.Property(x => x.Id)
.HasColumnName("ResourceId");
m.Property(x => x.Type)
.HasColumnName("ResourceTypeId");
m.HasDiscriminator(x => x.Type)
.HasValue<Room>(ResourceType.Room)
.HasValue<EmptyOffice>(ResourceType.EmptyOffice)
.HasValue<Service>(ResourceType.Service);
});
builder.Entity<Room>();
builder.Entity<EmptyOffice>();
builder.Entity<Service>();
When I run my code, EF Core throws the following exception:
System.Data.SqlClient.SqlException: 'Invalid column name 'Room_Capacity'.'
If I rename the Capacity property to Room_Capacity, it works but it's horrible.
How can I force EF Core 2.0 to target the capacity property for each of my child entities?
Thank you
Sebastien
This worked for me:
builder.Entity<Room>().Property(a => a.Capacity).HasColumnName("Capacity");
builder.Entity<EmptyRoom>().Property(a => a.Capacity).HasColumnName("Capacity");
You can't do that as the only inheritance pattern available in EF Core is table per class hierarchy. If you go with interfaces instead of base classes, you can, but each entity will be mapped to a different table. Mark any property you want to exclude with [NotMapped], or, using the code, with Ignore.
I did in project next code to make it more generic way.
private static void FindAndConfigureBackgroundJobResultTypes(ModelBuilder modelBuilder)
{
var backgroundJobResultTypes = typeof(BackgroundJobResult).Assembly.GetTypes().Where(x => x.IsSubclassOf(typeof(BackgroundJobResult))).ToList();
var sameTypeAndNameProperties = backgroundJobResultTypes
.SelectMany(x => x.GetProperties())
.GroupBy(d => new {d.Name, d.PropertyType})
.Select(grp => new
{
PropertyType = grp.Key.PropertyType,
PropertyName = grp.Key.Name,
Count = grp.Count()
})
.Where(x => x.Count > 1).ToList();
foreach (var backgroundJobResultType in backgroundJobResultTypes)
{
//Set base type , instead of exposing this type by DbSet
modelBuilder.Entity(backgroundJobResultType).HasBaseType(typeof(BackgroundJobResult));
//Map properties with the same name and type into one column, EF Core by default will create separate column for each type, and make it really strange way.
foreach (var propertyInfo in backgroundJobResultType.GetProperties())
{
if (sameTypeAndNameProperties.Any(x => x.PropertyType == propertyInfo.PropertyType && x.PropertyName == propertyInfo.Name))
{
modelBuilder.Entity(backgroundJobResultType).Property(propertyInfo.PropertyType, propertyInfo.Name).HasColumnName(propertyInfo.Name);
}
}
}
}
I created a repository for my entity Master. In the repository, I have a Get method to get my entity by Id using Entity Core.
The method receives:
public TEntity Get(object id, params Expression<Func<TEntity, object>>[] includedRelatedEntities)
{
return GetById(IncludeEntities(DBContext.Set<TEntity>().AsQueryable(), includedRelatedEntities), id);
}
Then, when I use it in my code, I just pass to the method the id of the entity I´m looking for and and expression tree of the related entities that I need to include in the query (Expression<Func<TEntity, object>>)
An example of use is the following one:
var master = MasterRepository.Get(1, x => x.BranchOffice.Location);
In that case I´m looking for the Master with Id = 1 and I want it to include the BranchOffice related entity and the Location related to that BranchOffice.
From one to many relationships, it works fine, but for related lists, I dont know how to resolve it using an expression.
For example, if I want to include the Product entity of the list of Detail named Details related to my Master, I dont know how to express it in the expression tree.
var master = MasterRepository.Get(1, x => x.Details.Product);
Details is a list, so I cant access product as it is in the example above.
How can I express that in a Expression<Func<TEntity, object>>?
EDIT:
I´ve already tried:
var master = MasterRepository.Get(1, x => x.Details.Select(y=> y.Product));
But I´m getting the following exception:
The property expression 'x => {from Detail y in [x].Details select
[y].Product}' is not valid. The expression should represent a property
access: 't => t.MyProperty'. For more information on including related
data, see go.microsoft.com/fwlink/?LinkID=746393.'
I don't know can you change or replace IncludeEntities implementations, so maybe answer would not be helpful for you. Well, x => x.Details.Product will looks like this DbContext.Set<SomeType>().Include(x => x.Details).ThenInclude(o => o.Product) in the EF.Core.
So if you want to include multiple levels I can suggest you to build a query at runtime that will contains Include and ThenInclude. So, this query will be built from input expression looks like this x => x.Details.Select(y => y.Product). It's method that build this query:
/// <summary>
/// Takes include looks like 'x => x.Collections.Select(o => o.List.Select(p => p.Date))'
/// </summary>
public static IQueryable<T> GetQueryWithIncludes<T>(IQueryable<T> query, Expression<Func<T, object>> arg)
{
// Tiny optimization
ParameterInfo[] parameters;
var includeInfo = typeof(EntityFrameworkQueryableExtensions).GetMethods().Where(info => info.Name == "Include" &&
(parameters = info.GetParameters()).Length == 2 &&
typeof(Expression).IsAssignableFrom(parameters[1].ParameterType)).Single();
// Retrieve then include that take first param as 'IIncludableQueryable<TEntity, ICollection<TPreviousProperty>>'
var thenIncludeInfo = typeof(EntityFrameworkQueryableExtensions).GetMethods().Where(info => info.Name == "ThenInclude").ToList()[1];
// Retrieve then include that take first param as 'IIncludableQueryable<TEntity, IEnumerable<TPreviousProperty>>'
var lastThenIncludeInfo = typeof(EntityFrameworkQueryableExtensions).GetMethods().Where(info => info.Name == "ThenInclude").ToList()[0];
// Retrieve all selection from input expression
var lambda = arg as LambdaExpression;
var method = arg.Body as MethodCallExpression;
var result = new List<Expression>();
while (method != null)
{
result.Add(Expression.Lambda(method.Arguments[0], lambda.Parameters[0]));
lambda = method.Arguments[1] as LambdaExpression;
method = lambda.Body as MethodCallExpression;
}
result.Add(lambda);
// Add Include and ThenInclude to IQueryable
for (int i = 0; i < result.Count; ++i)
{
var lambdaExp = result[i] as LambdaExpression;
query = i == 0
? includeInfo.MakeGenericMethod(lambdaExp.Parameters[0].Type, lambdaExp.ReturnType).Invoke(null, new object[] { query, lambdaExp }) as IQueryable<T>
: i == result.Count - 1
? lastThenIncludeInfo.MakeGenericMethod((result[0] as LambdaExpression).Parameters[0].Type, lambdaExp.Parameters[0].Type, lambdaExp.ReturnType).Invoke(null, new object[] { query, lambdaExp }) as IQueryable<T>
: thenIncludeInfo.MakeGenericMethod((result[0] as LambdaExpression).Parameters[0].Type, lambdaExp.Parameters[0].Type, lambdaExp.ReturnType).Invoke(null, new object[] { query, lambdaExp }) as IQueryable<T>;
}
return query;
}
By the way, method takes a one expression, but it can be lightly modified, so it will takes array of expression or you can directly invoke the method from a loop for all of expressions.
Code below is just usage. I wrote a tree small classes for testing:
public class Test
{
public int Id { get; set; }
public DateTime TestDate { get; set; }
public ICollection<Level> Levels { get; set; }
}
public class Level
{
public int Id { get; set; }
public ICollection<LevelDetail> LevelDetails { get; set; }
}
public class LevelDetail
{
public int Id { get; set; }
public DateTime LevelDate { get; set; }
}
...
// These results are the same and have the same expression trees
var resultByInclude = context.Tests
.Include(o => o.Levels)
.ThenInclude(p => p.LevelDetails).ToList();
var resultBySelect = GetQueryWithIncludes(context.Tests,
o => o.Levels.Select(p => p.LevelDetails)).ToList();
I hope it will helps you.
I'm trying to reuse an EF6 extension method with an associated entity (one to many relationship). Contrived example:
public class Parent
{
public string State { get; set; }
public ICollection<Child> Children { get; set; }
}
public class Child
{
public string Value { get; set; }
public Parent Parent { get; set; }
}
public static ParentNamedScopes
{
public static IQueryable<Parent> IsReady(this IQueryable<Parent> queryable)
{
return queryable.Where(p => p.State == "Ready" || p.State == "New");
}
}
// ...
var children = db.Children
// my goal, but can't cast Parent to IQueryable<Parent>
// ------------------v
.Where(c => c.Parent.IsReady())
.Where(c => c.Value == "Foobar");
I've seen examples of using AsQueryable() on associated collections in sub queries, but that isn't an options since Parent is a single record. I'm sure I'm missing something obvious and I apologize since my google foo has not turned up the answer today.
One option would be to start your query with the Parents:
var children = db.Parents.IsReady()
.SelectMany(p => p.Children)
.Where(c => c.Value == "Foobar");
The idea that IsReady would convert an IQueryable seems a little off to me, though. If your use case gets more complex, you may need to change that to just give you an Expression<Func<Parent, bool>>, and use something like LINQKit to manipulate your query to make it reusable:
Expression<Func<Parent, bool>> parentIsReady = ParentCriteria.IsReady();
var readyParents = db.Parents.Where(parentIsReady);
var childrenWithReadyParents = db.Children.AsExpandable()
.Where(c => parentIsReady.Invoke(c.Parent))
.Where(c => c.Value == "Foobar");