Iam have run into a little problem with a partial projection.
I have an method on our listboxes so that we can easily load entities into them, however loading full entities is of course not an option.
Meaning i have to do some projection, i have done this here where you can choose which Property to have as the DisplayValue.
Using the code looks like this:
dropCompany.LoadEntityList(Customer.Company, x => x.CompanyName, x => x.IsCompany);
Part of the implementation looks like this:
public void LoadEntityList<T>(T selectedEntity,
System.Linq.Expressions.Expression<Func<T, string>> selector,
System.Linq.Expressions.Expression<Func<T,bool>> where = null)
where T : FlexyBook.Infrastructure.Entity
{
var wCollection = new ObjectWrapperCollection<object>();
IQueryable<T> query = FlexyBook.Repository.DomainService<T>.GetDomainService().GetAll();
if (where != null)
query = query.Where(where);
foreach (var item in query.Select(x =>
new ObjectWrapper<object>()
{
Value = x.ID,
Text = selector.Compile()(x)
})
.ToList().OrderBy(x => x.Text))
{
wCollection.List.Add(item);
}
}
The problem is that this results in the full entity being loaded from the database and then projected.
The issue is also very obvious it is this line "selector.Compile()(x)".
My problem is i have no idea how to allow this partial select to carry on into NHibernate as a projection.
I obviously need the ID selector for identification of the entities and i dont want to change the way the method is called.
Is there anyway to solve this ?
One way do to perform the partial select would be to change your selector expression to be of type:
System.Linq.Expressions.Expression<Func<T, ObjectWrapper<object>>>
Then you would call it like:
dropCompany.LoadEntityList(Customer.Company, x => new ObjectWrapper<object>() { Value = x.ID, Text = x.CompanyName, x => x.IsCompany);
The implementation:
public void LoadEntityList<T>(T selectedEntity, System.Linq.Expressions.Expression<Func<T, ObjectWrapper<object>>> selector,System.Linq.Expressions.Expression<Func<T,bool>> where = null) where T : FlexyBook.Infrastructure.Entity
{
var wCollection = new ObjectWrapperCollection<object>();
IQueryable<T> query = FlexyBook.Repository.DomainService<T>.GetDomainService().GetAll();
if (where != null)
query = query.Where(where);
foreach (var item in query.Select(selector).ToList().OrderBy(x => x.Text))
{
wCollection.List.Add(item);
}
}
Related
I have AutoMapper configured in my .NET 5.0 project with a map between an entity (Setting) and its DTO (SettingByProfileDto). The said entity has a child collection of another entity (SettingValue) (one to many). The child collection (SettingValues) of the first entity is mapped to a single item (another DTO : SettingValueDto) inside the DTO because I only need a specific item from this list.
For the mapping configuration, I use the following lines :
int profileId = default;
profile.CreateMap<Setting, SettingByProfileDto>()
.ForMember(dto => dto.SettingValue, opt =>
{
opt.MapFrom(ss =>
ss.SettingValues
.FirstOrDefault(ssv => ssv.ProfileId == profileId)
);
});
When I want to retrieve the first entity I use the AutoMapper ProjectTo method in order to only request the fields the DTO has. I give to the ProjectTo method the value of the profileId parameter so the mapping can know on which Id the filter has to be done :
// ...
.Where(ss => ss.Id == request.Id)
.ProjectTo<SettingByProfileDto>(
_mapper.ConfigurationProvider,
new { profileId = request.ProfileId },
dest => dest.SettingValue
)
// ...
The result of the query and the mapping are both correct. However, the query sent to the database to fetch the results seems to be poorly optimized.
Here is the resulting query :
SELECT [s8].[Description], [s8].[DisplayName], [s8].[Id], [s8].[Name], CASE
WHEN (
SELECT TOP(1) [s].[Id]
FROM [SettingValue] AS [s]
WHERE ([s8].[Id] = [s].[SettingId]) AND ([s].[ProfileId] = #__profileId_1)) IS NULL THEN CAST(1 AS bit)
ELSE CAST(0 AS bit)
END, (
SELECT TOP(1) [s2].[Id]
FROM [SettingValue] AS [s2]
WHERE ([s8].[Id] = [s2].[SettingId]) AND ([s2].[ProfileId] = #__profileId_1)), (
SELECT TOP(1) [s3].[ProfileId]
FROM [SettingValue] AS [s3]
WHERE ([s8].[Id] = [s3].[SettingId]) AND ([s3].[ProfileId] = #__profileId_1)), (
SELECT TOP(1) [s4].[SettingId]
FROM [SettingValue] AS [s4]
WHERE ([s8].[Id] = [s4].[SettingId]) AND ([s4].[ProfileId] = #__profileId_1)), (
SELECT TOP(1) [s7].[Value]
FROM [SettingValue] AS [s7]
WHERE ([s8].[Id] = [s7].[SettingId]) AND ([s7].[ProfileId] = #__profileId_1))
FROM [Setting] AS [s8]
WHERE [s8].[Id] = #__request_Id_0
ORDER BY (SELECT 1)
OFFSET #__p_2 ROWS FETCH NEXT #__p_3 ROWS ONLY
And here is the query expression that is generated by AutoMapper and then transformed to SQL :
DbSet<Setting>()
.AsNoTracking()
.Where(ss => ss.Id == __request_Id_0)
.Select(dtoSetting => new Object_1800281414___SettingValue_Description_DisplayName_Id_Name{
__SettingValue = dtoSetting.SettingValues
.FirstOrDefault(ssv => ssv.ProfileId == __profileId_1),
Description = dtoSetting.Description,
DisplayName = dtoSetting.DisplayName,
Id = dtoSetting.Id,
Name = dtoSetting.Name,
}
)
.Select(dtoLet => new SettingByProfileDto{
Description = dtoLet.Description,
DisplayName = dtoLet.DisplayName,
Id = dtoLet.Id,
Name = dtoLet.Name,
SettingValue = dtoLet.__SettingValue == null ? null : new SettingValueDto{
Id = dtoLet.__SettingValue.Id,
ProfileId = dtoLet.__SettingValue.ProfileId,
SettingId = dtoLet.__SettingValue.SettingId,
Value = dtoLet.__SettingValue.Value
}
}
)
.Skip(__p_2)
.Take(__p_3)
I tried to replace the FirstOrDefault clause in the mapping configuration by a Where (with the same condition) and in this case, the query generated will use a LEFT JOIN which avoid repeating one WHERE per field. However, with this way, I can't map the child collection to a single item but only another collection (of dto).
My questions are the following :
Is there a better (more efficient?) way to achieve what I want (only keep one item of the child collection) ?
Is the above query considered optimized ? If so, I would be able to continue with my current way of doing
Thanks for your help !
AutoMapper projections work by injecting Select calls with generated mapping selectors, or just selectors in the query expression tree where the destination type doesn't match the source type.
The problem is where are these injected. For instance, in your example it generates something like this (pseudo code, MapTo just marks the mapping injection point)
SettingValue = source.SettingValues
.FirstOrDefault(ssv => ssv.ProfileId == profileId)
.MapTo<SettingByProfileDto>()
Here with the predicate version of FirstOrDefault it has no choice (well, relatively, keep reading), but even if you rewrite it to the equivalent Where(predicate) + FirstOrDefault() chain, it still injects the mapping at the end
SettingValue = source.SettingValues
.Where(ssv => ssv.ProfileId == profileId)
.FirstOrDefault()
.MapTo<SettingByProfileDto>()
which is too late, and that's why EF Core generates inefficient query.
Now, one may consider this to be EF Core query translation defect. But if the mapping is injected before the FirstOrDefault() call
SettingValue = source.SettingValues
.Where(ssv => ssv.ProfileId == profileId)
.MapTo<SettingByProfileDto>()
.FirstOrDefault()
then Core produces optimal translation.
I didn't find a way to force AM to do that, and also it is good all that to happen transparently. So I wrote a little custom extension. What it does is to to plug into AutoMapper pipeline and transform appropriately the following Enumerable extension methods (both predicate and non predicate overloads) - First, FirstOrDefault, Last, LastOrDefault, Single, SingleOrDefault.
Here is the source code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using AutoMapper.Internal;
using AutoMapper.QueryableExtensions;
using AutoMapper.QueryableExtensions.Impl;
namespace AutoMapper
{
public static class SingeResultQueryMapper
{
public static IMapperConfigurationExpression AddSingleResultQueryMapping(this IMapperConfigurationExpression config)
{
config.Advanced.QueryableBinders.Insert(0, new Binder());
config.Advanced.QueryableResultConverters.Insert(0, new ResultConverter());
return config;
}
static string[] TargetMethodNames => new[]
{
nameof(Enumerable.First),
nameof(Enumerable.FirstOrDefault),
nameof(Enumerable.Last),
nameof(Enumerable.LastOrDefault),
nameof(Enumerable.Single),
nameof(Enumerable.SingleOrDefault),
};
static HashSet<MethodInfo> TargetMethods { get; } =
(from method in typeof(Enumerable).GetTypeInfo().DeclaredMethods
join name in TargetMethodNames
on method.Name equals name
select method).ToHashSet();
static bool IsTarget(IMemberMap propertyMap) =>
propertyMap.SourceType != propertyMap.DestinationType &&
propertyMap.ProjectToCustomSource is null &&
propertyMap.CustomMapExpression?.Body is MethodCallExpression call &&
call.Method.IsGenericMethod &&
TargetMethods.Contains(call.Method.GetGenericMethodDefinition());
class ResultConverter : IExpressionResultConverter
{
public bool CanGetExpressionResolutionResult(ExpressionResolutionResult expressionResolutionResult, IMemberMap propertyMap)
=> IsTarget(propertyMap);
public ExpressionResolutionResult GetExpressionResolutionResult(ExpressionResolutionResult expressionResolutionResult, IMemberMap propertyMap, LetPropertyMaps letPropertyMaps)
=> new(propertyMap.CustomMapExpression.ReplaceParameters(propertyMap.CheckCustomSource(expressionResolutionResult, letPropertyMaps)));
}
class Binder : IExpressionBinder
{
public bool IsMatch(PropertyMap propertyMap, TypeMap propertyTypeMap, ExpressionResolutionResult result)
=> IsTarget(propertyMap);
public MemberAssignment Build(IConfigurationProvider configuration, PropertyMap propertyMap, TypeMap propertyTypeMap, ExpressionRequest request, ExpressionResolutionResult result, IDictionary<ExpressionRequest, int> typePairCount, LetPropertyMaps letPropertyMaps)
{
var call = (MethodCallExpression)result.ResolutionExpression;
var selectors = configuration.ExpressionBuilder.CreateMapExpression(
new(propertyMap.SourceType, propertyMap.DestinationType, request.MembersToExpand, request),
typePairCount, letPropertyMaps.New());
if (selectors == null) return null;
var query = call.Arguments[0];
var method = call.Method.GetGenericMethodDefinition();
if (call.Arguments.Count > 1)
{
// Predicate version of the method
// Convert query.Method(predicate) to query.Where(predicate).Method()
query = Expression.Call(typeof(Enumerable), nameof(Enumerable.Where), new[] { propertyMap.SourceType }, query, call.Arguments[1]);
method = TargetMethods.First(m => m.Name == method.Name && m.GetParameters().Length == 1);
}
method = method.MakeGenericMethod(propertyMap.DestinationType);
foreach (var selector in selectors)
query = Expression.Call(typeof(Enumerable), nameof(Enumerable.Select), new[] { selector.Parameters[0].Type, selector.ReturnType }, query, selector);
call = Expression.Call(method, query);
return Expression.Bind(propertyMap.DestinationMember, call);
}
}
}
}
Just put it in a code file inside the project where you configure AutoMapper, and then enable it with the provided configuration helper extension method (similar to Expression Mapping AutoMapper extension) like this
var mapper = new MapperConfiguration(cfg => {
cfg.AddSingleResultQueryMapping();
// The rest ...
}).CreateMapper();
or with DI
services.AddAutoMapper(cfg => {
cfg.AddSingleResultQueryMapping();
}, /* assemblies with profiles */);
And that's all. Now your original DTO, mapping and ProjectTo will produce optimal SQL query translation (single LEFT OUTER JOIN).
I have a question for you regarding the creation of a Dynamic select query in Entity Framework.
I already have a dynamic query for the select based on rights etc. But for each table I get 30+ fields that I have to parse via the .GetType().GetProperties().
Its complex and its quite costly in terms of resource due to the amount of data we have.
I have a service that tells me which fields I should select for each table. I would like to find a way to transform that into the query but I can't find something that is really dynamic.
That is not dynamic but manual:
using (var context = new StackOverflowContext())
{
var posts = context.Posts
.Where(p => p.Tags == "<sql-server>")
.Select(p => new {p.Id, p.Title});
// Do something;
}
I need to say, select only those fields but only the fields with this names.
I have the field list in a list of string but that could be changed.
Could you please help me?
Here is a .Net Fiddle code (made by msbendtsen) that allows to dynamically select columns (properties).
https://dotnetfiddle.net/3IMR1r
The sample is written for linq to objects but it should work with entity frameworks.
The key section is:
internal static IQueryable SelectProperties<T>(this IQueryable<T> queryable, IEnumerable<string> propertyNames)
{
// get propertyinfo's from original type
var properties = typeof(T).GetProperties().Where(p => propertyNames.Contains(p.Name));
// Create the x => expression
var lambdaParameterExpression = Expression.Parameter(typeof(T));
// Create the x.<propertyName>'s
var propertyExpressions = properties.Select(p => Expression.Property(lambdaParameterExpression, p));
// Creating anonymous type using dictionary of property name and property type
var anonymousType = AnonymousTypeUtils.CreateType(properties.ToDictionary(p => p.Name, p => p.PropertyType));
var anonymousTypeConstructor = anonymousType.GetConstructors().Single();
var anonymousTypeMembers = anonymousType.GetProperties().Cast<MemberInfo>().ToArray();
// Create the new {} expression using
var anonymousTypeNewExpression = Expression.New(anonymousTypeConstructor, propertyExpressions, anonymousTypeMembers);
var selectLambdaMethod = GetExpressionLambdaMethod(lambdaParameterExpression.Type, anonymousType);
var selectBodyLambdaParameters = new object[] { anonymousTypeNewExpression, new[] { lambdaParameterExpression } };
var selectBodyLambdaExpression = (LambdaExpression)selectLambdaMethod.Invoke(null, selectBodyLambdaParameters);
var selectMethod = GetQueryableSelectMethod(typeof(T), anonymousType);
var selectedQueryable = selectMethod.Invoke(null, new object[] { queryable, selectBodyLambdaExpression }) as IQueryable;
return selectedQueryable;
}
So currently i am writing a specific SQL function to get a specific row from a specific table.
However, I have dozens of tables, and noticed that I am writing these same 'get row' repository functions each time I make a new table.
Is it possible to write a generic function that works for every table, in this case to get a specific row?
Current (Example)
public Purchase GetPurchase(long purchaseId)
{
using (var db = new DbContext(_connStringKey))
{
var result = (from p in db.Purchases
where p.PurchaseId.Equals(purchaseId)
select p).FirstOrDefault();
return result;
}
}
Generic Example (To give you an idea)
public Object GenericGet (string tableName, string rowName, long rowId)
{
using (var db = new DbContext(_connStringKey))
{
var result = (from p in db.tableName
where p.rowName.Equals(rowId)
select p).FirstOrDefault();
return result;
}
}
You can do it using reflection but it is not a good approach. Instead of this, you could try something using the generics aspects of the language, and make sure what you want, for sample:
public T Get<T>(Expression<Func<T, bool>> filter)
where T : class
{
T result;
using (var db = new DbContext(_connStringKey))
{
result = db.Set<T>().FirstOrDefault(filter);
}
return result;
}
Remember that the T must be a reference type, so, define a constraint for class
Then, you could try this:
var product = Get<Product>(p => p.Name == "Something");
var supplier = Get<Supplier>(p => p.Sector == "Manufacturing");
I have a query that looks like this:
var caseList = (from x in context.Cases
where allowedCaseIds.Contains(x => x.CaseId)
select new Case {
CaseId = x.CaseId,
NotifierId = x.NotifierId,
Notifier = x.NotifierId.HasValue ? new Notifier { Name = x.Notifier.Name } : null // This line throws exception
}).ToList();
A Case class can have 0..1 Notifier
The query above will result in the following System.NotSupportedException:
Unable to create a null constant value of type 'Models.Notifier'. Only entity types, enumeration types or primitive types are supported
in this context.
At the moment the only workaround I found is to loop the query result afterwards and manually populate Notifierlike this:
foreach (var c in caseList.Where(x => x.NotifierId.HasValue)
{
c.Notifier = (from x in context.Notifiers
where x.CaseId == c.CaseId
select new Notifier {
Name = x.Name
}).FirstOrDefault();
}
But I really don't want to do this because in my actual scenario it would generate hundreds of additional queries.
Is there any possible solution for a situation like this?.
I think you need to do that in two steps. First you can fetch only the data what you need with an anonymous type in a single query:
var caseList = (from x in context.Cases
where allowedCaseIds.Contains(x => x.CaseId)
select new {
CaseId = x.CaseId,
NotifierId = x.NotifierId,
NotifierName = x.Notifier.Name
}).ToList();
After that, you can work in memory:
List<Case> cases = new List<Case>();
foreach (var c in caseList)
{
var case = new Case();
case.CaseId = c.CaseId;
case.NotifierId = c.NotifierId;
case.NotifierName = c.NotifierId.HasValue ? c.NotifierName : null;
cases.Add(case);
}
You could try writing your query as a chain of function calls rather than a query expression, then put an .AsEnumerable() in between:
var caseList = context.Clases
.Where(x => allowedCaseIds.Contains(x.CaseId))
.AsEnumerable() // Switch context
.Select(x => new Case() {
CaseId = x.CaseId,
NotifierId = x.NotifierId,
Notifier = x.NotifierId.HasValue
? new Notifier() { Name = x.Notifier.Name }
: null
})
.ToList();
This will cause EF to generate an SQL query only up to the point where you put the .AsEnumerable(), further down the road, LINQ to Objects will do all the work. This has the advantage that you can use code that cannot be translated to SQL and should not require a lot of changes to your existing code base (unless you're using a lot of let expressions...)
I understand that LINQ can't use properties that are not mapped to a database column, though I don't understand why one LINQ statement works inside a non static method but I get this error when attempting within one.
Here's my working method:
private TemplatesAPIContext db = new TemplatesAPIContext();
// GET api/Template
public IQueryable<TemplateDto> GetTemplates()
{
return db.TemplateModels.Include(t => t.Categories).Select(
x => new TemplateDto
{
TemplateID = x.TemplateID,
Name = x.Name,
HTMLShowcase = x.HTMLShowcase,
ShortDescription = x.ShortDescription,
CreationDate = x.CreationDate,
Downloads = x.Downloads,
Tags = x.Tags,
Categories = db.CategoryModels
.Where(c => x.Categories.Where(a => a.TemplateID == x.TemplateID)
.Select(a => a.CategoryID).Contains(c.CategoryID))
}
);
}
I don't want to repeat myself with this complex building of a DTO (I actually still need to add some other relationships still to it and it will get much more complex) and type this out on every method in the controller so I wanted to make a lambda expression and pass it to the methods.
So I did this:
private static readonly Expression<Func<TemplateModel, TemplateDto>> AsTemplateDto =
x => new TemplateDto
{
TemplateID = x.TemplateID,
Name = x.Name,
HTMLShowcase = x.HTMLShowcase,
ShortDescription = x.ShortDescription,
CreationDate = x.CreationDate,
Downloads = x.Downloads,
Tags = x.Tags,
Categories = new TemplatesAPIContext().CategoryModels
.Where(c => x.Categories.Where(a => a.TemplateID == x.TemplateID)
.Select(a => a.CategoryID).Contains(c.CategoryID))
};
In the hopes of calling:
// GET api/Template
public IQueryable<TemplateDto> GetTemplates()
{
return db.TemplateModels.Include(t => t.Categories).Select(AsTemplateDto);
}
But this returns this error, which doesn't make sense to me since its the exact same query, only difference being that I need to instantiate the dbContext in the lambda since I can't use the one instantiated in the controller as the lambda expression is static.
Error
The specified type member 'CategoryModels' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.
It's important that the same context be used within the query as the one that's making the query, for the query provider to understand what you're trying to do. So all you need is a way of making a copy of that expression that's specific to a given context, which isn't that hard, you've done almost all of the work.
//TODO rename method as appropriate
private static Expression<Func<TemplateModel, TemplateDto>>
CreateTemplateDTO(TemplatesAPIContext context)
{
return x => new TemplateDto
{
TemplateID = x.TemplateID,
Name = x.Name,
HTMLShowcase = x.HTMLShowcase,
ShortDescription = x.ShortDescription,
CreationDate = x.CreationDate,
Downloads = x.Downloads,
Tags = x.Tags,
Categories = context.CategoryModels
.Where(c => x.Categories.Where(a => a.TemplateID == x.TemplateID)
.Select(a => a.CategoryID).Contains(c.CategoryID))
};
}
Now you can write:
public IQueryable<TemplateDto> GetTemplates()
{
return db.TemplateModels.Include(t => t.Categories)
.Select(CreateTemplateDTO(db));
}
Your first method is a simple expression tree that contains only simple operations in tree nodes (like assign A to B) thus it can easily be compiled into SQL query.
The other method contains instantiation of TemplatesAPIContext. It's not possible for database query.