I need to write a generic Join() function to perform a query between two DBSets, entities of type TEntity and parentEntities of type TParent. What this will do is get me an IQueryable of cObjectNames, each object with the PK of the entity and the name of the parent entity. both types have an IBaseEntity interface so the Id column is available but I need a way to generically specify the foreign key column in entities (fkCol in the example) and the parentEntities name column (parentNameCol).
public static IQueryable<cObjectNames> Join<TEntity, TParent>(IQueryable<TEntity> entities, IQueryable<TParent> parenEntities,
string fkCol, string parentNameCol)
where TEntity : class, IBaseEntity where TParent : class, IBaseEntity
{
IQueryable<cObjectNames> qNames = entities.Join(parenEntities, e => e.fkCol, p => p.Id, (e, p) =>
new cObjectNames() { name = p.parentNameCol, eId = e.Id });
return qNames;
}
I know it is possible to use EF to get the parent object, but I need a generic solution for several such fk relationships, where even the parent name column is not constant. And please save me the Dynamic LINQ suggestions - LINQ generic expressions are so much cooler...
The definition of cObjectNames is
public class cObjectNames
{
public int eId{ get; set; }
public string name{ get; set; }
}
and the IBaseEntity interface is:
public interface IBaseEntity
{
int Id { get; set; }
DateTimeOffset Created { get; set; }
DateTimeOffset? Lastupdated { get; set; }
DateTimeOffset? Deleted { get; set; }
}
Thanks!
Here is implementation. I hope inline comments are useful.
public static class JoinExtensions
{
public static IQueryable<cObjectNames> Join<TEntity, TParent>(this IQueryable<TEntity> entities, IQueryable<TParent> parentEntities,
string fkCol, string parentNameCol)
where TEntity : class, IBaseEntity where TParent : class, IBaseEntity
{
// we can reuse this lambda and force compiler to do that
Expression<Func<TEntity, int>> entityKeySelector = e => e.Id;
Expression<Func<TParent, int>> parentKeySelector = p => p.Id;
var entityParam = entityKeySelector.Parameters[0];
var parentParam = parentKeySelector.Parameters[0];
// Ensure types are correct
var fkColExpression = (Expression)Expression.Property(entityParam, fkCol);
if (fkColExpression.Type != typeof(int))
fkColExpression = Expression.Convert(fkColExpression, typeof(int));
// e => e.fkCol
var fkColSelector = Expression.Lambda(fkColExpression, entityParam);
// (e, p) => new cObjectNames { name = p.parentNameCol, eId = e.Id }
var resultSelector = Expression.Lambda(Expression.MemberInit(Expression.New(cObjectNamesConstrtuctor),
Expression.Bind(cObjectNamesNameProp,
Expression.Property(parentParam, parentNameCol)),
Expression.Bind(cObjectNamesIdProp,
entityKeySelector.Body)),
entityParam, parentParam);
// full Join call
var queryExpr = Expression.Call(typeof(Queryable), nameof(Queryable.Join),
new Type[] { typeof(TEntity), typeof(TParent), typeof(int), typeof(cObjectNames) },
entities.Expression,
parentEntities.Expression,
Expression.Quote(fkColSelector),
Expression.Quote(parentKeySelector),
Expression.Quote(resultSelector)
);
var qNames = entities.Provider.CreateQuery<cObjectNames>(queryExpr);
return qNames;
}
static ConstructorInfo cObjectNamesConstrtuctor = typeof(cObjectNames).GetConstructor(Type.EmptyTypes) ??
throw new InvalidOperationException();
static MemberInfo cObjectNamesNameProp = typeof(cObjectNames).GetProperty(nameof(cObjectNames.name)) ??
throw new InvalidOperationException();
static MemberInfo cObjectNamesIdProp = typeof(cObjectNames).GetProperty(nameof(cObjectNames.eId)) ??
throw new InvalidOperationException();
}
You can retrieve meta-data from your EF-Core context:
IEntityType entityType = context.Model
.FindEntityTypes(typeof(TEntity))
.FirstOrDefault();
From here you can get the navigation properties:
foreach (IReadOnlyNavigation nav in entityType.GetNavigations()) {
if (nav.IsOnDependent) {
var parentProp = nav.Inverse?.PropertyInfo;
var childProp = nav.PropertyInfo;
Type parentType = nav.TargetEntityType.ClrType;
var foreignKeyProps = nav.ForeignKey.Properties;
... etc., etc.
}
}
At least, this is a starting point. Then you will have to create expressions through Reflection. See: How do I dynamically create an Expression<Func<MyClass, bool>> predicate from Expression<Func<MyClass, string>>?.
Related
I have created an API using ASP.Net Core 6. This API used to manipulate the store products. The getAllProducts endpoint returns the list of products and I have added the sorting functionality to the getAllProducts(list) endpoint by using an extension method.
Product Entity:
public class Product
{
[Required]
[StringLength(50)]
public string Name { get; set; }
public string Description { get; set; }
public decimal Price { get; set; }
public virtual ProductCategory Category { get; set; }
}
ProductCategory entity:
public class ProductCategory
{
public string Name { get; set; }
}
Extension method:
public static IQueryable<TEntity> OrderBy<TEntity>(this IQueryable<TEntity> list, string orderByProperty, bool desc)
{
string command = desc ? "OrderByDescending" : "OrderBy";
var type = typeof(TEntity);
var property = type.GetProperty(orderByProperty);
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var orderByExpression = Expression.Lambda(propertyAccess, parameter);
var resultExpression = Expression.Call(typeof(Queryable), command, new Type[] { type, property.PropertyType },
list.Expression, Expression.Quote(orderByExpression));
return list.Provider.CreateQuery<TEntity>(resultExpression);
}
I called the extension method this way.
productsList = productsList.AsQueryable().OrderBy("Category", true).ToList();
This works fine with all product entity properties except the 'Category'. It throws an exception System.InvalidOperationException: Failed to compare two elements in the array. when I pass 'Category' as orderByProperty. I want to customize the extension method so that if an object type property passed in, sort the list by name of the passed object. (by category name in this case). I expect your help. Thank You.
Runtime does not know how to compare two instances of ProductCategory so you need to provide a way for LINQ-to-Objects to do this. For example implementing IComparable<ProductCategory>:
public class ProductCategory : IComparable<ProductCategory>
{
public string Name { get; set; }
public int CompareTo(ProductCategory? other)
{
if (ReferenceEquals(this, other)) return 0;
if (ReferenceEquals(null, other)) return 1;
return string.Compare(Name, other.Name, StringComparison.Ordinal);
}
}
Also possibly you can consider passing expression into the method:
public static IQueryable<TEntity> OrderBy<TEntity, TProp>(this IQueryable<TEntity> list, Expression<Func<TEntity, TProp>> selector, bool desc)
{
// change accordingly
}
With usage changed to:
productsList = productsList.AsQueryable()
.OrderBy(p => p.Category.Name, true)
.ToList();
I have recently found global filters, which is great because I have been tasked with implementing soft deletes in my applicaiton.
Currently I have done this:
// Query filters https://learn.microsoft.com/en-us/ef/core/querying/filters
modelBuilder.Entity<Address>().HasQueryFilter(m => !m.Deleted);
modelBuilder.Entity<Attribute>().HasQueryFilter(m => !m.Deleted);
modelBuilder.Entity<Brand>().HasQueryFilter(m => !m.Deleted);
modelBuilder.Entity<BrandAddress>().HasQueryFilter(m => !m.Deleted);
modelBuilder.Entity<BrandCategory>().HasQueryFilter(m => !m.Deleted);
modelBuilder.Entity<Category>().HasQueryFilter(m => !m.Deleted);
// many more entity types....
All the entities inherit a BaseModel which looks like this:
public class BaseModel
{
public Guid CreatedBy { get; set; }
public Guid UpdatedBy { get; set; }
public DateTime DateCreated { get; set; }
public DateTime DateUpdated { get; set; }
public bool Deleted { get; set; }
}
Is it possible to add the query filter for any class that inherits the BaseModel?
Something like:
modelBuilder.Entity<BaseModel>().HasQueryFilter(m => !m.Deleted);
So I don't forget (at a later date) to add the query filter for models I add?
For the latest EF Core version (should work for 3.0 also, for earlier versions expression replacement should be handled manually, see ReplacingExpressionVisitor call) you can automate it using some reflection (minimal amount of it), expression trees and IMutableModel.GetEntityTypes in your OnModelCreating method. Something like this should work:
// define your filter expression tree
Expression<Func<BaseModel, bool>> filterExpr = bm => !bm.Deleted;
foreach (var mutableEntityType in modelBuilder.Model.GetEntityTypes())
{
// check if current entity type is child of BaseModel
if (mutableEntityType.ClrType.IsAssignableTo(typeof(BaseModel)))
{
// modify expression to handle correct child type
var parameter = Expression.Parameter(mutableEntityType.ClrType);
var body = ReplacingExpressionVisitor.Replace(filterExpr.Parameters.First(), parameter, filterExpr.Body);
var lambdaExpression = Expression.Lambda(body, parameter);
// set filter
mutableEntityType.SetQueryFilter(lambdaExpression);
}
}
You need to construct a lambda expression at runtime for:
instance => !instance.IsDeleted
Now, assume we have this interface, and a number of entities that implement this interface (directly or transitively):
interface ISoftDelete
{
bool IsDeleted { get; set; }
}
class Product : ISoftDelete
{
public int Id { get; set; }
public bool IsDeleted { get; set; }
// ...
}
We want to apply soft-delete query filter to all entities that implement this interface.
To find all entities registered in a DbContext model, we can use IMutableModel.GetEntityTypes(). Then we filter all entities implementing ISoftDelete and add set a custom query filter.
Here's an extension method that you can use directly:
internal static class SoftDeleteModelBuilderExtensions
{
public static ModelBuilder ApplySoftDeleteQueryFilter(this ModelBuilder modelBuilder)
{
foreach (var entityType in modelBuilder.Model.GetEntityTypes())
{
if (!typeof(ISoftDelete).IsAssignableFrom(entityType.ClrType))
{
continue;
}
var param = Expression.Parameter(entityType.ClrType, "entity");
var prop = Expression.PropertyOrField(param, nameof(ISoftDelete.IsDeleted));
var entityNotDeleted = Expression.Lambda(Expression.Equal(prop, Expression.Constant(false)), param);
entityType.SetQueryFilter(entityNotDeleted);
}
return modelBuilder;
}
}
Now, we can use this in our DbContext:
class AppDbContext : DbContext
{
// ...
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Product>();
modelBuilder.ApplySoftDeleteQueryFilter(); // <-- must come after all entity definitions
}
}
use below code to get all entities and filter a property:
foreach (var entityType in modelBuilder.Model.GetEntityTypes())
{
if (entityType.ClrType.GetCustomAttributes(typeof(AuditableAttribute), true).Length > 0)
{
modelBuilder.Entity(entityType.Name).Property<bool>("IsRemoved");
}
var isActiveProperty = entityType.FindProperty("IsRemoved");
if (isActiveProperty != null && isActiveProperty.ClrType == typeof(bool))
{
var entityBuilder = modelBuilder.Entity(entityType.ClrType);
var parameter = Expression.Parameter(entityType.ClrType, "e");
var methodInfo = typeof(EF).GetMethod(nameof(EF.Property))!.MakeGenericMethod(typeof(bool))!;
var efPropertyCall = Expression.Call(null, methodInfo, parameter, Expression.Constant("IsRemoved"));
var body = Expression.MakeBinary(ExpressionType.Equal, efPropertyCall, Expression.Constant(false));
var expression = Expression.Lambda(body, parameter);
entityBuilder.HasQueryFilter(expression);
}
}
i've got many ef core entities with a possibility to join a shared child table ( without FK-s). I created a generic join extension method, but got a bit stuck returning the main parent entity with the child mapped to it.
Heres what i've got :
public interface IBaseEntity
{
int Id { get; set; }
}
public interface IBaseAttachmentEntity:IBaseEntity
{
ICollection<ResourceAttachment> ResourceAttachment { get; set; }
}
public class ResourceAttachment:BaseEntity
{
//PK- Id of the parent table
public long ParentId { get; set; }
// type or enum of a parent table. Should point to which table it points to
public string ResourceType { get; set; }
public string AttachmentType { get; set; }
public string Value { get; set; }
}
public static class EFCoreExtension
{
//this must be a generic method, want to use it for ~30 tables where the entity inherits IBaseAttachmentEntity
public static IQueryable<TEntity> IncludeResourceAttachment<TEntity>
(this IQueryable<TEntity> queryable,IServiceProvider serviceProvider) where TEntity : class,IBaseAttachmentEntity
{
var className = queryable.ElementType.Name;
var attachmentRepository = serviceProvider.GetRequiredService<IResourceAttachmentRepository>();
var attachments = attachmentRepository
.FindAllQueriable(x => x.ResourceType == className); //this part is fine
var joined = (from q in queryable
join a in attachments on q.Id equals a.ParentId into qa
from a in qa.DefaultIfEmpty()
select new {Parent=q,Attachments = qa}); // joining the child table, ending up with a tuple like result
return joined.Select(x => x.Parent); // need to return the Parent together with Parent.ResourceAttachment which id defined in the
}
}
want to use the extension like this:
var result = _deviceServiceService.FindAllQueriable(CurrentUserId(), matchFilter)
.IncludeResourceAttachment(_serviceProvider).ToList();
EDIT:
i have also created a minimal sample project to run. Uses In memory Db with data seeding
https://github.com/rauntska/EFGenericChildJoin
Thanks!
THis might not be what you want, but you might have to settle for it:
public static IEnumerable<TEntity> IncludeResourceAttachment<TEntity>
(this IQueryable<TEntity> queryable, IServiceProvider serviceProvider) where TEntity : class, IBaseAttachmentEntity
{
var className = queryable.ElementType.Name;
var dataContext = serviceProvider.GetRequiredService<DataContext>();
var attachments = dataContext.Set<ResourceAttachment>().Where(x => x.ResourceType == className);
var joined =
from q in queryable
join a in attachments on q.Id equals a.ParentId into qa
select new { Parent = q, Attachments = qa };
foreach (var x in joined)
foreach (var y in x.Attachments)
x.Parent.ResourceAttachment.Add(y);
return joined.Select(x => x.Parent);
}
I am trying to do a unit test on a class using the MOQ framework. A public method in the class has a line like the below.
Looking at that line, I'm trying to mock the _currencyRepository but don't know how to. Considering it has a parameter like p => new { p.Id, p.Code }.
On analysing the code, it appears it is a way of outputting P - anonymous method I believe.
var currencies = await _currencyRepository.GetsAs(p => new { p.Id, p.Code }, p => p.Id == sellCurrencyId || p.Id == buyCurrencyId);
Hovering the mouse over var to get an idea of the type that get returns, the tootip shows IEnumerable<'a> currencies. Anonymous types: 'a is new{Guid Id, string code}.
..and GetAs definition is:
public async Task<IEnumerable<TOutput>> GetsAs<TOutput>(Expression<Func<TEntity, TOutput>> projector,
Expression<Func<TEntity, bool>> spec = null,
Func<IQueryable<TEntity>, IQueryable<TEntity>> preFilter = null,
params Func<IQueryable<TEntity>, IQueryable<TEntity>>[] postFilters)
{
if (projector == null)
{
throw new ArgumentNullException("projector");
}
return await FindCore(true, spec, preFilter, postFilters).Select(projector).ToListAsync();
}
GetAs() is also a member/method in a Generic class GenericRepository - whose class definition is:
public class GenericRepository<TContext, TEntity> : IGenericRepository<TEntity>
where TEntity : BaseEntity
where TContext : DbContext
{
protected readonly TContext DbContext;
public GenericRepository(TContext dbContext)
{
DbContext = dbContext;
}
//followed by method definitions including GetAs
}
The class above inherits from the generic interface IGenericRepository<TEntity>, which is defined as:
public interface IGenericRepository<TEntity>
where TEntity : class
{
//list of methods including GetAs
Task<IEnumerable<TOutput>> GetsAs<TOutput>(Expression<Func<TEntity, TOutput>> projector,
Expression<Func<TEntity, bool>> spec = null,
Func<IQueryable<TEntity>, IQueryable<TEntity>> preFilter = null,
params Func<IQueryable<TEntity>, IQueryable<TEntity>>[] postFilters);
...BaseEntity is just a class with properties:
public class BaseEntity
{
public BaseEntity()
{
Id = Guid.NewGuid();
CreatedDate = DateTime.Now;
}
public Guid Id { get; set; }
public DateTime CreatedDate { get; set; }
public DateTime? UpdatedDate { get; set; }
public Guid? CreatedBy { get; set; }
public Guid? UpdateddBy { get; set; }
}
I already have the list of currency that I'm trying to return, I'm just having errors with the parameters. Not sure what argument matchers to use (I'm aware argument matcher is NSubstitute talk. Not sure if it is called the same in Moq)
I tried a number of things. One of which is the below:
var mockCurrencies3 = new[] { new { Id = buyCurrencyId, Code = "EUR" }, new { Id = sellCurrencyId, Code = "GBP" } };
MockCurrencyRepository.Setup(x => x.GetsAs(
It.IsAny<Expression<Func<Currency, (Guid, string)>>>(),
It.IsAny<Expression<Func<Currency, bool>>>(),
It.IsAny<Func<IQueryable<Currency>, IQueryable<Currency>>>()))
.Returns(Task.FromResult(mockCurrencies3.AsEnumerable()));
Trying the above does not work as mockCurrencies3 value is not getting returned in the production code. I don't get anything back.
Using the provided information I was able to create a generic method to setup the desired behavior
[TestClass]
public class MyTestClass {
[TestMethod]
public async Task Should_Mock_Generic() {
Guid buyCurrencyId = Guid.NewGuid();
Guid sellCurrencyId = Guid.NewGuid();
var mockCurrencies3 = new[] { new { Id = buyCurrencyId, Code = "EUR" }, new { Id = sellCurrencyId, Code = "GBP" } };
var mock = new Mock<IGenericRepository<Currency>>();
SetupGetAs(mock, mockCurrencies3);
var _currencyRepository = mock.Object;
var currencies = await _currencyRepository.GetsAs(p => new { p.Id, p.Code }, p => p.Id == sellCurrencyId || p.Id == buyCurrencyId);
currencies.Should().NotBeNullOrEmpty();
}
Mock<IGenericRepository<TEntity>> SetupGetAs<TEntity, TOutput>(Mock<IGenericRepository<TEntity>> mock, IEnumerable<TOutput> source)
where TEntity : class {
mock.Setup(x => x.GetsAs(
It.IsAny<Expression<Func<TEntity, TOutput>>>(),
It.IsAny<Expression<Func<TEntity, bool>>>(),
It.IsAny<Func<IQueryable<TEntity>, IQueryable<TEntity>>>())
)
.ReturnsAsync(source);
return mock;
}
}
I am converting an Expression<T, bool> to an Expression<Y, bool> where T and Y are different entities not related in any way other than through an Automapper mapping. Essentially, I have a Model object that my code uses:
public class Store
{
public string StoreId { get; set; }
public string Name { get; set; }
public List<Phone> Phones { get; set; }
public Address Address { get; set; }
public Account Account { get; set; }
public Status Status { get; set; }
}
that I am mapping to an entity object to store in my mongo database:
public class Store : MongoEntity
{
public string AccountId { get; set; }
public string Name { get; set; }
public List<string> UserIds { get; set; }
public List<Phone> PhoneNumbers { get; set; }
public Address Address { get; set; }
}
public abstract class MongoEntity : IMongoEntity
{
[BsonId]
public ObjectId Id { get; set; }
public Status Status { get; set; }
}
I used the answer in this question to work out how to convert between expressions (Question), and that got me 90% there. I was able to modify the code to grab the AutoMapper mappings between my Model store and my entity store and grab the destination property from from the source property:
private Expression<Func<TNewTarget, bool>> TransformPredicateLambda<TOldTarget, TNewTarget>(
Expression<Func<TOldTarget, bool>> predicate)
{
var lambda = (LambdaExpression)predicate;
if (lambda == null)
{
throw new NotSupportedException();
}
//Modified here to get automapper mappings
var maps = Mapper.FindTypeMapFor<TOldTarget, TNewTarget>();
var mutator = new ExpressionTargetTypeMutator(t => typeof(TNewTarget), maps);
var explorer = new ExpressionTreeExplorer();
var converted = mutator.Visit(predicate.Body);
return Expression.Lambda<Func<TNewTarget, bool>>(
converted,
lambda.Name,
lambda.TailCall,
explorer.Explore(converted).OfType<ParameterExpression>());
}
protected override Expression VisitMember(MemberExpression node)
{
var dataContractType = node.Member.ReflectedType;
var activeRecordType = _typeConverter(dataContractType);
PropertyMap prop = null;
foreach (var propertyMap in _maps)
{
var source = propertyMap.SourceMember;
var dest = propertyMap.DestinationProperty;
if (source != null && source.Name == node.Member.Name)
{
prop = propertyMap;
}
}
if (prop == null)
{
return base.VisitMember(node);
}
var propertyName = prop.DestinationProperty.Name;
var property = activeRecordType.GetProperty(propertyName);
var converted = Expression.MakeMemberAccess(
base.Visit(node.Expression),
property
);
return converted;
}
The problem is, my entity object doesn't have all of the same properties as my Model object (Account object versus AccountId for example). When the Transformer gets to the Account property on the Model object, I get an Exception (because there is no matching property on my Entity object). I cannot return null from VisitMember, and new Expression() is not allowed either. How can I handle ignoring properties on my Model object that do not exist on my Entity object?
Updating with info from Comments
So, to be a little more clear, I am using Automapper to map from a Models.Store to an Entity.Store. My entity.Store only has an AccountId (because I don't want to duplicate all of the account data), but my Models.Store needs the whole account object (which I would get by querying the Accounts collection).
Automapper is bascially converting my Account object to just an AccountId on my entity. Therefore, when I search for x => x.Account.AccountId == abcd1234 (where x is a models.Store), I need my expression to convert to x => x.AccountId == abcd1234 (where x is an Entity.Store).
I have that part working (changing mS => mS.Account.AccountId == 1234 to mE => mE.AccountId == 1234). The problem I am having now is that after doing the AccountId property, VisitMember is called with Account as the node. Since there is no Account in my Entity.Store object, I get the exception.
It's rather hard to test a solution without testable/runnable code. But here's a guess
Given the following expression mS => mS.Account.AccountId == 1234 and looking to transform MemberExpressions, you'll get the following calls:
VisitMember(mS.Account.AccountId
VisitMember(mS.Account)
You want to transform the second one into mE.AccountId. This involves two transformations: One, changing the property access from (EntityType).(AccountType).AccountId to (MongoStoreType).AccountId, and also changing the underlying object. If you're already handling the parameter transformations in other methods of your ExpressionVisitor, probably VisitParameter and VisitLambda, you'll be fine there. You then just need to skip looking at the parent MemberAccess, and jump straight to the grandparent:
var converted = Expression.MakeMemberAccess(
base.Visit(node.Expression),
property
);
return converted;
becomes something like this:
var parentMember = node.Expression as MemberExpression;
if (parentMember != null)
{
var grandparent = parentMember.Expression;
var converted = Expression.MakeMemberAccess(
base.Visit(grandparent),
property
);
return converted;
}
else
{
var converted = Expression.MakeMemberAccess(
base.Visit(node.Expression),
property
);
return converted;
}