We Keep Coding

Is using EF Core include method inside a foreach can be a performance issue

These days I am working on a project, and this project depends on another project that will play the repository, my repository project based on Generic Repository pattern.
In my repository project I created a method that will get all data from a received entity and load data from selected entities in a received expression.
My method:
public List<TEntity> GetAll<TEntity>(params Expression<Func<TEntity, object>>[] expressions) where TEntity : class
{
var result = new List<TEntity>();
foreach (var expression in expressions)
{
result = Context.Set<TEntity>().Include(expression).ToList();
}
return result;
}
The question:
As you will notice in the above method I used Include method inside an iteration (Foreach), so my question is : is that using of Include inside foreach have a performance issue or any other problem ?

Probably you need this:
public List<TEntity> GetAll<TEntity>(params Expression<Func<TEntity, object>>[] expressions) where TEntity : class
{
var query = Context.Set<TEntity>().AsQueryable();
foreach (var expression in expressions)
{
query = query.Include(expression);
}
return query.ToList();
}

Many to Many using LinqKit with EF Core with Generic class

I'm currently converting an EF project to EF Core but I don't know how or whether it is possible to:
a) Use LinqKit on Many-To-Many with EF Core
b) Use a) with a Generic class
I'm currently using the following class:
protected virtual IQueryable<TEntity> GetQueryable(
Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = null,
bool isCollection = false,
int? skip = null,
int? take = null)
{
IQueryable<TEntity> query = this.Context.Set<TEntity>();
if (filter != null)
{
query = query.AsExpandable().Where(filter);
}
if (includeProperties != null)
{
foreach (var includeProperty in includeProperties.Split
(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
}
if (orderBy != null)
{
query = orderBy(query);
}
if (skip.HasValue)
{
query = query.Skip(skip.Value);
}
if (take.HasValue)
{
query = query.Take(take.Value);
}
return query;
}
In EF 6, I would set the type of my generic class to a main Entity I wanted to deal with and if it had a many to many, I would simply set the IncludeProperties to the child Entity.
For example:
var test = GetQueryable<Company>(null, "Users");
This would returns all companies and return all the users associated with each relevant company.
LinqKit would then come in to add additional filters, orders, etc...
But I cannot figure out how to use this with a Many to Many relation in EF Core.
I've seen an example where you do something similar to this:
this.Context.Companies.Include("CompanyUsers")
.ThenInclude("Users")
.ToListAsynch()
And the above looks like what I need but I need to use it to be generic and I need to apply filters with LinqKit or some other way but I need to set filters, sorting, paging, etc... dynamically.
Any ideas?
Thanks
UPDATE-1:
My many-to-many relation class is defined as follows:
public class CompanyUsers
{
public Guid UserId { get; set; }
public User User { get; set; }
public Guid CompanyId { get; set; }
public Company Company { get; set; }
}
The reason I'm adding this is that while my table of users is called "Users" the navigation property is called User and as suggested by Dennis, I should specify the Many-To-Many class (CompanyUsers) followed by the navigation property (User) so rather than being:
var test = GetQueryable<Company>(null, "CompanyUsers.Users");
it should be defined as instead:
var test = GetQueryable<Company>(null, "CompanyUsers.User");
Note the s was removed to match the navigation property rather than the table.

The simplest solution would be to use the string overload of the Include.*
var test = GetQueryable<Company>(null, "CompanyUsers.User");
*I don't recommend using magic strings. You should consider writing an method that returns this based on the models properties.

How to do Complex Joins in Repository pattern using NHibernate?

I am using Repository pattern to develop my project using Nhibernate. How to do complex join in Nhibernate. I am using below repository method for fetching data
public virtual TEntity FindBy(Expression<Func<TEntity, bool>> query)
{
try
{
return NHUnitOfWork.Session.Query<TEntity>().Where(query).FirstOrDefault();
}
catch (Exception ex)
{
throw;
}
}
Here I can fetch data without any joins. How to fetch data with joins?
for eg: Menu details are storing in Menu table and Menu Rights are storing to Menu Rights table. How to create a repository for that?

With QueryOver, accepting input Junction instead of Expression to your FindBy method will do.
Following code may be helpful. Remove columnList and top parameters if not needed to you. Please note that my ISession usage is different. You need to replace nhSession with NHUnitOfWork.Session. Another difference is that you are using Query and I am using QueryOver here.
public virtual IList<T> FindBy<T>(ProjectionList columnList, Junction where, int top) where T : BaseEntity
{
IList<T> instance = GetQueryOver<T>(columnList, where).Take(top).List();
return instance;
}
public virtual IQueryOver<T> GetQueryOver<T>(ProjectionList columnList, Junction where) where T : BaseEntity
{
IQueryOver<T> query = null;
if((columnList != null) && (where != null))
{
query = nhSession.QueryOver<T>()
.Select(columnList)
.TransformUsing(Transformers.AliasToBean<T>())
.Where(where);
}
else if((columnList != null) && (where == null))
{
query = nhSession.QueryOver<T>()
.Select(columnList)
.TransformUsing(Transformers.AliasToBean<T>());
}
else if((columnList == null) && (where != null))
{
query = nhSession.QueryOver<T>()
.Where(where);
}
else
{
query = nhSession.QueryOver<T>();
}
return query;
}
And then, you can call this at some other location like:
Junction where = Restrictions.Conjunction();
where.Add(Restrictions.Eq(Projections.Property<MyEntity>(x => x.Field), findValue));
where.Add(Restrictions.................);
where.Add(Restrictions.................);
entityList = Repository.FindBy<MyEntity>(null, where, 100);

If you want to eager load related entities, use Fetch and FetchMany.
public virtual Menu FindBy(Expression<Func<Menu, bool>> query)
{
return NHUnitOfWork.Session.Query<Menu>().Where(query)
.FetchMany(m => m.Rights)
// Required with FetchMany and First, otherwise only one right would be loaded.
.ToList()
.FirstOrDefault();
}
Or if your model has only one right per menu:
public virtual Menu FindBy(Expression<Func<Menu, bool>> query)
{
return NHUnitOfWork.Session.Query<Menu>().Where(query)
.Fetch(m => m.Right)
.FirstOrDefault();
}
But it seems you want to define some "generic repository" encapsulating the NHibernate API.
Maybe then:
public virtual TEntity FindBy<TFetched>(Expression<Func<TEntity, bool>> query,
Expression<Func<TEntity, IEnumerable<TFetched>>> fetchMany)
{
var query = NHUnitOfWork.Session.Query<TEntity>().Where(query);
if (fetchMany != null)
query = query.FetchMany(fetchMany);
return query
// Required with FetchMany and First, otherwise only one right would be loaded.
.ToList()
.FirstOrDefault();
}
Or if your model has only one right per menu:
public virtual TEntity FindBy<TFetched>(Expression<Func<TEntity, bool>> query,
Expression<Func<TEntity, TFetched>> fetch)
{
var query = NHUnitOfWork.Session.Query<TEntity>().Where(query);
if (fetch != null)
query = query.Fetch(fetch);
return query
.FirstOrDefault();
}
But how to do if you need many fetches*? And sub-fetches (ThenFetch/ThenFetchMany)? It looks to me like a cursed path. You may end up coding encapsulation for the whole NHibernate API following that path.
Indeed, exposing publicly an Expression where argument on repositories does not look good to me. Your repository is then no more responsible of defining how the data is queried.
If you want to do that, why not exposing directly the IQueryable? It would prove far less code verbose than trying to encapsulate it in a "repository" while still defining the query outside of this "repository".
*: beware of Cartesian products in case you do more than one FetchMany. To avoid it, either split your query in many (using ToFuture if you wish a single round-trip to DB), or use lazy loading with batching instead of fetching.

Repository generic method GetById using eager loading

I am using Entity Framework and would like to create generic GetById method in Repository class with eager loading:
Here is my method which uses lazy-loading:
public virtual TEntity GetById(object id)
{
return DbSet.Find(id);
}
I know method Find does not support eager loading, but how it is possible to modify this method to use eager loading, so that I use this method as follows(for an example):
_unitOfWork.MyRepository.GetById(includeProperties: "Users");

One possible way is to use FirstOrDefault with predicate over the DbSet with Includes. It's not hard to build manually a predicate using Expression.Equal method, but the main challenge is how to get the key property name. Luckily we can use some ObjectContext methods to do that, so the implementation could be like this (assuming we have access to the concrete DbContext instance):
public virtual TEntity GetById(object id, params string[] includeProperties)
{
var propertyName = ((System.Data.Entity.Infrastructure.IObjectContextAdapter)DbContext).ObjectContext
.CreateObjectSet<TEntity>().EntitySet.ElementType.KeyMembers.Single().Name;
var parameter = Expression.Parameter(typeof(TEntity), "e");
var predicate = Expression.Lambda<Func<TEntity, bool>>(
Expression.Equal(
Expression.PropertyOrField(parameter, propertyName),
Expression.Constant(id)),
parameter);
var query = DbSet.AsQueryable();
if (includeProperties != null && includeProperties.Length > 0)
query = includeProperties.Aggregate(query, System.Data.Entity.QueryableExtensions.Include);
return query.FirstOrDefault(predicate);
}

This is the update for Entity Framework Core 2.0. Also this method using the new metadata properties with EF to automatically get the first level navigation properties.
public virtual T Get(object id)
{
var propertyName = "AddressId";
//get all navigation properties defined for entity
var navigationProps = _context.Model.FindEntityType(typeof(T)).GetNavigations();
//turn those navigation properties into a string array
var includeProperties = navigationProps.Select(p => p.PropertyInfo.Name).ToArray();
//make parameter of type T
var parameter = Expression.Parameter(typeof(T), "e");
//create the lambda expression
var predicateLeft = Expression.PropertyOrField(parameter, propertyName);
var predicateRight = Expression.Constant(id);
var predicate = Expression.Lambda<Func<T, bool>>(Expression.Equal(predicateLeft, predicateRight), parameter);
//get queryable
var query = _context.Set<T>().AsQueryable();
//apply Include method to the query multiple times
query = includeProperties.Aggregate(query, Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.Include);
//return first item in query
return query.FirstOrDefault(predicate);
}

Use This In Repository
public IQueryable<T> FindByCondition(Expression<Func<T, bool>> expression)
{
return this.RepositoryContext.Set<T>().Where(expression).AsNoTracking();
}
and This In Action Method
var User = _IRepositoryWrapper.User.FindByCondition(x=>x.Id == id).Include(a=>a.Photos).FirstOrDefault();

IQueryable Include not giving correct results

I have a generic repository implementation that creates an IQueryable as follows:
internal IEnumerable<T> Get(Expression<Func<T, bool>> filter = null,
Func<IQueryable<T>, IOrderedQueryable<T>> orderBy = null,
List<Expression<Func<T, object>>> includeProperties = null,
int? page = null,
int? pageSize = null)
{
IQueryable<T> query = m_dbSet;
if (filter != null)
query = query.Where(filter);
if (orderBy != null)
query = orderBy(query);
if (page != null && pageSize != null)
query = query.Skip((page.Value - 1) * pageSize.Value)
.Take(pageSize.Value);
if (includeProperties != null)
includeProperties.Aggregate(query, (current, include) => current.Include(include));
return query.ToList();
}
The problem that I'm having is that the Expression I pass to the include does not get included in the output of the query. The SQL that is generated does not include the JOIN.
I have no other parameters on the query. Filter, orderBy, page and pageSize are all null. The List of includes contains a single Expression and the member referred to by the Expression is declared as
public virtual MyType m_Member { get; set; }
within the POCO class for the Repository objects.
I've read that IQueryable drops the Include on a Select or Projection but I'm not doing either here I believe. Does anyone know what's going on here?

You should assign the result of the aggregation to query, I think:
query = includeProperties.Aggregate(query, ...);