We Keep Coding

Generic Repository GetByID method with "Including" ....

I am trying to add a method to my generic repository that will allow me to query an entity by it's PrimaryKey (ID) and select what navigation properties to return. The kicker is, that in order to be truly generic I do not know the data type or field name of the Primary Key (Not supporting compound keys ... yet).
So here's what I started with ... the below snippet gets me the TEntity based on key regardless of the TYPE of the key ...
public virtual TEntity GetByID(Object entityID)
{
TEntity ret = DbSet.Find(entityID);
return ret;
}
The next step was to get records back, but including whatever navigation fields I chose to include ... This worked out great for that ... for multiple records.
public virtual IQueryable<TEntity> GetAllIncluding(params Expression<Func<TEntity, object>>[] includedProperties)
{
var query = DbSet.AsQueryable();
foreach (var prop in includedProperties)
{
query = query.Include(prop);
}
return query;
}
//
// example usage gets ALL books including Author and Publisher information
var repo = uow.GetRepository<Book>();
repo.GetAllIncluding(e=>e.Author, e=>e.Publisher);
So now I need to mix the two ... here's the rub. In order to dynamically add the "including" expressions I need to convert the DbSet to an IQueryable ... however in order to leverage the type-agnostic key lookup I need to use the "Find()" method on the DbSet ...
So in theory I need to convert the DbSet to an IQueryable, add the Include expressions then convert the IQueryable BACK to a DbSet so that I can access the type-agnostic "Find()" method ... this does not work ...
Anyone got any ideas on how this can be done ... ??
Here's some pseudo-code for what I am trying to get to work ...
public virtual TEntity GetByID(Object entityID, params Expression<Func<TEntity, object>>[] includedProperties)
{
var query = DbSet.AsQueryable();
foreach (var prop in includedProperties)
{
query = query.Include(prop);
}
TEntity ret = ((DbSet)query).Find(entityID);
return ret;
}

Can a generic method return the property I specify

Yesterday Ognyan helped me a great deal write this method:
public static class DbSetExtensions
{
public static T AddIfNotExists<T>(this DbSet<T> dbSet, T entity, Expression<Func<T, bool>> predicate = null) where T : class, new()
{
var exists = predicate != null ? dbSet.Any(predicate) : dbSet.Any();
return !exists ? dbSet.Add(entity) : null;
}
}
When this does do an Add of my entity, I get back the new ID of the entity, but if it exists it returns null.
There will be times that I would like to get back the ID of an entity that already exists. However, the Key ID property of my entities will be different depending on the model. For instance, Address model's key is AddressId, Profile's key is ProfileId.
So, I'd like to modify this query (or make another version of it) to accept the Id property name as a parameter. (Or use EF to recognize the Primary key.) And do something like this:
public static class DbSetExtensions
{
public static T AddIfNotExists<T>(this DbSet<T> dbSet,
T entity,
Expression<Func<T, bool>> predicate = null,
Expression<Func<T, TId>> keyColumnName) where T : class, new()
{
var exists = predicate != null ? dbSet.Where(predicate).Select(e => e.keyColumnName) : dbSet.Any();
return !exists ? dbSet.Add(entity) : exists ;
}
}
I'm not sure if that is the correct way to define the property I want to use. I also realize this may not be as fast as just doing an Any(), but at times it may be necessary for us to get the ID.
I would also like to understand better what all of these items mean and how they work together. I've been all over and gotten bits and pieces, but haven't been able to put the whole puzzle together.

Well maybe there is a better solution, but you can try this:
public static TResult AddIfNotExists<T,TResult>(this DbSet<T> dbSet,
T entity,
Expression<Func<T, bool>> predicate,
Expression<Func<T, TResult>> columns
) where T : class, new()
{
if (predicate==null || columns==null)
throw new Exception();
//Find if already exist the entity and select its key column(s)
var result = dbSet.Where(predicate).Select(columns).FirstOrDefault();
//the result could be a reference type (string or anonymous type) or a value type
if (result!=null && !result.Equals(default(TResult)))
return result;
var newElement = dbSet.Add(entity);
//Compile the Expresion to get the Func
var func = columns.Compile();
//To select the new element key(s), add the element to an array (or List) to apply the Select method and get the keys
var r = new[]{newElement};
return r.Select(func).FirstOrDefault();
}
As you can see, I always expect the parameters predicate to check if the element already exist and columns to select the key or keys in case you have an entity with composite PK (but notice you can select all the properties you want from that entity, not just the keys). Then, I apply the expressions to filter and select the key(s) in case the element exist. If it doesn't exist, I add the entity to its DbSet<T>, but there is a problem. How can you get the new element key(s)? Well, the only solution I found to this was adding that element to an array, compiling the expression to get a Func<T,TResult>, and apply that Func<T,TResult> to the array, (I know this may not be the best solution but it should work).
To call this extension method, for example, for an entity that have a composite PK, you can do this:
.AddIfNotExists(element, e => e.UserName== "admin", e => new{ e.Id1,e.Id2});

The more simple in your case should be:
public static class DbSetExtensions
{
public static T AddIfNotExists<T>(this DbSet<T> dbSet, T entity,
Expression<Func<T, bool>> predicate = null) where T : class, new()
{
T exists = predicate != null ?
dbSet.Where(predicate).FirstOrDefault():
dbSet.FirstOrDefault();
return exists == null ?
dbSet.Add(entity):
exists;
}
}
then, as you seem to know the PK property, you can check it from the returned value.

Pass lambda expressions of object properties to method to Select columns in EF

I am currently using one of the many repository patterns available online to perform CRUD operations with EF6. I am happy to use it as is but I have recently been handed a few legacy projects that have database tables with a very high number of columns. I would like a way to make my application as well as future applications smoother by devising a way to select only a subset of columns.
Current method.
public virtual TEntity Get(Expression<Func<TEntity, bool>> where,
params Expression<Func<TEntity, object>>[] navigationProperties)
{
TEntity item = null;
IQueryable<TEntity> dbQuery = this.Context.Set<TEntity>();
//Apply eager loading
foreach (Expression<Func<TEntity, object>> navigationProperty in navigationProperties)
dbQuery = dbQuery.Include<TEntity, object>(navigationProperty);
item = dbQuery
.AsNoTracking() //Don't track any changes for the selected item
.FirstOrDefault(where); //Apply where clause
return item;
}
I would like to enhance that method to retrieve only the columns I require but still return TEntity.
I do know I have to inject a Select after the '.AsNoTracking()' but I am unsure as to how I could pass the properties in as I am only starting out with Expression Trees.
In essence I would like to be able to do this.
public class Employee
{
public int EmployeeId { get;set; }
public string EmployeeRole { get;set; }
public string EmployeeFirstName { get;set; }
public string EmployeeLastName { get;set; }
public string DOB { get;set; }
...
}
Employee employee = EmployeeRepository.Get(where: e => e.EmployeeRole == "Developer",
columns: x => x.EmployeeFirstName, x => x.EmployeeLastName,
navigationProperties: null);
Where columns is a list of expressions specifying the columns to be added to the Select clause.
Any help would be appreciated.
Thanks in advance...
Update.
I ended up with using a DTO to do the necessary querying and extraction as I couldn't find an elegant way to perform it generically. There was a solution developed by a colleague of mine but it made the repository far too complex and would have been hard to manage in the future.
So I create a StaffBasicInformation class to hold the subset of columns I use regularly. I also created an interface for it if I needed in the future. The below code sample shows the final implementation of retrieving data for the DTO.
public virtual IStaffBasicInformation GetStaffBasicInformation<TEntity2>(Expression<Func<TEntity2, bool>> where)
where TEntity2 : ActiveStaffMember
{
TEntity2 item = null;
StaffBasicInformation resultItem = null;
IQueryable<TEntity2> dbQuery = this.Context.Set<TEntity2>();
resultItem =
dbQuery.Where(where)
.Select(x => new StaffBasicInformation
{
GivenName = x.GivenName,
Department = x.Department,
Description = x.Description,
DisplayName = x.DisplayName,
Gender = x.Gender,
IID = x.IID,
Mail = x.Mail,
Title = x.Title,
ID = x.Id
})
.FirstOrDefault();
return resultItem;
}

Your return value will not be of type TEntity anymore after you have done the projection, it will be an anonymous type. You have to decide, if you want to map this anonymous type to an instance of TEntity, including mapping all navigationproperties, or return dynamic or object from your repository. Both choices are not very pleasant. Mapping would include a huge amount of reflection, which will not be very fast. By returning a dynamic type you loose all type safety. You have seen this problem allready i assume.
That said: You will need to build this expression manually. Based on this answer you can modify the
public static IQueryable SelectDynamic(this IQueryable source, IEnumerable<string> fieldNames)
to
public static IQueryable SelectDynamic(this IQueryable source, IEnumerable<Expression> fieldNames)
end extract the property names from the expressions. I would suggest to use an ExpressionVisitor for that, but you could also use the code from this answer
For the mapping you can compile the expressions and use the returned Func to retrieve the value from the anonymous type. After that you would need to use expression and find the hosting type for the selected property by using an ExpressionVisitor again. Then you will need to create a object of type of TEntity via Activator.CreateType(), and objects for every hosting type. Assign the value from Func(AnonymousType) to the created object of the hosting type using the property name from the expression. After that you have to determin the relationship between TEntity and the hosting type and build it up.
I will try to post some code tomorrow for this scenario, although i am quite sure there is a better and faster way.

Accessing properties through Generic type parameter

I'm trying to create a generic repository for my models. Currently i've 3 different models which have no relationship between them. (Contacts, Notes, Reminders).
class Repository<T> where T:class
{
public IQueryable<T> SearchExact(string keyword)
{
//Is there a way i can make the below line generic
//return db.ContactModels.Where(i => i.Name == keyword)
//I also tried db.GetTable<T>().Where(i => i.Name == keyword)
//But the variable i doesn't have the Name property since it would know it only in the runtime
//db also has a method ITable GetTable(Type modelType) but don't think if that would help me
}
}
In MainViewModel, I call the Search method like this:
Repository<ContactModel> _contactRepository = new Repository<ContactModel>();
public void Search(string keyword)
{
var filteredList = _contactRepository.SearchExact(keyword).ToList();
}
Solution:
Finally went with Ray's Dynamic Expression solution:
public IQueryable<TModel> SearchExact(string searchKeyword, string columnName)
{
ParameterExpression param = Expression.Parameter(typeof(TModel), "i");
Expression left = Expression.Property(param, typeof(TModel).GetProperty(columnName));
Expression right = Expression.Constant(searchKeyword);
Expression expr = Expression.Equal(left, right);
}
query = db.GetTable<TModel>().Where(Expression.Lambda<Func<TModel, bool>>(expr, param));

Interface solution
If you can add an interface to your object you can use that. For example you could define:
public interface IName
{
string Name { get; }
}
Then your repository could be declared as:
class Repository<T> where T:class, IName
{
public IQueryable<T> SearchExact(string keyword)
{
return db.GetTable<T>().Where(i => i.Name == keyword);
}
}
Alternate interface solution
Alternatively you could put the "where" on your SearchExact method by using a second generic parameter:
class Repository<T> where T:class
{
public IQueryable<T> SearchExact<U>(string keyword) where U: T,IName
{
return db.GetTable<U>().Where(i => i.Name == keyword);
}
}
This allows the Repository class to be used with objects that don't implement IName, whereas the SearchExact method can only be used with objects that implement IName.
Reflection solution
If you can't add an IName-like interface to your objects, you can use reflection instead:
class Repository<T> where T:class
{
static PropertyInfo _nameProperty = typeof(T).GetProperty("Name");
public IQueryable<T> SearchExact(string keyword)
{
return db.GetTable<T>().Where(i => (string)_nameProperty.GetValue(i) == keyword);
}
}
This is slower than using an interface, but sometimes it is the only way.
More notes on interface solution and why you might use it
In your comment you mention that you can't use an interface but don't explain why. You say "Nothing in common is present in the three models. So i think making an interface out of them is not possible." From your question I understood that all three models have a "Name" property. In that case, it is possible to implement an interface on all three. Just implement the interface as shown and ", IName" to each of your three class definitions. This will give you the best performance for both local queries and SQL generation.
Even if the properties in question are not all called "Name", you can still use the nterface solution by adding a "Name" property to each and having its getter and setter access the other property.
Expression solution
If the IName solution won't work and you need the SQL conversion to work, you can do this by building your LINQ query using Expressions. This more work and is significantly less efficient for local use but will convert to SQL well. The code would be something like this:
class Repository<T> where T:Class
{
public IQueryable<T> SearchExact(string keyword,
Expression<Func<T,string>> getNameExpression)
{
var param = Expression.Parameter(typeof(T), "i");
return db.GetTable<T>().Where(
Expression.Lambda<Func<T,bool>>(
Expression.Equal(
Expression.Invoke(
Expression.Constant(getNameExpression),
param),
Expression.Constant(keyword),
param));
}
}
and it would be called thusly:
repository.SearchExact("Text To Find", i => i.Name)

Ray's method is quite good, and if you have the ability to add an interface definitely the superior however if for some reason you are unable to add an interface to these classes (Part of a class library you can't edit or something) then you could also consider passing a Func in which could tell it how to get the name.
EG:
class Repository<T>
{
public IQueryable<T> SearchExact(string keyword, Func<T, string> getSearchField)
{
return db.GetTable<T>().Where(i => getSearchField(i) == keyword);
}
}
You'd then have to call it as:
var filteredList = _contactRepository.SearchExact(keyword, cr => cr.Name).ToList();
Other than these two options you could always look into using reflection to access the Name property without any interface, but this has the downside that there's no compile-time check that makes sure the classes you're passing actually DO have a Name property and also has the side-effect that the LINQ will not be translated to SQL and the filtering will happen in .NET (Meaning the SQL server could get hit more than is needed).
You could also use a Dynamic LINQ query to achieve this SQL-side effect, but it has the same non type-safe issues listed above.

How to check for the presence of an OrderBy in a ObjectQuery<T> expression tree

I'm using T4 for generating repositories for LINQ to Entities entities.
The repository contains (amongst other things) a List method suitable for paging. The documentation for Supported and Unsupported Methods does not mention it, but you can't "call" Skip on a unordered IQueryable. It will raise the following exception:
System.NotSupportedException: The method 'Skip' is only supported for
sorted input in LINQ to Entities. The method 'OrderBy' must be called before
the method 'Skip'..
I solved it by allowing to define a default sorting via a partial method. But I'm having problems checking if the expression tree indeed contains an OrderBy.
I've reduced the problem to as less code as possible:
public partial class Repository
{
partial void ProvideDefaultSorting(ref IQueryable<Category> currentQuery);
public IQueryable<Category> List(int startIndex, int count)
{
IQueryable<Category> query = List();
ProvideDefaultSorting(ref query);
if (!IsSorted(query))
{
query = query.OrderBy(c => c.CategoryID);
}
return query.Skip(startIndex).Take(count);
}
public IQueryable<Category> List(string sortExpression, int startIndex, int count)
{
return List(sortExpression).Skip(startIndex).Take(count);
}
public IQueryable<Category> List(string sortExpression)
{
return AddSortingToTheExpressionTree(List(), sortExpression);
}
public IQueryable<Category> List()
{
NorthwindEntities ent = new NorthwindEntities();
return ent.Categories;
}
private Boolean IsSorted(IQueryable<Category> query)
{
return query is IOrderedQueryable<Category>;
}
}
public partial class Repository
{
partial void ProvideDefaultSorting(ref IQueryable<Category> currentQuery)
{
currentQuery = currentQuery.Where(c => c.CategoryName.Contains(" ")); // no sorting..
}
}
This is not my real implementation!
But my question is, how could I implement the IsSorted method? The problem is that LINQ to Entities query's are always of the type ObjectQuery, which implements IOrderedQueryable.
So how should I make sure an OrderBy method is present in the expression tree? Is the only option to parse the tree?
Update
I've added two other overloads to make clear that it's not about how to add sorting support to the repository, but how to check if the ProvideDefaultSorting partial method has indeed added an OrderBy to the expression tree.
The problem is, the first partial class is generate by a template and the implementation of the second part of the partial class is made by a team member at another time. You can compare it with the way the .NET Entity Framework generates the EntityContext, it allows extension points for other developers. So I want to try to make it robust and not crash when the ProvideDefaultSorting is not implemented correctly.
So maybe the question is more, how can I confirm that the ProvideDefaultSorting did indeed add sorting to the expression tree.
Update 2
The new question was answered, and accepted, I think I should change the title to match the question more. Or should I leave the current title because it will lead people with the same problem to this solution?

Paging depends on Ordering in a strong way. Why not tightly couple the operations? Here's one way to do that:
Support objects
public interface IOrderByExpression<T>
{
ApplyOrdering(ref IQueryable<T> query);
}
public class OrderByExpression<T, U> : IOrderByExpression<T>
{
public IQueryable<T> ApplyOrderBy(ref IQueryable<T> query)
{
query = query.OrderBy(exp);
}
//TODO OrderByDescending, ThenBy, ThenByDescending methods.
private Expression<Func<T, U>> exp = null;
//TODO bool descending?
public OrderByExpression (Expression<Func<T, U>> myExpression)
{
exp = myExpression;
}
}
The method under discussion:
public IQueryable<Category> List(int startIndex, int count, IOrderByExpression<Category> ordering)
{
NorthwindEntities ent = new NorthwindEntities();
IQueryable<Category> query = ent.Categories;
if (ordering == null)
{
ordering = new OrderByExpression<Category, int>(c => c.CategoryID)
}
ordering.ApplyOrdering(ref query);
return query.Skip(startIndex).Take(count);
}
Some time later, calling the method:
var query = List(20, 20, new OrderByExpression<Category, string>(c => c.CategoryName));

You can address this in the return type of ProvideDefaultSorting. This code does not build:
public IOrderedQueryable<int> GetOrderedQueryable()
{
IQueryable<int> myInts = new List<int>() { 3, 4, 1, 2 }.AsQueryable<int>();
return myInts.Where(i => i == 2);
}
This code builds, but is insidious and the coder gets what they deserve.
public IOrderedQueryable<int> GetOrderedQueryable()
{
IQueryable<int> myInts = new List<int>() { 3, 4, 1, 2 }.AsQueryable<int>();
return myInts.Where(i => i == 2) as IOrderedQueryable<int>;
}
Same story with ref (this does not build):
public void GetOrderedQueryable(ref IOrderedQueryable<int> query)
{
query = query.Where(i => i == 2);
}

I'm afraid it's a bit harder than that. You see, the Entity Framework will, in certain circumstances, silently ignore an OrderBy. So it isn't enough to just look for an OrderBy in the expression tree. The OrderBy has to be in the "right" place, and the definition of the "right" place is an implementation detail of the Entity Framework.
As you may have guessed by now, I'm in the same place as you are; I'm using the entity repository pattern and doing a Take/Skip on the presentation layer. The solution I have used, which is perhaps not ideal, but good enough for what I'm doing, is to not do any ordering until the last possible moment, to ensure that the OrderBy is always the last thing into the expression tree. So any action which is going to do a Take/Skip (directly or indirectly) inserts an OrderBy first. The code is structured such that this can only happen once.

Thanks to David B I've got a the following solution. (I had to add detection for the situation where the partial method was not executed or just returned it's parameter).
public partial class Repository
{
partial void ProvideDefaultSorting(ref IOrderedQueryable<Category> currentQuery);
public IQueryable<Category> List(int startIndex, int count)
{
NorthwindEntities ent = new NorthwindEntities();
IOrderedQueryable<Category> query = ent.CategorySet;
var oldQuery = query;
ProvideDefaultSorting(ref query);
if (oldQuery.Equals(query)) // the partial method did nothing with the query, or just didn't exist
{
query = query.OrderBy(c => c.CategoryID);
}
return query.Skip(startIndex).Take(count);
}
// the rest..
}
public partial class Repository
{
partial void ProvideDefaultSorting(ref IOrderedQueryable<Category> currentQuery)
{
currentQuery = currentQuery.Where(c => c.CategoryName.Contains(" ")).OrderBy(c => c.CategoryName); // compile time forced sotring
}
}
It ensures at compile time that if the partial method is implemented, it should at least keep it an IOrderdQueryable.
And when the partial method is not implemented or just returns its parameter, the query will not be changed, and that will use the fallback sort.

ProvideDefaultSorting(ref query);
if (!IsSorted(query))
{
query = query.OrderBy(c => c.CategoryID);
}
Change to:
//apply a default ordering
query = query.OrderBy(c => c.CategoryID);
//add to the ordering
ProvideDefaultSorting(ref query);
It's not a perfect solution.
It doesn't solve the "filter in the ordering function" problem you've stated. It does solve "I forgot to implement ordering" or "I choose not to order".
I tested this solution in LinqToSql:
public void OrderManyTimes()
{
DataClasses1DataContext myDC = new DataClasses1DataContext();
var query = myDC.Customers.OrderBy(c => c.Field3);
query = query.OrderBy(c => c.Field2);
query = query.OrderBy(c => c.Field1);
Console.WriteLine(myDC.GetCommand(query).CommandText);
}
Generates (note the reverse order of orderings):
SELECT Field1, Field2, Field3
FROM [dbo].[Customers] AS [t0]
ORDER BY [t0].[Field1], [t0].[Field2], [t0].[Field3]

I have implemented a solution that sorts whatever collection by its primary key as the default sort order is not specified. Perhaps that will work for you.
See http://johnkaster.wordpress.com/2011/05/19/a-bug-fix-for-system-linq-dynamic-and-a-solution-for-the-entity-framework-4-skip-problem/ for the discussion and the general-purpose code. (And an incidental bug fix for Dynamic LINQ.)