We Keep Coding

What is the best way to create a reusable function to return a value based on related data in an EF Core data model?

I have a standard EF Core data model with several one-many and many-to-many relationships.
I want to create a way to produce various data calculations or commonly run procedures. These should return values to be used throughout the application and not values to be stored in the database.
I'll give a fictional example here of a scenario:
Entity 1 - YearGroup
Entity 2 - Student
Relationship - One YearGroup to many Students
Now I understand you could simply write in the controller:
int student = _context.Students.Where(s => s.YearGroupId == ygId).Count()
But let's say I wanted to simplify this by creating a method somewhere which returns data such as this, so I don't have to specify the query every time I get the number of Students in a YearGroup. I appreciate this is a simple example, but others could be more complex.
I was thinking adding a field to yeargroup.cs model such as:
public int TotalStudents { //code here to get students from students table }
Which I could then use like:
#model.YearGroup.TotalStudents
But I don't know how to get this to include relational data, i.e. from the students table.
I would prefer not create random methods in separate, unrelated classes such as GetStudentsInYearGroup(ygId). It would be nice to include this in the object model if possible.
What would be the best practice way of acheiving this?

Note: I don't have a code editor nor a project setup so what I am going to write might not be compilable.
As simple as your fictional example
For any case as simple as your fictional example, if you just want to get the total students per a year group, and I assume their relationships are properly setup, you can just simply use the navigation property:
// Find the target year group
var yearGroup = _context.YearGroups
.SingleOrDefault(x => x.Id == ygId);
int totalStudents = 0;
if (yearGroup != null)
{
totalStudents = yearGroup.Students.Count();
}
Extension methods
The other way I can think of is to define whatever you need as extension methods of the entity:
public static class YearGroupExtensions
{
public static int GetTotalStudents(this YearGroup yearGroup)
{
if (yearGroup == null)
{
return 0;
}
return yearGroup.Students.Count();
}
public static int GetTotalStudents(this YearGroup yearGroup, Gender gender)
{
if (yearGroup == null)
{
return 0;
}
if (gender == null)
{
return yearGroup.GetTotalStudents();
}
return yearGroup
.Students
.Count(x => x.Gender == gender);
}
}
// usage
// YearGroup yearGroup = GetYearGroup(ygId);
// int totalStudents = yearGroup.GetTotalStudents();
Repository Pattern
If you find yourself repeating similar methods you need for most of your entities, it might be good to define a generic repository for them.
I am not here arguing whether this is just a wrapper of DbContext as that itself is already using repository pattern.
public interface IEntity { }
public interface IRepository<T> where T : IEntity
{
IEnumerable<T> GetAll();
T GetById(int id);
void Insert(T entity);
...
}
public abstract class RepositoryBase<T> : IRepository<T> where T : IEntity
{
protected readonly AppDbContext _dbContext;
protected DbSet<T> _entities;
private RepositoryBase(AppDbContext dbContext)
{
_dbContext = dbContext;
_entities = dbContext.Set<T>();
}
public virtual IEnumerable<T> GetAll()
{
return _entities.AsEnumerable();
}
...
}
public class YearGroupRepository : RepositoryBase<YearGroup>
{
...
}
Separate Persistence and Domain Model
This is my preference just because I'm a DDD guy, and I want to build anything from the domain first (what businesss problems you're trying to solve), without thinking its backend persistence.
The basic idea here is to have 2 sets of models. They could be similar, or totally different. 1 set of models is called the domain model, which reflects your business domain. The other set of models is called the persistence model. That could be your regular Entity Framework Entities.
More on this: https://stackoverflow.com/a/14042539/2410655

Given a DbContext, an entity and a navigation property, you can construct an IQueryable as follows;
public static IQueryable AsQueryable(DbContext context, object entity, string navigation){
var entry = context.Entry(entity);
if (entry.State == EntityState.Detatched)
return null;
var nav = entry.Navigation(navigation);
return nav.Query();
}
I feel like there should be an existing method for this, but I can't seem to find one right now.
You should then be able to use this method in a fairly generic way for any navigation property, without needing to replicate the foreign key criteria all over the place.
public int TotalStudents(DbContext context) =>
AsQueryable(context, this, nameof(Students))?.Count() ?? 0;
While this would add some minor performance overhead, you could extract the base entity and navigation property from a LamdaExpression, and write some extension methods;
public class QueryableVisitor : ExpressionVisitor
{
private object obj;
public object BaseObject { get; private set; }
public string Navigation { get; private set; }
protected override Expression VisitConstant(ConstantExpression node)
{
BaseObject = obj = node.Value;
return base.VisitConstant(node);
}
protected override Expression VisitMember(MemberExpression node)
{
Visit(node.Expression);
BaseObject = obj;
if (node.Member is PropertyInfo prop)
obj = prop.GetValue(obj);
else if (node.Member is FieldInfo field)
obj = field.GetValue(obj);
Navigation = node.Member.Name;
return node;
}
}
public static IQueryable<T> AsQueryable<T>(this DbContext context, Expression<Func<IEnumerable<T>>> expression)
{
var visitor = new QueryableVisitor();
visitor.Visit(expression);
var query = AsQueryable(context, visitor.BaseObject, visitor.Navigation);
return (IQueryable<T>)query;
}
public static int Count<T>(this DbContext context, Expression<Func<IEnumerable<T>>> expression) =>
AsQueryable(context, expression)?.Count() ?? 0;
Enabling strongly typed usage like the following;
public int TotalStudents(DbContext context) =>
context.Count(() => this.Students);
public int ActiveStudents(DbContext context) =>
context.AsQueryable(() => this.Students)?.Where(s => s.Active).Count() ?? 0;

Is it possible to store, or pre-define a LINQ query for re-use?

I currently have a linq query that i use about 5 times within the same MVC class, is it possible to define the query somewhere within the page in the same way you can public const a string or an int, without having to create a method which calls the linq query?
i.e.
const LinqQuery myQuery = from cat in db.Categories where cat.CategoryID != null select cat;
...
public ActionResult Edit(long id = 0)
{
ViewBag.ParentCategoryID = myQuery;
...
}
public ActionResult Create()
{
ViewBag.ParentCategoryID = myQuery;
...
}
From what i can see, the only way is creating a method, but i would like to avoid it if there is a nicer way of doing things.

My crystal ball tells me that Linq queries are only executed when you iterate over the result so:
static List<int> list = new List<int> { 1, 2, 3 };
static IEnumerable<int> result = from i in list where i > 2 select i;
static void Main(string[] args)
{
Console.WriteLine(result.Sum()); // 3
list.Add(5);
Console.WriteLine(result.Sum()); // 8
}

MSDN Has an article on precompiled queries:
http://msdn.microsoft.com/en-us/library/bb399335.aspx
In your example it could look something like this:
public static readonly Func<MyContext, int, IQueryable<Category>>
myQuery = CompiledQuery.Compile((MyContext db, int categoryID) =>
from cat in db.Categories where cat.CategoryID == categoryID select cat);
I added readonly as it is going to be the closest you'll get to a constant.

You're in luck, sorta, While it's similar, you can declare a Func<> and call that. It looks cleaner / nicer than a method when you only have a very simple instruction you want to execute.
Here's sorta an example on their implementation (This is from a project where I had to use SetWindowPos multiple times along with some other stuff):
Func<Process, winPosData, bool> swp = (p, w) => SetWindowPos(p.MainWindowHandle, (IntPtr)w.hWndInsertAfter, w.x, w.y, w.cx, w.cy, w.uFlags);

Yes, just put it in the constructor (I assume the type of db.Catagories is Catagory)
public class ClassName
{
public ClassName()
{
db = //do whatever is needed to initialize the db context
myQuery = from cat in db.Categories where cat.CategoryID != null select cat;
}
DataSourceContext db;
IQueryable<Catagory> myQuery;
public ActionResult Edit(long id = 0)
{
ViewBag.ParentCategoryID = myQuery;
...
}
public ActionResult Create()
{
ViewBag.ParentCategoryID = myQuery;
...
}
}
This could also be done in the static constructor with static members if access to db is done in a thread safe manner (via internally inside DataSourceContext or by you in defensive coding)
However a better way would be to call a static function that returns the query you want and passing in the context of the database you want to connect to.
public static IQueryable<Catagory> NonNullCatagoriesQuery(DataSourceContext db)
{
return from cat in db.Categories where cat.CategoryID != null select cat;
}
Then in your code you just do
public ActionResult Edit(long id = 0)
{
ViewBag.ParentCategoryID = NonNullCatagoriesQuery(db);
...
}
public ActionResult Create()
{
ViewBag.ParentCategoryID = NonNullCatagoriesQuery(db);
...
}
You also could move the function in to a repository class that held the db context too so you would not need to pass it in, but you did not include in your code example how you got db.

How to apply the same query to different entities?

So, I'm beggining to use EF, and I'm developing an application using it as ORM. The thing is I haven't had much time to dig into the documentation (I Plan to, in due time)and I'm kind of lost in some things. For example, I have these two queries:
public static int GetNextPadlockNumber()
{
LockersDBEntities1 entities = new LockersDBEntities1();
var query = (from p in entities.PadLocks select p.PadlockNumber).DefaultIfEmpty(0).Max();
return (int)query + 1;
}
public static Data.PadLock GetPadLockByNumber(int number)
{
Data.LockersDBEntities1 entities = new LockersDBEntities1();
var query = (from p in entities.PadLocks where p.PadlockNumber == number select p).FirstOrDefault();
return query;
}
and
public static int GetNextLockerNumber()
{
LockersDBEntities1 entities = new LockersDBEntities1();
var query = (from l in entities.Lockers select l.LockerNumber).DefaultIfEmpty(0).Max();
return (int)query+1;
}
public static Data.Locker GetLockerByNumber(int number)
{
Data.LockersDBEntities1 entities = new LockersDBEntities1();
var query = (from l in entities.Lockers where l.LockerNumber == number select l).FirstOrDefault();
return query;
}
And the thing is They're Exactly the same query. Isn't there Any way to do this without having to specify that I want a locker or a padlock? Thanks in advance heroes.

You could have used the LockerNumber and PadlockNumber as Identity Autoincrement fields since that is what you are doing, and then named them the same (Id for instance).
And that way the need to "get next locker number" is no longer necessary since every new entity inserted will get the next available number and with a Repository pattern you could have those methods as common methods in the Base Repository class as something like this:
public IEntity GetEntityById(int number)
{
return ObjectSet.Single(x => x.Id == number);
}

One of the great things with EF is that you can do things like that. It's not trivial, and it will take some trial and error, but you can abstract a lot of the database complexities away.
Now, assuming that LockersDBEntities1 is a DbContext, you could something like this:
public static class LockersDBEntities1Extensions
{
public static int GetNextNumber<T>(
this LockersDBEntities1 context,
Expression<Func<T, int>> getNumberExpression)
{
var query = (from item in context.Set<T>
select getNumberExpression(item))
.DefaultIfEmpty(0)
.Max();
return (int)query + 1;
}
}
and use it like:
int nextPadlockNumber = new LockersDBEntities1()
.GetNextNumber<Padlock>(p => p.PadlockNumber)
and
int nextPadlockNumber = new LockersDBEntities1()
.GetNextNumber<Locker>(l => l.LockerNumber)
The getNumberExpression expression is needed because there is no common way to access the number across all entities. It might be overdesign, but if that is an issue I would do something like this:
//this should be a better name, to reflect the real scenarion
interface ILockNumberProvider
{
int Number {get; }
}
and implement that interface on Locker, Padlock and other classes that need to provide lock numbers. Then, in the method above the expression can be omitted, and a generic constraint can be used, like:
public static class LockersDBEntities1Extensions
{
public static int GetNextNumber<T>(this LockersDBEntities1 context)
where T:ILockNumberProvider
{
var query = (from item in context.Set<T>
select item.Number)
.DefaultIfEmpty(0)
.Max();
return (int)query + 1;
}
}

Confuse with Generics and Entity Framework

I am new to c# and now bit confused with Generics and Entity Framework. I have two tables in database, which I retrieve in my code using Entity Framework. On my aspx page, I have a grid which should populate the data based on what table user selects.
In future there will be more tables. So I wanted to write a factory pattern to get the source list for datagrid. I can not make it to work because i am very confused.
Here is my code for BaseClass and two child classes.
static class Factory
{
public static ReportBase GetReport(string name)
{
switch (name)
{
case "Child1":
return new Child1();
case "Child2":
return new Child1();
default:
return null;
}
}
}
//Base Class
class ReportBase<T>
{
public List<T> _list;
public abstract void Load();
public abstract List<T> Filter(DateTime statrtDate, DateTime endDate);
}
//Child 1
class Child1 : ReportBase
{
public List<GetChild1> _list;
public Child1(){}
public override void Load()
{
//GetChild1 is the name of database table
var info = from p in Context.GetChild1 select p;
_list = info.ToList();
}
public List<GetChild1> Filter(DateTime startDate, DateTime endDate)
{
var filteredValues = from p in _list where p.UploadedDate <= startDate select p;
return filteredValues.ToList();
}
}
//Child 2
class Child2 : ReportBase
{
public List<GetChild2> _list;
public Child2() { }
public override void Load()
{
//GetChild2 is the name of database table
return (from p in Context.GetChild2 select p).ToList();
}
public List<GetChild2> Filter(DateTime startDate, DateTime endDate)
{
return (from p in _list where p.UploadedDate <= startDate select p).ToList();
}
}
Can Someone please correct the code accordingly? Do I have to use Generics here? I tried using it in BaseClass but it doesn't work properly, because I have to fix my child classes accordingly, for which I have no clue.

First of all, with generics you can write better code that is more readable and short.
I think it is better not to use a factory class for repositories in the end you need to know which type you are dealing with it in your code such as:
Child1 report = new Child1();
report.SomeBehaviorInChild1(); // this is not achievable with base class.
If you want to use this factory you can do the following:
ReportBase report = Factory.GetReport();
Secondly, it is a bad idea to let your list public because you don't want to those who are using your class to create a new list from their code such as:
report._list = new List();
You don't want this thing to happen to the objects in your class. So it is better to have your list private and depend only on the methods to return the data source of your report.
private List _list;
public List<T> GetDataSource()
{
return _list;
}
Thirdly, if you implement a generic reportbase you won't need to write the child classes unless they have something special that the base doesn't implement.
Fourthly, the current implementation of Filter method is soooo bad because what you are doing here is getting all the records from database then filtering them in memory. This implementation is a bad practice. A better approach is to use the IQueryable which is a deffered execution object i.e. the result won't be populated until you request it.
public List<GetChild1> FilterExample()
{
IQueryable<GetChild1> result = _context.GetChild1;
result = from p in result
where p.UploadDate < startDate;
select p;
//until this moment the query is still not send to the database.
result = result.OrderBy(p => p.UploadDate);
//when you call the ToList method the query will be executed on the database and the list of filtered and sorted items will be returned. Notice the sorting and filtering is done in the database which is faster than doing it in memory
List<GetChild1> populatedResult = result.ToList();
return populatedResult;
}
So this was a better approach to your problem. I think it is good for you to read a book called "More Effective C# 2008" it talks about queryable and linq in general.
Now if we apply this on your BaseReportClass we can get the following:
//Base Class
class ReportBase<T> where T: class
{
private DbContext _context;
public ReportBase(DbContext context)
{
_context = context;
}
//I think GetAll is a more suitable name for your method than load :D
public IQueryable<T> GetAll()
{
return _context.Set<T>();
}
public IQueryable<T> Filter(Func<T,bool> filterFunction)
{
var result = from p in GetAll()
where filterFunction(p)
select p;
return result;
}
}
Now to defining the additional behavior for the child classes.
//Child 1
class Child1 : ReportBase<GetChild1>
{
public ReportBase(DbContext context):base(context)
{
}
public List<GetChild1> FilterOnStartDate(DateTime startDate)
{
//I don't know if you are familiar with lambda expressions but if you are not you can research it on the internet.
//The parameter here is converted into a method that its parameter is "p" of type GetChild1
// and the body is "p.UploadDate < startDate".
// Note you can remove the type of the parameter because the compiler can predict it.
return Filter((GetChild1 p) => p.UploadDate < startDate).ToList();
}
}
This is the code that uses your classes:
Child1 report = new Child1(new GetChild1Entities());
report.FilterOnStartDate(DateTime.Now);
Hope this was useful.

Generics for defining how to display each entity type (or table) is the wrong approach.
Instead, using a pattern such as MVVM or MVC, where according to the type of the data (the model) you select a view.
MVVM
MVC
MVC for ASP.NET

EF + generic repository + loading related entity: only explicit load works

I have two simple POCO classes; I'm trying to get the MyY property below hydrated with the an instance of Y. I've tried a number of ways to do this, and think I might be missing something obvious or simple.
public class X
{
public int Id { get; set;}
public virtual Y MyY { get; set; }
}
public class Y
{
public int Id { get; set; }
// ...
}
I've turned lazy loading off via this call in my subclass of DbContext's constructor:
Configuration.LazyLoadingEnabled = false;
When retrieving an X I have tried
context.Set<X>.Include("MyY").FirstOrDefault(x => ....);
which did not work. I tried
var result = context.Set<X>.FirstOrDefault(x => ....);
context.Entry(result).Reference("MyY").Load();
which works, but requires two round-trips to the database. I tried
context.Set<X>.Select(x => new { X = x, Y = x.MyY }).FirstOrDefault(x => ...);
which also works, but "weakens" my model (ordinarily projecting to a new type is not so bad, but the "shape" of these EF POCOs works perfectly for the DTOs I'll be sending through WCF later).
I finally tried removing virtual from the MyY property as suggested in an answer to another question, but that had no effect at all.
Finally, I want to use the generic repository pattern. What I have ended up with is the following design, shown in part, which supports explicit-load (not preferred) and eager-load when modified to work properly. How do I modify it to get the single db round-trip eager-load?
public class EFRepository : IRepository
{
public T Get<T>(Specification<T> specification) where T : class, IEntity
{
var result = ApplyEagerLoading(context.Set<T>()).FirstOrDefault(specification.IsMatch);
ApplyPostQueryLoading(new List<T> { result });
return result;
}
// doesn't really seem to work yet...
private DbSet<T> ApplyEagerLoading<T>(DbSet<T> set) where T : class, IEntity
{
var ls = loadSpecs.GetOrAdd(typeof(T), () => new List<LoadSpec>());
foreach (var spec in ls.Where(s => !s.ExplicitLoad))
set.Include(spec.PropertyName);
return set;
}
// works, but wrong on so many levels...
private void ApplyPostQueryLoading<T>(IEnumerable<T> entities) where T : class, IEntity
{
var ls = loadSpecs.GetOrAdd(typeof(T), () => new List<LoadSpec>());
foreach (var e in entities)
foreach (var spec in ls.Where(s => s.ExplicitLoad))
if (spec.IsCollection)
context.Entry(e).Collection(spec.PropertyName).Load();
else
context.Entry(e).Reference(spec.PropertyName).Load();
}
private readonly IDictionary<Type, IList<LoadSpec>> loadSpecs = new Dictionary<Type, IList<LoadSpec>>();
private class LoadSpec
{
internal string PropertyName;
internal bool ExplicitLoad;
internal bool IsCollection;
}
}
Example uses:
// add a rule to load MyY explicitly
repository.AddLoadRule<X>(x => x.MyY, explicit:true, isCollection:false)
...
var x = repository.Get<X>(new Specification<X>(x => x.Id == 5));
// add a rule to load MyY with X
repository.AddLoadRule<X>(x => x.MyY, explicit:false)
...
// x.MyY will be null! Doesn't work!
var x = repository.Get<X>(new Specification<X>(x => x.Id == 5));
An Update Based on The Answer:
It turns out my temp code examples lied (those one-liners above). I had actually cached the result of .Include in a local variable but applied the .FirstOrDefault against the .Set<X> not the result of .Include. Here is the fix to ApplyEagerLoading, which mirrors what others have suggested in related questions:
private IQueryable<T> ApplyEagerLoading<T>(IEnumerable<T> set) where T : class, IEntity
{
var ls = loadSpecs.GetOrAdd(typeof(T), () => new List<LoadSpec>());
var query = set.AsQueryable();
return ls.Where(s => !s.ExplicitLoad).Aggregate(query, (current, spec) => current.Include(spec.PropertyName));
}

This should work:
X entity = context.Set<X>().Include(x => x.MyY).FirstOrDefault();
IF it doesn't the problem must be elsewhere.
If you need some eager loading strategy check this answer.