I am in the middle of rewriting legacy software to .Net and have created a Data Access Layer using Linq 2 SQL that is working pretty well. The database I am working with has over 230 tables and is not something I can really change. The problem I am running into is with the Business Layer. I would like for the developers to be able to query the business objects and have those queries map to the data layer. So something like Customers.Query(c=>c.ID=="MyID")
and have that be able to be passed to my DAL, context.CUSTOMERS.Query(c=>c.UID == "MyID")
I have generic Query methods in my DAL that will allow me to pass in the DAL query.
This is where I am stuck. I can create a method that uses an Expression but how do I get and then map those fields to the corresponding DAL fields and get the value that is trying to be matched. What I don't want is to have to expose the DAL objects to the end developers who are doing presentation layer stuff. I am open to ideas and suggestions.
Do the developers need to query the business objects using an expression? The reason I ask is because mapping expressions may be complicated. At some point, some mapping has to be done. The DAL is usually the mapping layer that turns DB objects into domain objects, and vice-versa.
Two other approaches to consider:
Use the repository pattern, where the caller passes in a query object. The DAL would be responsible for turning that query object into an expression. An example of the repository pattern is shown in a question that I asked.
Expose more specific methods, like:
public Customer GetCustomersById(int id) { ... }
I believe either of those two approaches would make things easier to query.
So I think I have been able to find a solution to this. Using ExpressionVisitor and help from this post
I modified the VisitMember method:
protected override Expression VisitMember(MemberExpression node)
{
string sDbField = ((SFWBusinessAttributes)node.Expression.Type.GetProperty(node.Member.Name).GetCustomAttribu`tes(typeof(SFWBusinessAttributes), true)[0]).DBColumn;
var expr = Visit(node.Expression);
if (expr.Type != node.Type)
{
MemberInfo newMember = expr.Type.GetMember(sDbField).Single();
return Expression.MakeMemberAccess(expr, newMember);
}
return base.VisitMember(node);
}
To pull the Attribute off of the Property of the Business Object.
To call everything I can do from the app:
BusObj.Query(a => a.IsDeleted != true && a.Company.Contains("Demo"))
Which calls a method in the business object
public List<Account> Query(Expression<Func<Account, bool>> expression)
{
using (Data.CustomerData data = new Data.CustomerData(_connstring))
{
return MapList(data.Query<Data.Database.ACCOUNT>(expression.Convert<Account, Data.Database.ACCOUNT>()).ToList());
}
Performance seems pretty good, I know there is going to be a hit with the mapping but it is something I can live with for now.
Related
I have a long-time burning question about how to avoid null errors with data queried via Entity Framework (version 6 - not Core yet, sadly).
Let's say you have a table Employees, and it has a relationship with another table, EmployeePayments (one employee has many employee payments).
On your Employee domain object you create a property TotalPayments which relies on you having loaded the EmployeePayments for that object.
I try to ensure that any time I do a query, I "include" the dependency, for example:
var employees = context.Employees.Include(e => e.EmployeePayments);
The problem is, I have a lot of queries around the place (I use the generic repository pattern, so I call repository functions like GetAll or GetSingle from my service library), and so that's a lot of places to remember to add the includes. If I don't include them, I run the risk of having a null exception if the TotalPayments property is used.
What's the best way to handle this?
Note 1: we have a lot of tables and I don't really want to have to revert to using specific repositories for each one, we take advantage of the generic repository in a lot of ways.... but I will be happy to hear strong arguments for the alternative :)
Note 2: I do not have lazy loading turned on, and don't plan on turning it on, for performance reasons.
This is one reason I consider the Generic Repository an anti-pattern for EF. I use a repository pattern, but scope it like I would a controller. I.e. a CreateOrderController would have a CreateOrderRepository. This repository would provide access to all relevant entities via IQueryable. Common stuff like lookups etc. would have their own secondary repository. Using generic repositories that are geared to working with a single entity type mean adding references to several repositories to do specific things and running into issues like this when attempting to load entities. Sometimes you want related data, sometimes you don't. Simply adding convenient methods in top level entities effectively "breaks" that an object should always be considered complete or complete-able without relying on lazy-loading which brings significant performance costs.
Having repositories return IQueryable avoids many of the problems by giving control to the calling code how entities are consumed. For instance I don't put helper methods in the entities, but rather code needing to populate a view model relies on Linq to build the view model. If my view model wants a sum of payments for an employee, then my repository returning IQueryable can do the following:
public IQueryable<Employee> GetEmployeeById(int employeeId)
{
return Context.Employees.Where(x => x.EmployeeId == employeeId);
}
then in the controller / service:
using (var contextScope = ContextScopeFactory.Create())
{
var employeeViewModel = EmployeeRepository.GetEmployeeById(employeeId)
.Select(x => new EmployeeSummaryViewModel
{
EmployeeId = x.EmployeeId,
EmployeeName = x.LastName + ", " + x.FirstName,
TotalPayments = x.Payments.Where(p => p.IsActive).Sum(p => p.Amount)
}).Single();
}
I use a repository because it is easier to mock out than the DbContext and it's DbSets. For Synchronous code I just have the mock to populate and return List<Employee>().AsQueryable(). For Async code I need to add a wrapper for an Async List.
This pattern may go against more traditional views of a repository and separation of concerns that the calling code needs to "know" about the entities, that EF-isms are leaked. However, no matter what approach you try to rationalize to get around the inefficiencies of trying to "hide" EF behind a repository, either you will be left with very inefficient code where repositories return pre-populated DTOs or contain dozens of near identical methods to return different DTOs (or worse, entities in various degrees of completeness) or you are adding complexities like passing in magic strings or expression trees into your methods to tell EF how to filter, how to sort, what to include, paging, etc. Passing in expressions or strings requires the calling code to "know" about the entities and leaks EF restrictions. (Passed in expressions / strings still have to be able to be ultimately understood by EF)
So this may not be a viable answer to the current state of your project, but it might be worth looking into whether your dependency on the repositories can be better managed without splitting them with the Generic pattern, and/or leveraging EF's excellent IQueryable / Linq capabilities to let your controllers/services project the entities into view models / DTOs rather than embedding these reduce elements in the entities themselves.
I have inherited a code base that uses DTOs in the business layer, these are populated via a set of mappers from Entity Framework. This has some quite serious limitation in terms of querying so I am working on a new "optimised" querying service.
My first issue is that I need to translate my LINQ query on my DTO to work with my Entity object but the calling context has no knowledge of the EF entities. Let's assume we can rely on the properties on each object having matching names.
This is where I have got to in terms of stubbing out what I want:
public static List<TDataObject> GetFiltered<TDataObject(Expression<Func<TDataObject, TDataObject>> projection, Func<TDataObject, bool> filter)
{
// 1. translate the filter parameter to work with my equivalent Entity object
// 2. build the EF query with the modified filter expression and also a Select() projection so we only return the properties we need. (this should generate an optimised SQL query under the hood)
// 3. map the results from the EF query back onto my TDataObject and return (I already have AutoMapper maps in place for this)
}
It is item 1 that I am struggling with so if anyone has any code examples for blogs posts for achieving this I'd appreciate it.
Also if anyone has any alternate suggestions I'd be happy to hear them.
One way to handle this is to build primitives around queries (instead of layers with repositories etc). Here's what we do:
http://lostechies.com/jimmybogard/2013/10/29/put-your-controllers-on-a-diet-gets-and-queries/
The calling code (controller) knows about a query and the result (DTO), but the piece doing the mapping knows exactly about EF Context/NHibernate ISession. Works very well for us and keeps our controllers light and thin.
Alternatively, get rid of your layers, and expose the EF objects directly:
var dtos = context.Employees.Where(e => e.IsActive)
.Project().ToArray<EmployeeShowViewModel>();
Put this in the controller because who cares, layers and abstractions are productivity preventers and time wasters.
The other day I asked this question:
Should the repository layer return data-transfer-objects (DTO)?
The answer (well by just one person, but I already had a hunch that it wasn't a good idea) was that no, the repository later should not have to deal with the DTO objects (their purpose is purely to be sent over the wire) and the service layer should deal with that.
Now I've come up with a construction in the meantime that I need your opinion on. The idea is that, when it makes sense to do so, the repository layer can return an interface type I've defined called IProjectable. This wraps the query (the repository layer does not execute the query yet) but does not allow the consumer to change the query (it's not IQueryable), just to perform projection operations on it (so far for me only First and ToPagedList) that would perform the projection and actually execute the query.
So something like this in the repository:
public IProjectable<User> GetUser(int id)
{
var query = from u in Set<User>()
where u.UserID == id
select u;
return query.AsProjectable();
}
And in the service layer something like this:
var dto = repository.GetUser(16).Single(u => new SimpleUserDto
{
FullName = u.FirstName + " " + u.LastName,
DisplayAddress = u.Address.Street + u.Address.HouseNumber,
OrderCount = u.Orders.Count()
});
return dto;
Am I correct in saying that doing the actual data access here is still the responsibility of the repository layer (as it should be) and that the projection to a serializable form is the responsibility of the service layer (as it should be)?
The only other way I see to do this efficiently (returning a User from the repository and doing the Count() on his Orders in the service layer would result in an extra query to the database) is to define a type that has all these properties and return it from the repository layer and just don't call it "Dto", which seems silly as it would be identical to the DTO just not named the same for the sake of "purity". This way, it seems, I can have my cake and eat it for the most part too.
The downside I see is that you can get a mismatch where the service layer performs projections that can't actually be translated to SQL which it shouldn't have to worry about, or where it performs such complex projections that makes it questionable what layer is doing the actual data access.
I'm using Entity Framework 4 by the way, if it matters.
Am I correct in saying that doing the
actual data access here is still the
responsibility of the repository layer
(as it should be) and that the
projection to a serializable form is
the responsibility of the service
layer (as it should be)?
Yes you are, the service layer still has no idea how the actual DataAccess is being performed (as it should not have not). Are the calls send to SQL? is there a caching layer in between?
The downside I see is that you can get
a mismatch where the service layer
performs projections that can't
actually be translated to SQL which it
shouldn't have to worry about, or
where it performs such complex
projections that makes it questionable
what layer is doing the actual data
access.
For this problem i use a pipeline pattern which basicly is just a set of extension methods over IProjectable which can perform tested projections. Next, in your serviceLayer you can just write your query using a composition of these pipeline methods, for example:
var users = repository.GetUsers().FilterByName("Polity").OrderByAge().ToTransferObjects();
One of developers I most respect ayende (http://ayende.com/Blog/Default.aspx) said: "ORM is your repository" video here -> http://community.devexpress.com/blogs/seth/archive/2011/03/09/interview-with-ayende-rahien-aka-oren-eini.aspx
Question is do you really need Repository pattern?
Just my opinion :)
I'm kinda stuggling with this being best practice or not..
I have a repository that returns an IQueryable lets say. Is this correct usage in the controller?
var whatever = ObjectRepository.GetWhatever(id);
var videoId = whatever.UsersInObject1InObjects2.First().Object.Video.ExternalVideoId;
Where in the 2nd line above ".Object" and ".Video" are references to tables that are related to "whatever" table.
Or should I be making another function in a different repository to get the ExternalVideoId?
The way I usually do this is this. I create a separate model class that encapsulates data that the controller need from the database. (I.e. as a rule I do not use ORM class for this). This allows me annotating the model class members with all sort of attributes I might need. In the repository code I query the ORM and then return the model class (or IQueryable/IEnumerbale of model) as the result.
I've heard an opinion that returning IQueryable from an ORM such as Entity Framework is not advised, because by doing this, you risk executing a query against your back end several times if you are not careful with your model in controller/view. This is because in EF IQueryable represent a query that is not executed yet. By returning IEnumerable instead of IQueryable you make sure that query execution is confined to your repository class.
I however find, that sometimes it's more convenient to return IQueryable. For example, for table paging/sorting scenarios I can apply page number, sort direction, page size, etc to the IQueryable before executing it. I feel that this logic belongs more to controller rather then to repository. Some may disagree.
What you're actually doing is using a collection to do a query.
I think you need to have a better look at linq queries.
You don't want to add custom queries to your repository.
You just want to expose the queryable interface like so..
public IQueryable<T> Get( IUser identity )
{
return Context.Set<T>();
}
public IQueryable<IBindable> GetItem( IUser identity )
{
return Context.Set<T>().Cast<IBindable>();
}
then you can use linq to sql
Disclaimer: I'm pretty new to DDD and its associated terminology, so if i'm mislabeling any concepts, please correct me.
I'm currently working on a site with a relatively simple domain model (Catalog items, each of which stores a collection of CatalogImage items.)
My repository follows the standard interface of FindbyID(int ID) GetAll() etc...
The problem arises when trying to find a particular image by its ID; I end up with methods such as FindImagebyID(int CatalogItemID, int ImgID)
As new requirments develop, and the object graph becomes more heavily nested, I could see an explosion of methods such as Find{NestedType}ByID(int catalogItemID,.....,int nestedTypeID)
Should I simply be returning an IEnumerable from the FindAll() method, and using Linq in a higher layer to form these queries? Or will that be a violation of SoC?
It sounds to me like you have a justification for building multiple repositories.
Example
interface CatalogRepository
{
Catalog FindByID(int ID);
}
interface CatalogImageRepository
{
CatalogImage FindByID(int ID);
}
This will properly separate out your concerns, since each repository is only responsible for knowing how to deal with that specific entity.
I would filter the model at a layer above the repository, with LINQ if you like. Makes the repository simple. If you are using LINQ to get the data from the database this method works very well, if you are having to use ADO or some other legacy data access layer than it might make it more difficult to make the repository so simple. Linq makes it easy so that you can have the repository return IQueryable and let the next layer add the filtering and the actual retrieval of data does not happen until it is asked for. This makes it possible to have a method on the repository like GetImages() that gets all images, and the next layer adds the filtering for a specific image. If you are using ADO, you are probably not going to want to bring back all images then filter....so could be a trade off.