We Keep Coding

To Repository or Not To Repository

When I first learnt about Domain Driven Design, I was also introduced to the repository and unit of work patterns that once seemed to be top notch for the cool kids that threw SQL queries like cavemans against databases. The deeper I got into that topic, the more I learnt that they don't seem to be necessary anymore because of ORMs like EF and NHibernate that implement both unit of work and repositories into one API, called session or context.
Now I'm unsure what to do. To repository or not to repository. I really understand the argument that such leaky abstractions only over-complicate things while adding absolutely nothing that may simplify data access, however, it doesn't feel right to couple every possible aspect of my application to e.g. Entity Framework. Usually, I follow a few simple guidelines:
The domain layer is the heart of the system, containing entities, services, repositories...
The infrastructure layer provides implementations of domain interfaces of a infrastructural concern, e.g. file, database, protocols..
The application layer hosts a composition root that wire things up and orchestrates everything.
My solutions usually look like this:
Domain.Module1
Domain.Module2
IModule2Repo
IModule2Service
Module2
Infrastructure.Persistence
Repositories
EntityFrameworkRepositoryBase
MyApp
Boostrapper
-> inject EntityFrameworkRepositoryBase into IRepository etc.
I keep my domain layer clean by using a IRepository<'T> which is also a domain concern not depending on anything else that tells me how to access data. When I now would make a concrete implementation of IModule2Service that requires data access, I would have to inject DbContext and by this, coupling it directly to the infrastructure layer.
(Coming to Visual Studio project, this can end up really tricky because of circular dependencies!)
Additionally What can be an alternative to depositories and fucktons of works? CQRS? How does one abstract a pure infrastructural framework?

"Depository" lol....
Anyways, you got right the bit that you don't want the domain coupled to EF that's why the T in your repository interface should be domain aggregate root and NOT an EF entity (or a 'domain' object designed to work properly with EF).
Your Domain layer is never coupled to persistence because it only knows about abstractions. When Module2Service needs data access it either uses a DAO (or a repository - not necessarily the DDD version - if it makes sense) or the service itself is an implemented in DAL (if it doesn't contain business logic).
In your case probably the best approach is the DAO/repository which of course will 'hide' the EF part. If it seems you're writing too much code, you really aren't and I think it matters the most to keep the proper separations of concerns than saving 50 LoC (a whole 5-10 minutes).
CQRS is always good idea with a rich domain but as any solution it comes to the drawback that it requires more code (and I understand that every coder is lazy by definition, but we are required to do a 'fuckton' of work, an app doesn't build itself, a maintainable app is even more work at the beginning).
If by abstracting a pure infrastructural framework you mean hiding EF, the repository is your best bet and you don't even have to name the class 'repository', it's the principle that matters and that's what the repo does: abstracts persistence for the Domain.

What am I missing with the Respository Pattern? How do you use it in the real world?

Every example of the Repository Pattern I have seen deals with a very simple use case - one object type and the most basic CRUD operations. The Repository is then very often plugged straight into an MVC controller.
Real-world data access just isn't like this. Real-world data access scenarios can involve complex graphs of objects and some form of transactional wrapper. For example, suppose I want to save a new Order. This involves writing to the Order, OrderDetails, Invoice, User, History and ItemStock tables. All of this must be transacted, committed or rolled back. Normally I'd pass around something like an IDbTransaction and an IDbConnection and bundles the whole operation in a service layer.
Where does the Repository Pattern fit with this? Am I missing something (Unit Of Work perhaps)? Are there any more realistic examples of Repositories in use than the usual canned blog snippets?
Appreciate any light.

This is a very controversial subject but here is my catch from my own experience.
Repository works at the aggregate root. For example if an OrderItem is always retrieved as part of Order and does not have a life of its own outside order, then it will be loaded by the OrderRepository, otherwise it will have its own repository.
UnitOfWork is an important concept. Let's say OrderItem is an aggregate root and has its own repository. So at the time of creating an order, OrderManager will create a UnitOfWork of work in a using block, initialise OrderItemRepository and OrderRepository and then commit.
UPDATE
Yes, exactly. Imagine - in our case order - an order is being inserted. This needs to be in control of the transaction and enter order and order items separately inside the same transaction. This cannot be managed at the repository level. This is the sole reason for existence of UnitOfWork concept which is passed to the repository so that it does not own or initialise it. UnitOfWork usually is created at the business layer.

O/R-Mappers like Hibernate basically implement the Repository pattern for object graphs while fully supporting transactions. It's often a leaky abstraction, but it certainly can be made to work in complex real-world scenarios.

If you need a good full blown, widely used expample of the repository pattern look at Cocoa's Core Data I realize it is not in the realm of programming languages that you note. But note that it is NOT an O/R mapper. It is a complete abstraction of an object store. No Sql statements to execute, and while you may pick the format of the external storage that is used, you never interact with it directly.

I like to think of a repository as another layer of abstraction. You should only add layers of abstraction when the cost of their implementation is less than the cost of NOT doing the implementation (code maintenance, support, enhancements, etc).
Remember that the purpose of the repository pattern is to separate the logic that retrieves the data (CRUD) and maps it to the entity model from the business logic that acts on the model. What this often ends up doing/looking like in the real world is some form of business entities, that abstract the underlying physical data model.
As far as your transaction question, yes, this relates more to the Unit of Work pattern. Since you mentioned services, I would encourage you to NOT pass around your connection to your various data access classes/methods, but to instead allow WCF to manage the transaction for you using auto enlistment. Here is an extract of Juval Lowy's WCF book (highly recommended) that explains the how and why of this method of transaction management.
So to answer your question, the repository pattern fits in as a way to abstract the physical data model and to separate the CRUD/mapping from the business logic.

n-layer business/service layer design

I’m trying to reevaluate our n-layer architecture and would love to get some suggestions based on your experiences. Here is our typical .NET n-layer (sometimes n-tier) design.
Project.UI
Project.Services
Project.Business
Project.Model
Project.DataAccess
DataAccess typically consists of Entity Framework 4 and Repository classes. I attempt to follow the Aggregate Root concept in order to avoid having a repository for table, easier said than done in my experience. I tend to have ~70% match between Repositories and Tables.
Model usually consists of my Entity Framework 4 entities, I've been using Self-Tracking EF entities with success.
Business is what I struggle with the most. I typically have a Manager class for every Repository. This class will contain methods like .Add() which will perform business validation before forwarding down to repository.Add().
Services, typically I will only implement this if in fact I am looking to create a web service based solution. This layer will be tasked with marshaling requests/responses between DTOs and entities. And most importantly provide the more coarse grained interface. For example a TradingService.SubmitTrade(), which is really a facade for a business transaction which might include AccountManager.ValidateCash(), OrderManager.SubmitOrder(), etc.
Concerns
My business layer is very entity centric, really it's just the glue between the entities and the repository, with validation in between. I've seen many designs where the Service Layer is what holds a reference to the repositories (in essence skipping the "business layer"). In essence it serves the same purpose as my Business layer, it does the validation, however its' responsibility (and naming) is a higher level, more coarse grained business transaction. Using the example above the TradingService.submitTrade() will not delegate to any business manager classes, it would itself query the necessary repositories, perform all the validation etc.
I like my design in a sense that I can reuse a business layer method in multiple service calls, however I hate the fact that for every repository I have a matching business layer manager, creating tons of extra work. Maybe the solution is a different type of grouping at the Business Layer level? For example combine individual Manager classes like PhoneManager and EmailManager (note I have Phone entities and Email entities) into a logical Manager class such as ContactsManager (note I don't have a "Contact" entity type). With methods such as ContactManager.GetPhones() and ContactManager.GetEmail(), etc.
I guess more than anything I am wondering how others organize and delegate responsibilities, whether they have the Service layer, Business layer, both, etc. What holds the ORM context reference, etc.

I tend to do what you outlined near the end of your concerns, and at the business layer group things into managers that make more logical sense from a business point of view.
Using Contacts for example, I certainly wouldn't have a PhoneManager or EmailManager. "ContactsManager" is a more useful grouping to me, accomplishing pretty much the same thing only with a lot fewer managers to deal with. From a business point of view, phone numbers and emails are just small pieces of a Contact anyway. Just because they have their own tables and entities doesn't mean they need their own manager. That doesn't mean that you can't reuse them. If you have a need to do work with email addresses in several places, ContactsManager can have the relevant methods.
What we tend to have in our environment is a database/entity layer, then a business layer that sits on the server and handles business rules and business logic. That business layer is exposed as a service via WCF to the client (or at least, stuff relevant to clients is).
It sounds like you already know what you want to do, so I'd suggest doing a bit of prototyping work on it and see how it works for you. :)

Is DTO plus UnitOfWork pattern a good approach to design a DAL for a web application?

I'm implementing a DAL using entity framework. On our application, we have three layers (DAL, business layer and presentation). This is a web app. When we began implementing the DAL, our team thought that DAL should have classes whose methods receive a ObjectContext given by services on the business layer and operate over it. The rationale behind this decision is that different ObjectContexts see diferent DB states, so some operations can be rejected due to problems with foreign keys match and other inconsistencies.
We noticed that generating and propagating an object context from the services layer generates high coupling between layers. Therefore we decided to use DTOs mapped by Automapper (not unmanaged entities or self-tracking entities arguing high coupling, exposing entities to upper layers and low efficiency) and UnitOfWork. So, here are my questions:
Is this the correct approach to design a web application's DAL? Why?
If you answered "yes" to 1., how is this to be reconciled the concept of DTO with the UnitOfWork patterns?
If you answered "no" to 1., which could be a correct approach to design a DAL for a Web application?
Please, if possible give bibliography supporting your answer.
About the current design:
The application has been planned to be developed on three layers: Presentation, business and DAL. Business layer has both facades and services
There is an interface called ITransaction (with only two methods to dispose and save changes) only visible at services. To manage a transaction, there is a class Transaction extending a ObjectContext and ITransaction. We've designed this having in mind that at business layer we do not want other ObjectContext methods to be accessible.
On the DAL, we created an abstract repository using two generic types (one for the entity and the other for its associated DTO). This repository has CRUD methods implemented in a generic way and two generic methods to map the DTOs and entities of the generic repository with AutoMapper. The abstract repository constructor takes an ITransaction as argument and it expects the ITransaction to be an ObjectContext in order to assign it to its proctected ObjectContext property.
The concrete repositories should only receive and return .net types and DTOs.
We now are facing this problem: the generic method to create does not generate a temporal or a persistent id for the attached entities (until we use SaveChanges(), therefore breaking the transactionality we want); this implies that service methods cannot use it to associate DTOs in the BL)

There are a number of things going on here...The assumption I'll make is that you're using a 3-Tier architecture. That said, I'm unclear on a few design decisions you've made and what the motivations were behind making them. In general, I would say that your ObjectContext should not be passed around in your classes. There should be some sort of manager or repository class which handles the connection management. This solves your DB state management issue. I find that a Repository pattern works really well here. From there, you should be able to implement the unit of work pattern fairly easily since your connection management will be handled in one place. Given what I know about your architecture, I would say that you should be using a POCO strategy. Using POCOs does not tightly couple you to any ORM provider. The advantage is that your POCOs will be able to interact with your ObjectContext (probably via Repository of some sort) and this will give you visibility into change tracking. Again, from there you will be able to implement the Unit of Work (transaction) pattern to give you full control over how your business transaction should behave. I find this is an incredibly useful article for explaining how all this fits together. The code is buggy but accurately illustrates best practices for the type of architecture you're describing: Repository, Specification and Unit of Work Implementation
The short version of my answer to question number 1 is "no". The above link provides what I believe to be a better approach for you.

I always believed that code can explain things better than worlds for programmers. And this is especially true for this topic. Thats why I suggest you to look at the great sample application in witch all consepts you expecting are implemented.
Project is called Sharp Architecture, it is centered around MVC and NHibernate, but you can use the same approaches just replacing NHibernate parts with EF ones when you need them. The purpose of this project is to provide an application template with all community best practices for building web applications.
It covers all common and most of the uncommon topics when using ORM's, managing transactions, managing dependencies with IoC containers, use of DTOs, etc.
And here is a sample application.
I insist on reading and trying this, it will be a real trasure for you like it was for me.

You should take a look what dependency injection and inversion of control in general means. That would provide ability to control life cycle of ObjectContext "from outside". You could ensure that only 1 instance of object context is used for every http request. To avoid managing dependencies manually, I would recommend using StructureMap as a container.
Another useful (but quite tricky and hard to do it right) technique is abstraction of persistence. Instead of using ObjectContext directly, You would use so called Repository which is responsible to provide collection like API for Your data store. This provides useful seam which You can use to switch underlying data storing mechanism or to mock out persistence completely for tests.
As Jason suggested already - You should also use POCO`s (plain old clr objects). Despite that there would still be implicit coupling with entity framework You should be aware of, it's much better than using generated classes.
Things You might not find elsewhere fast enough:
Try to avoid usage of unit of work. Your model should define transactional boundaries.
Try to avoid usage of generic repositories (do note point about IQueryable too).
It's not mandatory to spam Your code with repository pattern name.
Also, You might enjoy reading about domain driven design. It helps to deal with complex business logic and gives great guidelines to makes code less procedural, more object oriented.

I'll focus on your current issues: To be honest, I don't think you should be passing around your ObjectContext. I think that is going to lead to problems. I'm assuming that a controller or a business service will be passing the ObjectContext/ITransaction to the Repository. How will you ensure that your ObjectContext is disposed of properly down stream? What happens when you use nested transactions? What manages the rollbacks, for transactions down stream?
I think your best bet lies in putting some more definition around how you expect to manage transactions in your architecture. Using TransactionScope in your controller/service is a good start since the ObjectContext respects it. Of course you may need to take into account that controllers/services may make calls to other controllers/services which have transactions in them. In order to allow for scenarios where you want full control over your business transactions and the subsequent database calls, you'll need to create some sort of TransactionManager class which enlists, and generally manages transactions up and down your stack. I've found that NCommon does an extraordinary job at both abstracting and managing transactions. Take a look at UnitOfWorkScope and TransactionManager classes in there. Although I disagree with NCommon's approach of forcing the Repository to rely on the UnitOfWork, that could easily be refactored out if you wanted.
As far as your persistantID issue goes, check this out

Business Objects and Data Layer

This site has provided me with many useful answers, however after a hours search I haven't found anything that specifically answers my needs. So here goes...
The company I'm working for is in the process of designing a new Business Objects Layer and a Data Access Layer - these will reside in separate assemblies.
The problem is I'm having a hard time getting my head around the interaction between these two layers - specifically, should the DAL know about the BOL, I've read numerous articles that have said the dependency order should go something like this:
GUI / Presentation --> BOL ---> DAL
But as far as I can see, the DAL needs a reference to the BOL in order to be able to 'return' objects to the BOL layer.
I'm going for a intermediate assembly between the BOL and DAL which will be basically a thin layer filled with interfaces to decouple those two DLL's, so the framework can use different DALs if the need arises.
This lead me to the idea of introducing another thin layer with a bunch of interfaces that the BOs implement, then when the BOL calls the DAL interface, it passes it an object which implements one of these BO interfaces and then the DAL proceeds to populate the object. This removes all dependencies between the BOL and the DAL - however, I'm finding it hard to justify it to be honest.
Ideally we would like to use an ORM as it just removes the need to write CRUD stuff but our clients have a habit of fiddling with column lengths on their database and this is the cause of most of our errors to-date using the strongly typed DataTables. I've heard Linq2SQL also stores column lengths at compile time, not sure if NHibernate does or not (but, I'm not sure if our Database Schema is designed cleanly enough for NHibernate, pitfalls of dealing with legacy systems).
So yea, any insight on the relationship between a BOL and a DAL would be very much welcome - apologies if the above is poorly written, if anyone needs clarification I'll be happy to provide more detail.
Marlon

The way the I do this is the BO expects a DataReader or a DataContext or whatever back from the DAL, not the actual formed object. It is then the job of the BO layer to take and fill itself from the object that came back. The DAL isn't returning back a completed BO. The main thing to remember is that changing something in the BO layer shouldn't cause issues for the DAL layer, but changing something in the DAL layer could cause issues for the BO layer.
A short example of what I typically do
In the BO layer
FillData(){
DataReader dr = DataLayer.GetData("SomePropertyForAStoreProcedure");
If dr.Read(){
Property1 = dr.GetValue("Property1");
//So on and so forth
}
}
In the DAL
DataReader GetData(String SPProperty){
}

take a look at SubSonic http://subsonicproject.com/ it does most of the data access tedious work for you and it's easier than most ORMs out there

The DAL needs a reference to the BOL so that it can populate the objects. What you do not want to have is any reference or coupling from the BOL back to the DAL - doing so causes your BOL to be coupled to a specific database implementation. When you think about it this makes sense. Your DAL knows details about the business objects down to the level of properties and how to retrieve their data from the database - of course the DAL is inherently tightly coupled to the BOL. So the reference that way is fine. And if you think about it what is on the other side? The database. "Tightly coupling" going from your object data to your database? Yeah, it is pretty darn tight. The concept is not very meaningful even.
It is all the other direction where you need to decouple. So yes as long as there is no direct coupling from the DAL into the BOL you can change your data platform anyway you want.
Not much point in creating interfaces for BOs and passing them to DAL in this scenario. You might sometimes need to go the other way however. As a rule business objects should not have to know anything about how they are either created or persisted.
In practice even with most ORMs, for example, creating a business layer completely free of any sort of persistence artifacts can become very difficult, sometimes effectively not possible. So occasionally you have something that is just too difficult to work around though, and you might find that strictly avoiding having any data knowledge in BOs is leading you to over complexity that is degrading rather than adding value.
If you feel like there is no better way and you need to have something persisted from within the BOL, create a simple interface so that the DAL functionality can be passed into the BOL. That way you can still keep the BOL decoupled from the specific database implementation at least.
Also, although it is a lot of additional work, unless this is a very simple throwaway app, I strongly recommend that you also add another layer between the UI and the BOL. The MVP (Model-View-Presenter) pattern is a general purpose design pattern for reducing coupling between the core app and the UI. There are a lot of variants on presenters, don't get too caught up in the specific details, just start off with the simple MVP if you have never used it.
The patterns is not that hard, it is just that UI itself is so messy that it may take you at least a couple of major iterations / applications before you feel like the code you are writing at any time is systematically and methodically working to decouple the UI. Just keep working at it, start to acquire an arsenal of techniques, and don't get hung up on the fact that you really have not achieved a sharp clean separation yet. Anything and everything you learn and can do that even contributes a little to creating creating a well defined boundary at the UI is a big step in the right direction.

The 'correct' approach is going to vary depending on business needs. To be honest, there are many projects where I feel the old style ado recordsets incurred less development time and were easier to maintain than many of the ORM's out now. Take some time to identify what your needs are, and remember that development time and maintainability are design goals that should be properly weighed as well.

It also depends on if/what library/ORM (Object-Relational Mapper) you use. When using a (good) ORM, the DAL should be a very remote concern, because it is almost completely hidden by the ORM; however, best practices dictate that even then, for medium to large size applications, you should introduce another layer between the BOL and ORM, usually DTO (Data Transfer Objects). DTOs can also be used without an ORM, as they are just dumb objects defined in a separate library, and the DAL can be responsible for persisting them (transforming them from/to database structures), while the BOL can query the DAL and receive those objects.
Decoupling the layers can be achieved in a variety of ways, most commonly through interfaces and/or MEF or another DI/IOC framework. Any such technique achieves more than sufficient decoupling if used effectively.
Also, depending on the technology used, as Sisyphus said, one of the layered architectural patterns will help separate concerns nicely: MVC, MVP, MVVM etc. I personally recommend MVVM with WPF (desktop) or Silverlight (web) but I'm highly biased - i.e. I love both of them to death :)

These are my findings,
1. Use interfaces
2. Use DTOs [Data Transfer Objects] between DAL & BLL
3. Split BLL into two,
a. BLL
b. Service Layer
4. Use Inversion of Control (IoC) container for keep coupling as low as possible.