We Keep Coding

What am I missing with the Respository Pattern? How do you use it in the real world?

Every example of the Repository Pattern I have seen deals with a very simple use case - one object type and the most basic CRUD operations. The Repository is then very often plugged straight into an MVC controller.
Real-world data access just isn't like this. Real-world data access scenarios can involve complex graphs of objects and some form of transactional wrapper. For example, suppose I want to save a new Order. This involves writing to the Order, OrderDetails, Invoice, User, History and ItemStock tables. All of this must be transacted, committed or rolled back. Normally I'd pass around something like an IDbTransaction and an IDbConnection and bundles the whole operation in a service layer.
Where does the Repository Pattern fit with this? Am I missing something (Unit Of Work perhaps)? Are there any more realistic examples of Repositories in use than the usual canned blog snippets?
Appreciate any light.

This is a very controversial subject but here is my catch from my own experience.
Repository works at the aggregate root. For example if an OrderItem is always retrieved as part of Order and does not have a life of its own outside order, then it will be loaded by the OrderRepository, otherwise it will have its own repository.
UnitOfWork is an important concept. Let's say OrderItem is an aggregate root and has its own repository. So at the time of creating an order, OrderManager will create a UnitOfWork of work in a using block, initialise OrderItemRepository and OrderRepository and then commit.
UPDATE
Yes, exactly. Imagine - in our case order - an order is being inserted. This needs to be in control of the transaction and enter order and order items separately inside the same transaction. This cannot be managed at the repository level. This is the sole reason for existence of UnitOfWork concept which is passed to the repository so that it does not own or initialise it. UnitOfWork usually is created at the business layer.

O/R-Mappers like Hibernate basically implement the Repository pattern for object graphs while fully supporting transactions. It's often a leaky abstraction, but it certainly can be made to work in complex real-world scenarios.

If you need a good full blown, widely used expample of the repository pattern look at Cocoa's Core Data I realize it is not in the realm of programming languages that you note. But note that it is NOT an O/R mapper. It is a complete abstraction of an object store. No Sql statements to execute, and while you may pick the format of the external storage that is used, you never interact with it directly.

I like to think of a repository as another layer of abstraction. You should only add layers of abstraction when the cost of their implementation is less than the cost of NOT doing the implementation (code maintenance, support, enhancements, etc).
Remember that the purpose of the repository pattern is to separate the logic that retrieves the data (CRUD) and maps it to the entity model from the business logic that acts on the model. What this often ends up doing/looking like in the real world is some form of business entities, that abstract the underlying physical data model.
As far as your transaction question, yes, this relates more to the Unit of Work pattern. Since you mentioned services, I would encourage you to NOT pass around your connection to your various data access classes/methods, but to instead allow WCF to manage the transaction for you using auto enlistment. Here is an extract of Juval Lowy's WCF book (highly recommended) that explains the how and why of this method of transaction management.
So to answer your question, the repository pattern fits in as a way to abstract the physical data model and to separate the CRUD/mapping from the business logic.

Architectural concerns: Fluent NHibernate, The Repository pattern and ASP.NET MVC

I've just started a new project and have naturally opted to use a lot of new tech.
I'm using (Fluent) NHibernate, ASP.NET MVC 3 and am trying to apply the Repository pattern.
I've decided to seperate my Business Logic into a seperate project and define services which wrap my repositories so that I can return POCOs instead of the NHibernate proxies and maintain more seperation between my Front end and DA logic. This will also give me the power to easily provide the same logic as an API later (a requirement).
I have chosen to use a generic IRepository<T> interface where T is one of my NHibernate mapped Entities which all implement IEntity (my interface only a marker really).
The problem is this goes against the aggregate root pattern and I'm starting to feel the pain of the anemic domain model.
If I change an object that is hanging of another
Root <- changed
Child <- changed
In my service I have to do the following:
public void AddNewChild(ChildDto child, rootId)
{
var childEntity = Mapper.Map<ChildDto,ChildEntity>(child);
var rootEntity = _rootrepository.FindById(rootId);
rootEntity.Children.Add(childEntity);
_childRepository.SaveOrUpdate(child);
_rootRepository.SaveOrUpdate(root);
}
If I don't save the child first I get an exception from NHibernate. I feel like my generic repository (I currently require 5 of them in one service) is not the right way to go.
public Service(IRepository<ThingEntity> thingRepo, IRepository<RootEntity> rootRepo, IRepository<ChildEntity> childRepo, IRepository<CategoryEntity> catRepo, IRepository<ProductEntity> productRepo)
I feel like instead of making my code more flexible, it's making it more brittle. If I add a new table I need to go and change the constructor in all my tests (I'm using DI for the implementation so that's not too bad) but it seems a bit smelly.
Does anyone have any advice on how to restructure this sort of architecture?
Should I be making my repositories more specific? Is the service abstraction layer a step too far?
EDIT: There's some great related questions which are helping:
Repository Pattern Best Practice
repository pattern help
Architectural conundrum

When you have an Aggregate, the Repository is the same for the aggregate parent (root) and its children because the life cycle of the children is controlled by the root object.
Your "Save" method for the root object type should be also directly responsible for persisting the changes to the children records instead of delegating it into yet another repository(ies).
Additionally, in a "proper" Aggregate pattern, the child records have no identity of their own (at least one that is visible outside the Aggregate). This has several direct consequences:
There can be no foreign keys from outside records/aggregates to those children records.
Resulting from point 1., every time you save the root object state, you can delete and recreate the child records on the database. This usually will make your persistence logic easier if you bump into precedence problems.
Note: the reverse case of 1. is not true. Child records in an aggregate can have foreign keys to other root records.

I feel like instead of making my code more flexible, it's making it more brittle. If I add a new table I need to go and change the constructor in all my tests (I'm using DI for the implementation so that's not too bad) but it seems a bit smelly.
Implement the Unit Of Work pattern in your repository. That means practically you have a unit of work class which holds your services inject via ctor or property injection. Futheremore it holds a commit and/or transaction method. Only inject the IUnitOfWork instance in your services. When you add a repository you just have to change the unit of work not touch the business logic (services).

Is DTO plus UnitOfWork pattern a good approach to design a DAL for a web application?

I'm implementing a DAL using entity framework. On our application, we have three layers (DAL, business layer and presentation). This is a web app. When we began implementing the DAL, our team thought that DAL should have classes whose methods receive a ObjectContext given by services on the business layer and operate over it. The rationale behind this decision is that different ObjectContexts see diferent DB states, so some operations can be rejected due to problems with foreign keys match and other inconsistencies.
We noticed that generating and propagating an object context from the services layer generates high coupling between layers. Therefore we decided to use DTOs mapped by Automapper (not unmanaged entities or self-tracking entities arguing high coupling, exposing entities to upper layers and low efficiency) and UnitOfWork. So, here are my questions:
Is this the correct approach to design a web application's DAL? Why?
If you answered "yes" to 1., how is this to be reconciled the concept of DTO with the UnitOfWork patterns?
If you answered "no" to 1., which could be a correct approach to design a DAL for a Web application?
Please, if possible give bibliography supporting your answer.
About the current design:
The application has been planned to be developed on three layers: Presentation, business and DAL. Business layer has both facades and services
There is an interface called ITransaction (with only two methods to dispose and save changes) only visible at services. To manage a transaction, there is a class Transaction extending a ObjectContext and ITransaction. We've designed this having in mind that at business layer we do not want other ObjectContext methods to be accessible.
On the DAL, we created an abstract repository using two generic types (one for the entity and the other for its associated DTO). This repository has CRUD methods implemented in a generic way and two generic methods to map the DTOs and entities of the generic repository with AutoMapper. The abstract repository constructor takes an ITransaction as argument and it expects the ITransaction to be an ObjectContext in order to assign it to its proctected ObjectContext property.
The concrete repositories should only receive and return .net types and DTOs.
We now are facing this problem: the generic method to create does not generate a temporal or a persistent id for the attached entities (until we use SaveChanges(), therefore breaking the transactionality we want); this implies that service methods cannot use it to associate DTOs in the BL)

There are a number of things going on here...The assumption I'll make is that you're using a 3-Tier architecture. That said, I'm unclear on a few design decisions you've made and what the motivations were behind making them. In general, I would say that your ObjectContext should not be passed around in your classes. There should be some sort of manager or repository class which handles the connection management. This solves your DB state management issue. I find that a Repository pattern works really well here. From there, you should be able to implement the unit of work pattern fairly easily since your connection management will be handled in one place. Given what I know about your architecture, I would say that you should be using a POCO strategy. Using POCOs does not tightly couple you to any ORM provider. The advantage is that your POCOs will be able to interact with your ObjectContext (probably via Repository of some sort) and this will give you visibility into change tracking. Again, from there you will be able to implement the Unit of Work (transaction) pattern to give you full control over how your business transaction should behave. I find this is an incredibly useful article for explaining how all this fits together. The code is buggy but accurately illustrates best practices for the type of architecture you're describing: Repository, Specification and Unit of Work Implementation
The short version of my answer to question number 1 is "no". The above link provides what I believe to be a better approach for you.

I always believed that code can explain things better than worlds for programmers. And this is especially true for this topic. Thats why I suggest you to look at the great sample application in witch all consepts you expecting are implemented.
Project is called Sharp Architecture, it is centered around MVC and NHibernate, but you can use the same approaches just replacing NHibernate parts with EF ones when you need them. The purpose of this project is to provide an application template with all community best practices for building web applications.
It covers all common and most of the uncommon topics when using ORM's, managing transactions, managing dependencies with IoC containers, use of DTOs, etc.
And here is a sample application.
I insist on reading and trying this, it will be a real trasure for you like it was for me.

You should take a look what dependency injection and inversion of control in general means. That would provide ability to control life cycle of ObjectContext "from outside". You could ensure that only 1 instance of object context is used for every http request. To avoid managing dependencies manually, I would recommend using StructureMap as a container.
Another useful (but quite tricky and hard to do it right) technique is abstraction of persistence. Instead of using ObjectContext directly, You would use so called Repository which is responsible to provide collection like API for Your data store. This provides useful seam which You can use to switch underlying data storing mechanism or to mock out persistence completely for tests.
As Jason suggested already - You should also use POCO`s (plain old clr objects). Despite that there would still be implicit coupling with entity framework You should be aware of, it's much better than using generated classes.
Things You might not find elsewhere fast enough:
Try to avoid usage of unit of work. Your model should define transactional boundaries.
Try to avoid usage of generic repositories (do note point about IQueryable too).
It's not mandatory to spam Your code with repository pattern name.
Also, You might enjoy reading about domain driven design. It helps to deal with complex business logic and gives great guidelines to makes code less procedural, more object oriented.

I'll focus on your current issues: To be honest, I don't think you should be passing around your ObjectContext. I think that is going to lead to problems. I'm assuming that a controller or a business service will be passing the ObjectContext/ITransaction to the Repository. How will you ensure that your ObjectContext is disposed of properly down stream? What happens when you use nested transactions? What manages the rollbacks, for transactions down stream?
I think your best bet lies in putting some more definition around how you expect to manage transactions in your architecture. Using TransactionScope in your controller/service is a good start since the ObjectContext respects it. Of course you may need to take into account that controllers/services may make calls to other controllers/services which have transactions in them. In order to allow for scenarios where you want full control over your business transactions and the subsequent database calls, you'll need to create some sort of TransactionManager class which enlists, and generally manages transactions up and down your stack. I've found that NCommon does an extraordinary job at both abstracting and managing transactions. Take a look at UnitOfWorkScope and TransactionManager classes in there. Although I disagree with NCommon's approach of forcing the Repository to rely on the UnitOfWork, that could easily be refactored out if you wanted.
As far as your persistantID issue goes, check this out

Enterprise Design Pattern Question

Something on my mind about structuring a system at a high level.
Let's say you have a system with the following layers:
UI
Service Layer
Domain Model
Data Access
The service layer is used to populate a graph of objects in the domain model. In an attempt to avoid coupling, the domain model will be not be persistence aware and will not have any dependencies on any data access layer.
However, using this approach how would one object in the domain model be able to call other objects without being able to load them with persistence, thus coupling everything together - which I'd be trying to avoid.
e.g. an Order Object would need to check an Inventory object and would obviously need to tell the Inventory object to load in some way, or populate it somehow.
Any thoughts?

You could inject any dependencies from the service layer, including populated object graphs.
I would also add that a repository can be a dependency - if you have declared an interface for the repository, you can code to it without adding any coupling.

One way of doing this is to have a mapping layer between the Data Layer and the domain model.
Have a look at the mapping, repository and facade patterns.
The basic idea is that on one side you have data access objects and on the other you have domain objects.

To decouple you have to: "Program to an 'interface', not an 'implementation'." (Gang of Four 1995:18)
Here are some links on the subject:
Gamma interview on patterns
Random blog article
Googling for "Program to an interface, not an implementation" will yield many useful resources.

Have the domain model layer define interfaces for the methods you'll need to call, and POCOs for the objects that need to be returned by those methods. The data layer can then implement those interfaces by pulling data out of your data store and mapping it into the domain model POCOs.
Any domain-level class that requires a particular data-access service can just depend on the interface via constructor arguments. Then you can leverage a dependency-injection framework to build the dependency graph and provide the correct implementations of your interfaces wherever they are required.

Before writing tons of code in order to separate everything you might want to ask yourself a few questions:
Is the Domain Model truly separate from the DAL? And yes, I'm serious and you should think about this because it is exceedingly rare for an RDBMS to actually be swapped out in favor of a different one for an existing project. Quite frankly it is much more common for the language the app was written in to be replaced than the database itself.
What exactly is this separation buying you? And, just as important, what are you losing? Separation of Concerns (SoC) is a nice term that is thrown about quite a bit. However, most people rarely understand why they are Concerned with the Separation to begin with.
I bring these up because more often than not applications can benefit from a tighter coupling to the underlying data model. Never mind that most ORM's almost enforce a tight coupling due to the nature of code generation. I've seen lot's of supposedly SoC projects come to a crash during testing because someone added a field to a table and the DAL wasn't regenerated... This kind of defeats the purpose, IMHO...
Another factor is where should the business logic live? No doubt there are strong arguments in favor of putting large swaths of BL in the actual database itself. At the same time there are cases where the BL needs to live in or very near your domain classes. With BL spread in such a way, can you truly separate these two items anyway? Even those who hate the idea of putting BL in a database will fall back on using identity keys and letting the DB enforce referential integrity, which is also business logic..
Without knowing more, I would suggest you consider flattening the Data Access and Domain Model layers. You could move to a "provider" or "factory" type architecture in which the service layer itself doesn't care about the underlying access, but the factory handles it all. Just some radical food for thought.

You should take a look at Martin Fowler's Repository and UnitOfWork patterns to use interfaces in your system

Until now I have seen that application can be well layered into three layers: Presentation-->Logic-->Data--and Entities (or Bussines Object). In the Logic Layer case you can use some pattern such as Transaction Script or Domain Model I'm supposing you're using this last. The domain model can use a Data Mapper for interacting with the data layer and create business objects, but you can also use a Table Module pattern.
All this patterns are described in Marttin's Fowler Patterns of Enterprise Application Architecture book. Personally I use Transaction Script because it is simplest than Domanin Model.

One solution is to make your Data Access layer subclass your domain entities (using Castle DynamicProxy, for example) and inject itself into the derived instances that it returns.
That way, your domain entity classes remain persistence-ignorant while the instances your applications use can still hit databases to lazy-load secondary data.
Having said that, this approach typically requires you to make a few concessions to your ORM's architecture, like marking certain methods virtual, adding otherwise unnecessary default constructors, etc..
Moreover, it's often unnecessary - especially for line-of-business applications that don't have onerous performance requirements, you can consider eagerly loading all the relevant data: just bring the inventory items up with the order.

I felt this was different enough from my previous answer, so here's a new one.
Another approach is to leverage the concept of Inversion of Control (IoC). Build an Interface that your Data Access layer implements. Each of the DAL methods should take a list of parameters and return a Data Table.
The service layer would instantiate the DAL through the interface and pass that reference to your Domain Model. The domain model would then make it's own calls into the DAL, using the interface methods, and decide when it needs to load child objects or whatever.
Something like:
interface IDBModel {
DataTable LoadUser(Int32 userId);
}
class MyDbModel : IDBModel {
DataTable LoadUser(Int32 userId) {
// make the appropriate DB calls here, return a data table
}
}
class User {
public User(IDBModel dbModel, Int32 userId) {
DataTable data = dbModel.LoadUser(userId);
// assign properties.. load any additional data as necessary
}
// You can do cool things like call User.Save()
// and have the object validate and save itself to the passed in
// datamodel. Makes for simpler coding.
}
class MyServiceLayer {
public User GetUser(Int32 userId) {
IDBModel model = new MyDbModel();
return new User(model, userId);
}
}
With this mechanism, you can actually swap out your db models on demand. For example, if you decide to support multiple databases then you can have code that is specific to a particular database vendors way of doing things and just have the service layer pick which one to use.
The domain objects themselves are responsible for loading their own data and you can keep any necessary business logic within the domain model. Another point is that the Domain Model doesn't have a direct dependency on the data layer, which preserves your mocking ability for independent testing of business logic.
Further, the DAL has no knowledge of the domain objects, so you can swap those out as necessary or even just test the DAL independently.

How should I separate entities methods?

Good people of SO,
Today I have some serious concerns on my business layer design.
It is based on Entity POCO objects and
I want to add logic to these entities BUT, there are 2 types of logic:
Pure C# logic
Persistence logic (LinqToEntities in my case)
My question is simple:
How should I separate these two kinds ?
First, I was thinking about adding these two as methods to the entities. And using partial classes to split them.
Second, I thought that I wouldn't want an overweight object with a LOT of methods.
So maybe why not static classes or singleton with methods doing the LinqToEntities stuff, and leave the pure C# in entity methods.
Then I would have several classes grouped by fonctionnality providing the logic, the entity is passed as argument to the classes methods.
It really bothers me, because the second solution seems cleaner but it looks like it breaks the object-oriented paradigm. On the other hand the first one seems like an anti-pattern.
What do you think ? Do you have a bright solution solving this paradox ?
Schizophrenic edit: in fact what I call persistence logic should go to the DAL and the pure c# logic in the BLL. POCO entities are produced by the DAL. I can then extend these entities in my BLL to add methods. In my DAL I should structure the logic as exposed in the second solution.

The logic that describes how an entity should be saved/loaded doesn't belong to the entity itself ; it's more likely to be the role of a persistence service, a data access object, etc.
I would let the object specific logic in the object -- we're here talking about the object behavior, then create a service that handles persistence concerns for this object type.