We Keep Coding

How to change persistence layer at runtime in C#

I am embarking on a new project and I need some guidance from veteran architects/design pattern gurus!
My new project needs to have a number of persistence layers whereby the client can decide at runtime where the data will be stored, for example, in house SQL database, MS Exchange or Google storage.
The functionality will essentially be the same just the storage/implementation of each will be different.
What I'm not looking for a here is how you do it just a pointer to the best patterns to use to serve my purpose whilst still providing flexibility down the road as their will be CHANGE. I am trying to avoid concrete implementations that will inevitably lead to some nasty code smells.
I know it will involve some kind of DI along the way but any pointers here would be greatly appreciated.

There is nothing special with your case really, so if you would follow standard practices with DI and use container to ease your task like SimpleInjector that will do the trick. The main point for you should be to not depend on concrete classes but on abstraction and that's where DI-container will help you organize this.
E.g. if you plan to save user you might have some IUserRepository with a method SaveUser. Then you will implement SqlUserRepository, GoogleStorageRepository, etc. The same goes for any other data access layer interface. If you just do that, you will need to configure your DI in a way where you can supply the required repository at a runtime based on your needs. Do not forget to never depend on GoogleStorageRepository, etc. directly, but only on a common interface. I would create a project for interfaces (and corresponding BI data model that DL will be aware of) and a project per each implementation as well to separate it even further.

Repository pattern is all about creating a separation between the persistence layer and the business layer.
Many examples on the web demonstrates it incorrectly by just using it as an wrapper over their data entities. That is incorrect. The design of a repository class/interface should be driven by the business requirements and not from how the first data store looks like.
Thus it's a perfect pattern for your use case. You define an repository interface from the business layer perspective and then create an implementation for each data store like MSSQL. I even put that interface in my business layer to further demonstrate that perspective.

Implementing caching of services in Domain project being used by a Web API

My question is: how do I implement caching in my domain project, which is working like a normal stack with the repository pattern.
I have a setup that looks like the following:
ASP.NET MVC website
Web API
Domain project (using IoC, with Windsor)
My domain project for instance have:
IOrderRepository.cs
OrderRepository.cs
Order.cs
My ASP.NET MVC website calls the Web API and gets back some DTO classes. My Web API then maps these objects to business objects in my domain project, and makes the application work.
Nowhere in my application have I implemented caching.
Where should be caching be implemented?
I thought about doing it inside the methods in the OrderRepository, so my Get, GetBySpecification and Update methods has to call some generic cache handler injected by the OrderRepository.
This obviously gives some very ugly code, and isn't very generic.
How to maintain the cache?
Let's say we have a cache key like "OrderRepostory_123". When I call the Update method, should I call cacheHandler.Delete("OrderRepository_123") ? Because that seems very ugly as well
My own thoughts...
I can't really see a decent way to do it besides some of the messy methods I have described. Maybe I could make some cache layer, but I guess that would mean my WebAPI wouldn't call my OrderRepository anymore, but my CacheOrderRepository-something?

Personally, I am not a fan of including caching directly in repository classes. A class should have a single reason to change, and adding caching often adds a second reason. Given your starting point you have at least two likely reasonable options:
Create a new class that adds caching to the repository and exposes the same interface
Create a new service interface that uses one or more repositories and adds caching
In my experience #2 is often more valuable, since the objects you'd like to cache as a single unit may cross repositories. Of course, this depends on how you have scoped your repositories. A lot may depend on whether your repositories are based on aggregate roots (ala DDD), tables, or something else.

There are probably a million different ways to do this, but it seems to me (given the intent of caching is to improve performance) implementing the cache similar to a repository pattern - where the domain objects interact with the cache instead of the database, then perhaps a background thread could keep the database and cache in sync, and the initial startup of the app pool would fill the cache (assuming eager loading is desired). A whole raft of technical issues start to crop up, such as what to do if the cache is modified in a way that violates a database constraint. Code maintenance becomes a concern where any data structure related concerns possibly need to be implemented in multiple places. Concurrency issues start to enter the fray. Just some thoughts...

SQLCacheDependency with System.Web.Caching.Cache, http://weblogs.asp.net/andrewrea/archive/2008/07/13/sqlcachedependency-i-think-it-is-absolutely-brilliant.aspx . This will get you caching that gets invalidated based on other systems applying updates also.

there are multiple levels of caching depending on the situation however if you are looking for generic centralized caching with low number of changes I think you will be looking for EF second level caching and for more details check the following http://msdn.microsoft.com/en-us/magazine/hh394143.aspx
Also you can use caching on webapi level
Kindly consider if MVC and WebAPI the network traffic if they are hosted in 2 different data centers
and for huge read access portal you might consider Redis http://Redis.io

It sounds like you want to use a .NET caching mechanism rather than a distributed cache like Redis or Memcache. I would recommend using the System.Runtime.Caching.MemoryCache class instead of the traditional System.Web.Caching.Cache class. Doing this allows you to create your caching layer independent of your MVC/API layer because the MemoryCache has no dependencies on System.Web.
Caching your DTO objects would speed up your application greatly. This prevents you from having to wait for data to be assembled from a cache that mirrors your data layer. For example, requesting Order123 would only require a single cache read rather than to several reads to any FK data. Your caching layer would of course need to contain the logic to invalidate the cache on UPDATEs you perform. A recommended way would be to retrieve the cached order object and modify its properties directly, then persist to the DB asynchronously.

Dependency Injection vs Layered Architecture

I've been reading a lot about dependency injection and the service locator (anti-?) pattern - a lot of it on StackOverflow (thanks guys :). I have a question about how this pattern works when it's within a n-layer architecture.
I've seen a lot of blog posts where they describe injecting a IDataAccess component into the business objects. E.g.
public class Address
{
IDataAccess _dataAccess;
public Address(IDataAccess dataAccess)
{
this._dataAccess = dataAccess;
}
}
However, I was under the impression that in an n-layer architecture, the UI layer should not need to have any knowledge of the data access layer... or even know that there /is/ a data access layer! If DI requires exposing the IDataAccess interface in the constructors of the BusinessObjects, this then exposes to the UI the fact that the Business Layer uses a data access layer under the hood - something the UI doesn't need to know or care about surely?
So, my fundamental question is: Does DI require that I expose all my lower layer interfaces to all upper layers and is this a good or a bad thing?
Thanks
Edit: To clarify (after a few comments), I know my business object should be ignorant of the which specific implementation of which IDataAccess it uses (hence the Dependency being injected in the constructor) but I thought that the layers above the BO should not know that the Business Object even requires a dependency on a DAL.

This is really a fairly complex topic, and there are many ways of doing an n-tier architecture. No one way is "the right way", and how you do it depends on your needs as much as it does your personal preferences.
Dependency Injection is about managing dependencies. If your object should be unaware of any dependency, then you would not write your objet in the way you mentioned. You would instead have some other service or method that would populate the data in an agnostic way. Data doesn't mean "Database" either. So IDataAccess could mean it comes from a database, or it comes from a network socket or it comes from a file on disk. The whole point here is that Address does not choose what dependencies it creates. This is done through configuration at the composition root.
Things need data, otherwise your app is probably useless. Making your Address object load itself, however, may not be the best way to go about things. A better approach may be with a factory class or service method.

I think the answer is rather simple. Your bottom layers (interface, bll, dal, entities) are just a bunch of libraries. It is up to the client to decide which libraries to be used and it will increase client's flexibility. Moreover they are libraries, so any application-related configurations (connection strings, data caching, etc) lies on the client. Those configuration itself, sometimes also need to be injected and included into Composition Root.
However, if you want to has an uniform logic and not client's flexibility, you can choose web/app services as an additional layer.
1st Layer Entities
2nd Layer Interface
3rd Layer BLL & DAL
4th Layer Web/App Services
5th Layer UI
This way, your composition root exists in one layer (4th). And add your UI just need to add service reference to 4th layer (or 1st if needed). However, this implies the same Mark Seeman's article again, layering is worth the mapping. I assume that you can change the app/web service to Composition Root.
Moreover, this (app/web service) design has pros/cons. Pros:
Your app is encapsulated
Your app is being bridged by app/web services. It is guranteed that your UI don't know the DataAccess, thus fulfill your requirements.
Your app is secured
Simply said, having UI need to access app service is a huge gain in security aspect.
Access Portability
Now your app can be accessed everywhere. It can be connected by 3rd party app (other web) without has relying on dlls.
Cons:
Overhead cost during service call
Authentication, network connection, etc, will cause overhead during webservice call. I'm inexperienced for the performance impact but it should be enough for high traffic app.
Inflexibility of client
Client now need to access BLL/Services by using services instead of normal objects.
More Service for Different Type of Client
Now you need to provide more service than needed. Such as WebRequestRetriever, MobileRequestRetriever instead of accessing to a mere IRequestRetriever and let the composition root wire up the rest.
Apologize if this answer boarden the topic (just realized after finished).

IMHO:
It depends on who does the injection !-
It seems you need to/expect to have an MVC or MVP architecture to be in place, where a controller or Presenter does the job of translating the UI calls to business objects ,back and forth -
Creating concrete implementations of IDataAccess, Sending it to Address class.
So that the UI is totally unaware of who is providing the data it needs, and it provides you the expected scalability.
Thanks
Tarriq

Best practice for application facade pattern

I have a multi tiered SOA application and a database with over 100 tables. I'm using entity framework for my data layer which takes care of all CRUD operations.
I have 1 facade class which is hosted on a service, callable by client apps all over.
This facade class contains methods such as
private void DoSomething()
{
//insert to table1
//insert to table 2
//delete from table 3
//more CRUD operations
}
And the facade class is basically full with loads of other methods similar to DoSomething()
So the client will basically create an instance of the facade class, and gain access to all these methods.
My question now is, is this the best practice for a facade pattern? I feel that the facade class is way too "heavy" and I'm not sure if it will affect the performance if my application scales bigger.
Will creating an instance of the facade class be a very expensive operation if I have loads of methods in it?

Well, Facade is highly adequate for SOA architectures, once it encapsulates a subsystem as an object and provides an aggregated interface for all business objects and services to your client. It also reduces the coupling in your architecture. I think your approach isn't heavy, and won't affect the scalability of your system.

Will creating an instance of the facade class be a very expensive
operation if I have loads of methods in it?
Assuming those methods are not called during construction they should have no impact on the 'weight' of the object. My understanding is that a method that isn't doing anything doesn't cost memory or cycles for practical purposes.
(as a side note, there are technical limits that don't add value to your question. see How many methods can a C# class have)
Make your facade work for you! A good practice is to assume you're handing it over to someone else and ask "does this neatly describe what the capabilities of my API are?". Sixty sounds like it's reaching the upper-bounds of my personal preference but that's still only CRUD for 15 objects.
Think of the facade as a unified interface that frees you up to create more concise and sophisticated Services (which in turn should encapsulate even more concise repositories which in turn should encapsulate even more concise records of data in tables, BLOBS in buckets, etc.).

Enterprise Design Pattern Question

Something on my mind about structuring a system at a high level.
Let's say you have a system with the following layers:
UI
Service Layer
Domain Model
Data Access
The service layer is used to populate a graph of objects in the domain model. In an attempt to avoid coupling, the domain model will be not be persistence aware and will not have any dependencies on any data access layer.
However, using this approach how would one object in the domain model be able to call other objects without being able to load them with persistence, thus coupling everything together - which I'd be trying to avoid.
e.g. an Order Object would need to check an Inventory object and would obviously need to tell the Inventory object to load in some way, or populate it somehow.
Any thoughts?

You could inject any dependencies from the service layer, including populated object graphs.
I would also add that a repository can be a dependency - if you have declared an interface for the repository, you can code to it without adding any coupling.

One way of doing this is to have a mapping layer between the Data Layer and the domain model.
Have a look at the mapping, repository and facade patterns.
The basic idea is that on one side you have data access objects and on the other you have domain objects.

To decouple you have to: "Program to an 'interface', not an 'implementation'." (Gang of Four 1995:18)
Here are some links on the subject:
Gamma interview on patterns
Random blog article
Googling for "Program to an interface, not an implementation" will yield many useful resources.

Have the domain model layer define interfaces for the methods you'll need to call, and POCOs for the objects that need to be returned by those methods. The data layer can then implement those interfaces by pulling data out of your data store and mapping it into the domain model POCOs.
Any domain-level class that requires a particular data-access service can just depend on the interface via constructor arguments. Then you can leverage a dependency-injection framework to build the dependency graph and provide the correct implementations of your interfaces wherever they are required.

Before writing tons of code in order to separate everything you might want to ask yourself a few questions:
Is the Domain Model truly separate from the DAL? And yes, I'm serious and you should think about this because it is exceedingly rare for an RDBMS to actually be swapped out in favor of a different one for an existing project. Quite frankly it is much more common for the language the app was written in to be replaced than the database itself.
What exactly is this separation buying you? And, just as important, what are you losing? Separation of Concerns (SoC) is a nice term that is thrown about quite a bit. However, most people rarely understand why they are Concerned with the Separation to begin with.
I bring these up because more often than not applications can benefit from a tighter coupling to the underlying data model. Never mind that most ORM's almost enforce a tight coupling due to the nature of code generation. I've seen lot's of supposedly SoC projects come to a crash during testing because someone added a field to a table and the DAL wasn't regenerated... This kind of defeats the purpose, IMHO...
Another factor is where should the business logic live? No doubt there are strong arguments in favor of putting large swaths of BL in the actual database itself. At the same time there are cases where the BL needs to live in or very near your domain classes. With BL spread in such a way, can you truly separate these two items anyway? Even those who hate the idea of putting BL in a database will fall back on using identity keys and letting the DB enforce referential integrity, which is also business logic..
Without knowing more, I would suggest you consider flattening the Data Access and Domain Model layers. You could move to a "provider" or "factory" type architecture in which the service layer itself doesn't care about the underlying access, but the factory handles it all. Just some radical food for thought.

You should take a look at Martin Fowler's Repository and UnitOfWork patterns to use interfaces in your system

Until now I have seen that application can be well layered into three layers: Presentation-->Logic-->Data--and Entities (or Bussines Object). In the Logic Layer case you can use some pattern such as Transaction Script or Domain Model I'm supposing you're using this last. The domain model can use a Data Mapper for interacting with the data layer and create business objects, but you can also use a Table Module pattern.
All this patterns are described in Marttin's Fowler Patterns of Enterprise Application Architecture book. Personally I use Transaction Script because it is simplest than Domanin Model.

One solution is to make your Data Access layer subclass your domain entities (using Castle DynamicProxy, for example) and inject itself into the derived instances that it returns.
That way, your domain entity classes remain persistence-ignorant while the instances your applications use can still hit databases to lazy-load secondary data.
Having said that, this approach typically requires you to make a few concessions to your ORM's architecture, like marking certain methods virtual, adding otherwise unnecessary default constructors, etc..
Moreover, it's often unnecessary - especially for line-of-business applications that don't have onerous performance requirements, you can consider eagerly loading all the relevant data: just bring the inventory items up with the order.

I felt this was different enough from my previous answer, so here's a new one.
Another approach is to leverage the concept of Inversion of Control (IoC). Build an Interface that your Data Access layer implements. Each of the DAL methods should take a list of parameters and return a Data Table.
The service layer would instantiate the DAL through the interface and pass that reference to your Domain Model. The domain model would then make it's own calls into the DAL, using the interface methods, and decide when it needs to load child objects or whatever.
Something like:
interface IDBModel {
DataTable LoadUser(Int32 userId);
}
class MyDbModel : IDBModel {
DataTable LoadUser(Int32 userId) {
// make the appropriate DB calls here, return a data table
}
}
class User {
public User(IDBModel dbModel, Int32 userId) {
DataTable data = dbModel.LoadUser(userId);
// assign properties.. load any additional data as necessary
}
// You can do cool things like call User.Save()
// and have the object validate and save itself to the passed in
// datamodel. Makes for simpler coding.
}
class MyServiceLayer {
public User GetUser(Int32 userId) {
IDBModel model = new MyDbModel();
return new User(model, userId);
}
}
With this mechanism, you can actually swap out your db models on demand. For example, if you decide to support multiple databases then you can have code that is specific to a particular database vendors way of doing things and just have the service layer pick which one to use.
The domain objects themselves are responsible for loading their own data and you can keep any necessary business logic within the domain model. Another point is that the Domain Model doesn't have a direct dependency on the data layer, which preserves your mocking ability for independent testing of business logic.
Further, the DAL has no knowledge of the domain objects, so you can swap those out as necessary or even just test the DAL independently.