I read many documentation about CRUD and I still don't understand what exactly should be CRUDable! It seems most of people are talking about CRUD entities but they architecture doesn't show any Create, Read, Update or Delete methods in their entities. They implements these CRUD operations in a separate class. I like to call these kind of classes CRUD controllers.
Is it correct to create POCO entites with CRUD controller? What should be CRUD?
My take is that you should have a repository which performs the CRUD operations.
Then a controller should call the appropriate CRUD method in the repository, possibly via an intermediate service layer.
Read more about the repository pattern here and here.
These classes are usually called repositories. A repository provides access to your entities with means of adding, updating, removing and retrieving one or more entities. So the repository Creates, Reads, Updates and Deletes (CRUD).
When using a database your POCO is normally an object your database entity is converted into, e.g. with AutoMapper in the repository.
In architectural terms, CRUD means that you have entities without business rules. This entities, have, at most, some simple validations. When you have this kind of entity you speak of CRUD because you can modify the data without worrying about anything else (but validations). This can be used for example for maintaining a list of contacts: name + phone no. + address: at most you can validate that the name is not empty, the phone no. is valid, and the address is valid. But there are no business rules in there.
If there are involved business rules, you should avoid using CRUD to make sure that the business rules are respected. For example you should not allow CRUD for an order detail, because there are business rules involved: perhaps you cannot change an order detail if the order is already paid or sent, or confirmed to the customer. Besides the total order amount depends on the order details. In this case you should use the order with its details as a whole, and read / write / update it all at once. (In DDD this is called "aggregate").
Speaking about CRUD is not a question of how you implement it (repository, ORM like DbContext or NHibernate, or wichever you want to use), but a more philosophical question.
Implementing CRUD is much faster than implementing any other architecture which involves business rules (for example DDD). If you can use CRUD for an entity, is advisable to use it, but not in the other cases.
As to your comment:
but they architecture doesn't show any Create, Read, Update or Delete methods in their entities
That's natural... you can do CRUD with EF for example, without explicitly declaring the CRUD methods. Create an entity in the context, or remove it or modify it and the CRUD operations will be implicitly executed on the SaveChanges.
Related
One of the main aspects of software development that I struggle with is delegating the correct responsibility into classes within my program. Working at my first junior role I'm also being exposed to a lot of different design patterns and ideas, and sometimes the information can be overwhelming.
Obviously when we are building software we tend to state that a class should be responsible for one thing and one thing only. It should do that thing well and nothing more. So in the case of the Factory pattern the Factory class should be responsible for building the product and exposing an interface that allows the director to extract the product from the factory.
However the factory class obviously needs to receive the data to build the product from somewhere, without input data we have no output product. Therefore I'd like to know whether including functionality for a factory to query a database is appropriate? My rationale for this is that if the factory is tasked with building a particular product, then it should also be responsible for retrieving the data required to build that product. But I'm not 100% sure if this is correct.
Alternatively, should there be a repository class who's responsibility is to retrieve the data in question from the database, which can be then passed to the factory for assembly into the required product? The use of a repository class seems a bit excessive in this case as we have a class that will hold a large number of different pieces of data which then must be shipped into the factory class.
If we also bear in mind Uncle Bob's teachings that state that functions and methods should have absolutely no more than three parameters then we will be breaking this rule by passing in a large amount of data to the factory. If we first assemble the data into an encompassing class before passing to the factory then we are essentially doing the factory's job within the repository class.
Some guidance would be really appreciated on this, as in my head the lines are very blurry and I'm not sure how I should proceed.
You shouldn't use the factory pattern to object building that extracted from a database. There are the Repository pattern and the Data Mapper pattern for this goal. Those patterns must encapsulate all logic of work with the data storage. Those patterns must have the following responsibility:
the Repository must give an interface to business logic for work with data storage
the Data Mapper must convert data from database to concrete object
The algorithm of cooperation between objects can look like:
business logic uses a repository to read/persist objects.
the repository uses a Data Mapper to convert objects to INSERT or UPDATE queries and to convert data from data storage to object
Also, you can read more details about the repository pattern in C# on the site of Microsoft and you can see C# example of the repository pattern
Use 2 different classes.
A Data Access Object (DAO) provides an abstract interface to the database, and hides its details.
A factory abstracts and hides the details of the creation of your objects. For example, for unit testing you might want to configure the factory so that it doesn't use the Database at all.
To reduce the number of parameters between the DAO and the factory, wrap your many pieces of data in a few logically related classes.
Is it apropriate for a Factory class to also include functionality of extracting data from a database
My rationale for this is that if the factory is tasked with building a particular product, then it should also be responsible for retrieving the data required to build that product. But I'm not 100% sure if this is correct.
A product to retrieve from database is not a trivial object, it is a domain model.
A domain model (aka business model, aka entity(which might indicate a particular instance of it)) belongs to your domain layer (aka business layer).
In this regard, there are some patterns you should be a minimum familiar with...
(Active Record) VS (Data Mapper + Repository) VS (Table Data Gateway + Factory)
Active record pattern kind of violate the Single Responsibility Principle by leading you to implement database access logic inside your domain model and tightly couples them.
Ideally, to avoid the cons above for the cost of a slightly increased complexity (on short term only), we separate the database access logic into a supplementary layer, the data access layer. One of the main component of this layer being the data mapper which in our context (READ operation) is in charge to retrieve data from database and map it to a new domain model instance, your specific product (entity). More generally it encapsulates CRUD operations to the database abstracting this database. Its API inputs and outputs are entity objects and possibly Query Objects.
Optionally, a featured data mapper would make use of patterns such as:
Unit Of Work
Lazy Loading
Identity Map
Transaction
Lock Strategies
Metadata Mapping
Alternatively, should there be a repository class who's responsibility is to retrieve the data in question from the database, which can be then passed to the factory for assembly into the required product? The use of a repository class seems a bit excessive in this case as we have a class that will hold a large number of different pieces of data which then must be shipped into the factory class.
Repository is not a part of your data access layer, but of your domain layer. So it is client of your data access layer. It doesn't encapsulate any database access logic but uses the data mapper.
A repository encapsulates query logic for a particular domain model plus a collection of in-memory entities you've previously retrieved.
A very basic example:
class ProductRepository
{
private $productCollection;
public function findById($id)
{
if (!$this->productCollection->has($id)) {
$product = $this->dataMapper->get(new Query(Product::class, $id));
$this->productCollection->add($product);
return $product;
}
return $this->productCollection->get($id);
}
}
Finally, we can encapsulate database access logic in a table data gateway and use it in a factory. This would result in a solution similar to Gonen I's one. It is simple to implement but there might be cons compared to the data mapper solution. I've never implemented, used or even studied this approach so I can't tell much...
You'd definitely learn a lot by attempting to implement all that by yourself and I'd encourage you to, but keep in mind that if you need a serious solution, ORMs might be interesting for you.
If you're keen to learn more about all this, I recommend Martin Fowler's Patterns of Enterprise Application Architecture book which is summarized here: https://www.martinfowler.com/eaaCatalog/index.html
Background
At the company I work for I have been ordered to update an old MVC app and implement a repository pattern for a SQL database. I have created the context of the database using Entity Framework Database-First and got 23 entities.
The first question
Do I need to create a repository for each entity or implement a generic repository for the context? I'm asking this because I have found following while searching internet:
One repository per domain
You should think of a repository as a collection of domain objects in memory. If you’re building an application called Vega, you shouldn’t have a repository like the following:
public class VegaRepository {}
Instead, you should have a separate repository per domain class, like OrderRepository, ShippingRepository and ProductRepository.
Source: Programming with Mosh: 4 Common Mistakes with the Repository Pattern
The second question
Does a generic repository work for Entity Framework Database-First? This is because I have found following while searching internet:
Entity framework
Do note that the repository pattern is only useful if you have POCOs which are mapped using code first. Otherwise you’ll just break the abstraction with the entities instead (= the repository pattern isn’t very useful then). You can follow this article if you want to get a foundation generated for you.
Source: CodeProject: Repository pattern, done right
To begin with, if you are using full ORM like Entity Framework or NHibernate, you should avoid implementing additional layer of Repository and Unit Of Work.
This is because; the ORM itself exposes both Generic Repository and Unit Of Work.
In case of EF, your DbContext is Unit Of Work and DbSet is Generic Repository. In case of NHibernate, it is ISession itself.
Building new wrapper of Generic Repository over same existing one is repeat work. Why reinvent the wheel?
But, some argue that using ORM directly in calling code has following issues:
It makes code little more complicated due to lack of separation of concerns.
Data access code is merged in business logic. As a result, redundant complex query logic spread at multiple places; hard to manage.
As many ORM objects are used in-line in calling code, it is very hard to unit test the code.
As ORM only exposes Generic Repository, it causes many issues mentioned below.
Apart from all above, one other issue generally discussed is "What if we decide to change ORM in future". This should not be key point while taking decision because:
You rarely change ORM, mostly NEVER – YAGNI.
If you change ORM, you have to do huge changes anyway. You may minimize efforts by encapsulating complete data access code (NOT just ORM) inside something. We will discuss that something below.
Considering four issues mentioned above, it may be necessary to create Repositories even though you are using full ORM - This is per case decision though.
Even in that case, Generic Repository must be avoided. It is considered an anti-pattern.
Why generic repository is anti-pattern?
A repository is a part of the domain being modeled, and that domain is not generic.
Not every entity can be deleted.
Not every entity can be added
Not every entity has a repository.
Queries vary wildly; the repository API becomes as unique as the entity itself.
For GetById(), identifier types may be different.
Updating specific fields (DML) not possible.
Generic query mechanism is the responsibility of an ORM.
Most of the ORMs expose an implementation that closely resemble with Generic Repository.
Repositories should be implementing the SPECIFIC queries for entities by using the generic query mechanism exposed by ORM.
Working with composite keys is not possible.
It leaks DAL logic in Services anyway.
Predicate criteria if you accept as parameter needs to be provided from Service layer. If this is ORM specific class, it leaks ORM into Services.
I suggest you read these (1, 2, 3, 4, 5) articles explaining why generic repository is an anti-pattern. This other answer discusses about Repository Pattern in general.
So, I will suggest:
Do NOT use repository at all, directly use ORM in your calling code.
If you have to use repository, then do not try to implement everything with Generic Repository.
Instead, optionally create very simple and small Generic Repository as abstract base class. OR you can use Generic Repository exposed by your ORM as base repository if ORM allows it.
Implement Concrete Repositories as per your need and derive all them from Generic Repository. Expose concrete repositories to calling code.
This way you get all the good of generic repository still bypassing its drawbacks.
Even though very rare, this also helps switching ORM in future as ORM code is cleanly abstracted in DAL/Repositories. Please understand that switching ORM is not a primary objective of Data Access Layer or Repository.
In any case, do not expose Generic Repository to calling code.
Also, do not return IQueryable from concrete repositories. This violates basic purpose of existence of Repositories - To abstract data access. With exposing IQueryable outside the repository, many data access decisions leak into calling code and Repository lose the control over it.
do I need to create a repository for each entity or implement a generic repository for the context
As suggested above, creating repository for each entity is better approach. Note that, Repository should ideally return Domain Model instead of Entity. But this is different topic for discussion.
does a generic repository works for EF Database First?
As suggested above, EF itself exposes Generic Repository. Building one more layer on it is useless. Your image is saying the same thing.
My team devolops a web api application using entity framework,
The Gui is developed by a seperate team.
My question is how should the models be defined? Should we have two projects - one for domain models (database entities) and one for Dtos which are serializable?
Where should the parsing from Dto to domain models should happen and when should it happen the opposite way?
Moreover, sometimes all the data is needed to be sent to the clients.. Should a Dto be created for those cases as well? Or should I return a domain model?
Generally speaking, it's a good idea to not let your entities (database models) leak out of your database layer. However, as with everything in software - this can have its downfalls. One such downfall being is that it starts to increase complexity of your data layer as it involves mapping your entities to their DTO within your database layer, ultimately leaving repositories that are full of similar methods returning different DTO types.
Some people also feel that exposing IQueryables from your data layer is also a bad thing as you start to leak abstractions to different layers - though this has always seemed a little extreme.
Personally, I favour what I feel is a more pragmatic approach and I prefer to use a tool like AutoMapper to automatically map my entities to my DTOs within the business logic layer.
For example:
// Initial configuration loaded on start up of application and cached by AutoMapper
AutoMapper.Mapper.CreateMap<BlogPostEntity, BlogPostDto>();
// Usage
BlogPostDto blogPostDto = AutoMapper.Mapper.Map<BlogPostDto>(blogPostEntity);
AutoMapper also has the ability to configure more complex mapping, though you should try and avoid this if possible by sticking to flatter DTOs.
In addition, another great feature of AutoMapper is the ability to automatically project your entities to DTOs. This results in much cleaner SQL where only the columns within your DTO are queried:
public IEnumerable<BlogPostDto> GetRecentPosts()
{
IEnumerable<BlogPostDto> blogPosts = this.blogRepository.FindAll().Project(this.mappingEngine).To<BlogPostDto>().ToList();
return blogPosts;
}
Moreover, sometimes all the data is needed to be sent to the clients.. Should a Dto be created for those cases as well? Or should I return a domain model?
DTOs should be created for those. Ultimately you don't want your client depending on your data schema, which is exactly what will happen if you expose your entities.
Alternatives: Command/Query Segregation
It behooves me to also highlight that there are also some other alternatives to a typical layered architecture, such as the Command/Query Segregation approach where you model your commands and queries via a mediator. I won't go into it in too much detail as it's a whole other subject but it's one I would definitely favour over a layered approach discussed above. This would result in you mapping your entities to your DTOs directly within the modelled command or query.
I would recommend taking a look at Mediatr for this. The author, Jimmy Bogard who also created AutoMapper also has this video talking about the same subject.
I've had similar requirements in several projects and in most cases we separated at least three layers:
Database Layer
The database objects are simple one-to-one representations of the database tables. Nothing else.
Domain Layer
The domain layer defines entity objects which represent a complete business object. In our defintion an entity aggregates all data which is directly associated to the entity and can not be regarded as a dedicated entity.
An exmaple: In an application which handles invoices you have a table invoice and invoice_items. The business logic reads both tables and combines the data into a entity object Invoice.
Application Layer
In the application layer we define models for all kind of data we want to send to the client. Pass-through of domain entity objects to save time is tempting but strictly prohibited. The risk to publish any data which shouldn't be published is too high. Furthermore you gain more freedom regarding the design of your API. That's what helps you to fit your last requirement (send all data to the client): Just built a new model which aggregates the data of all domain objects you need to send.
This is the minimum set of layers we use in all projects. There were hundreds of cases where we've been very happy to have several abstraction layers which gave us enough possibilities to enhance and scale an application.
I use Entity Framework as ORM in my .net MVC project. I've implemented the Repository-Pattern (generic) to get/save/update/remove DAOs (Data Access Objects). I also have Business Objects which contain all the business logic. I have - for example - a DAO called Student and a BO (Business Object) called Student as well. The BO contains the logic, the DAO just the data stored in the DB.
Now I am wondering if the Student-Repository should return the Business-Object instead of the DAO?
I could achieve that using Automapper by converting the DAO to a Business Object before returning it from the Repository.Get(). Same with all the other methods. But is this a good practice?
Update
I have a Data Access Layer project and a project for the Business Logic. Entity Framework creates its entities in partial classes (into the Data Access Project) so I could actually extend the entities with other partial classes but the problem is that I reference the Data Access Project in my Business project and I don't have access to the logic code within the Data Access project. So I have to put the logic inside the Business project but as it is not possible to create partial classes over two projects I have to go another way... or do you have a good idea how to structure and solve the problem in a better way?
IMHO there are several goals (some competing):
Make business logic testable in isolation
Design domain objects to match your domain
Decouple data access from everything else
Keep it simple
Can you test your business logic without a database? Probably yes, whether the classes are EF POCO entities or mapped from DAOs.
Do your domain objects match your domain? Are their names well-chosen? Are they always in a valid state? (This can be difficult with a bunch of public read/write properties.) Domain-driven design considerations apply here. (I'm no expert in that.)
Could you swap out EF for Dapper, SQL Server for MongoDB, or current data access for a web service call without changing anything outside the data access layer - with confidence? My suspicion is no. Generic repositories tend to leak IQueryable into other layers. Not everything supports querying, and provider implementations vary. Unit tests typically use LINQ to Objects, which does not behave the same as LINQ to Entities. Also, if you want to extract a web service contract, you would have to look through all classes to find all the queries. See IQueryable is Tight Coupling.
Finally, do you need all of this? If your application's purpose is CRUD data access with no business logic above simple validation, maybe not. These considerations definitely apply to a complex application or site.
Yes, that's totally good practice. Usually you have repository interfaces defined in domain assembly. These interfaces are used by domain services, and implemented in persistence assembly. Entity Framework allows you to map business entities fluently, without polluting them with attributes or forcing them to inherit from some specific base class (POCO entities). That makes your domain model Persistence Ignorant.
I've just started working with the Entity framework and I'm confused about how the classes normally in the business layer fit in with the entities that are created by the Entity Framework.
When working with classic ADO.NET, I would have a class called Customer for example and then another class called DALCustomer to handle database interaction, in this structure I would have put the code to do calculations, filtering and delcare an instance of the DAL withing Customer for saving, updating and deleting in the Customer class.
With the Entity Framework, if you have a table called Customer, the Entity framework creates an entity called Customer and this is where my confusion begins, does this entity remove the need for a Customer in the business layer? So in essence all the fields and methods that normally go in the business layer go in the entity generated by the Entity Framework? Or should a class still exist in the business layer called CustomerBL for example, that still contains the fields and methods needed to accomplish the business logic required for calculations, filtering and still needs an instance of the EF DAL declared to handle data access?
If there should be a business class, in this case CustomerBL, one other question jumps to mind, should the fields that are created in the customer entity be recreated in CustomerBL or should an instance of the Customer entity be declared in CustomerBL so there would be no need to have the fields declared in 2 locations?
Entity framework was designed with separation between data model and conceptual model in mind. It supports inheritance, entity splitting (not EF core), table splitting, complex types or owned types, and transparent many-to-many associations (not EF core), all of which allow molding the domain model to one's needs without being constrained too much by the data store model. EF core supports shadow properties which can be used to hide cross-cutting concerns from the exposed class model.
The code-first approach allows working with POCOs in which only a few properties are mapped to data store columns and others serve domain goals. Model-first and Database-first generate partial classes, allowing one to extend the generated code.
Of course this separation of conceptual model and store model can only succeed to a certain extent. Some things work against this goal of persistence ignorance. For instance -
If lazy loading is desirable, it is necessary to declare navigation properties as virtual, so EF can override them in proxy types. Domain-driven design (DDD) would encourage using virtual only when polymorphism is required.
It is very convenient to have primitive foreign key properties (say, ParentId) accompanying the "real" associations (a Parent reference). Purists consider this a violation of DDD principles.
The EF class model will is part of a data access layer and should primarily serve that goal. Therefore, it will contain many reciprocal relationships, in order to benefit from navigation properties as much as possible when writing LINQ queries. These mutual relationships are another violation of DDD principles.
There is a large number of differences between LINQ-to-objects and LINQ-to-entities. You just can't ignore the fact that you are LINQ-ing against a totally different universe than objects in memory. This is referred to as tight coupling, or leaky abstraction.
EF can only map concrete classes, no interfaces.
But then... generally I'm happy with using generated EF classes or POCOs from a code-first model as domain classes. So far, I've never seen a frictionless transition from one data store or ORM to another, if it happens at all. Persistence ignorance is a fiction. Idiosyncrasies from the DAL easily leave a footprint in the domain. Only when you have to code for different data stores/models or when stores/models are expected to change relatively often it pays off to minimize this footprint as much as possible or abstract it away completely.
Another factor that may promote EF classes as domain classes is that many applications today have multiple tiers, where (serialized) different view models or DTOs are sent to a client. Using domain classes in UIs hardly ever fits the bill. You may as well use the EF class model as the domain and have services return dedicated models and DTOs as required by a UI or service consumers. Another abstraction layer may be more of a burden than a blessing, if only performance-wise.
In my opinion the whole point of using POCOs as entities that can be persisted is to remove the distinction between "database entities" and "business entities". "Entities" are supposed to be "business entities" that directly can be persisted to and loaded from a data store and therefore act as "database entities" at the same time. By using POCOs the business entities are decoupled from the specific mechanism to interact with a database.
You can move the entities into a separate project - for example - that has no references to any EF assembly and yet use them in a database layer project to manage persistence.
This does not mean that you can design your business entities completely without having the requirements for EF in mind. There are limitations you need to know to avoid trouble when you come to the point to map the business entities to a database schema using EF, for instance:
You must make navigation properties (references or collections of references to other entities) virtual to support lazy loading with EF
You cannot use IEnumerable<T> for collections that have to be persisted. It must be ICollection<T> or a more derived type.
It's not easy to persist private properties
The type char is not supported by EF and you can't use it if you want to persist its values
and more...
But an additional set of entities is - in my opinion - an additional layer of complexity that should be justified to be really needed if the mentioned limitations are too tight for your project.
YA2C (Yet another 2 cents :))
I don't know if it's considered a good practice by others but personally this is how i handled this in the past:
The classes generated by EF are your DAL, and then for BL create a complementary set of classes in which you will have the structure you require (like maybe merging data from related entities in a one to one relationship) and other business logic concerns are handled (custom validation like implementing IDataErrorInfo to make it play nice with the UI in WPF for instance) and also create classes that would contain all the business layer methods relating to a type of entity, that use the BL instances and convert to and from EF entities to the BL objects.
So, for instance, you have Customer in your db. EF will generate a class Customer, and in the BL there will be a Customer (prefix, suffix, etc.) class and a CustomerLogic class. In the BL Customer class you can do whatever is needed to satisfy requirements without having to tamper with the EF entities and in the CustomerLogic class you would have BL methods (load most valued customers, save customer with extra data, etc.).
Now, this enables you to be loosely coupled to the datasource implementation. Another example of why this has benefited me in the past (in a WPF project) is that you can do stuff like implement IDataErrorInfo and implement validation logic in the CustomerBL classes so that when you bind the entity to a create/edit form on the UI you will benefit from the built in functionality provided by WPF.
...My 2 cents, i am also curious to find out what is the best practice or what other solutions/points of view are.
Also perhaps related to this topic - Code-first vs Model/Database-first
This topic maybe a bit old but this may help. Andras Nemes pointed out in his blog the concern of using DDD (domain driven design) over the technology driven design such as EF, MVC, etc.
http://dotnetcodr.com/2013/09/12/a-model-net-web-service-based-on-domain-driven-design-part-1-introduction/
I used the business logic to write my methods and return the results in its created view like:
namespace Template.BusinessLogic
{
public interface IApplicantBusiness
{
List<Template.Model.ApplicantView> GetAllApplicants();
void InsertApplicant(Template.Model.ApplicantView applicant);
}
}