I'm creating ASP.NET Core application using Dapper as ORM.
What is the proper flow to saves objects into multiple tables?
In my app architecture I got standard web api controllers that invoke command/query handlers that calculate/invoke other services/repositories etc.
My entities/db tables are User, Order, Product. One of my CommandHandler create user, then calculate products, orders etc. I just wonder how to save these objects into database. I see 2 solutions:
1.) I create some kind of DTO for all the calculated stuff during command handler:
public class TestDto
{
public User User;
public IList<Orders> Orders;
}
calculate all the stuff, fill the DTO one by one, and then at the end of command handler invoke all repositories:
...using (var ts = new Transaction)
{
_userRepository.Save(dto.User);
_ordersRepository.Save(dto.Orders);
ts.Complete
}
etc..
2.) Create transaction per whole command handler, and save user immediately after calculating him in memory, then calculate orders and save them immediately as well, and the same with orders.
You're going to need at least one transaction regardless. You don't want to save an incomplete order. I say at least one because it depends on if your user can exist separately from the order.
You could create and save the user first in one transaction, then save the order in a second transaction. If anything goes wrong while saving the order, you still have the user committed to the database. With option #2, you could perform the two transactions within your command handler.
With option #1, I'd be concerned about introducing an extra object tying users and orders together. You may find times when you need to make changes to just the user or an order separately from each other. The TestDto wouldn't be helpful.
Related
I'm using .Net Core 2.1. I'm using more than one DbContext. I'm creating a DbContextFactory for every context. But, I want to do this in a generic way. I want to create only one DbContextFactory. How can I achieve this?
MyDbContextFactory.cs
public interface IDbContextFactory<TContext> where TContext : DbContext
{
DbContext Create();
}
public class MyDbContextFactory : IDbContextFactory<MyDbContext>
{
public IJwtHelper JwtHelper { get; set; }
public MyDbContextCreate()
{
return new MyDbContext(this.JwtHelper);
}
DbContext IDbContextFactory<MyDbContext>.Create()
{
throw new NotImplementedException();
}
}
UnitOfWork.cs
public class UnitOfWork<TContext> : IUnitOfWork<TContext> where TContext : DbContext
{
public static Func<TContext> CreateDbContextFunction { get; set; }
protected readonly DbContext DataContext;
public UnitOfWork()
{
DataContext = CreateDbContextFunction();
}
}
MyDbContext.cs
public class MyDbContext : DbContext
{
private readonly IJwtHelper jwtHelper;
public MyDbContext(IJwtHelper jwtHelper) : base()
{
this.jwtHelper = jwtHelper;
}
}
So you have a database, and a class that represent this database: your DbContext, it should represent the tables and the relations between the tables that are in your database, nothing more.
You decided to separate the operations on your database from the database itself. That is a good thing, because if several users of your database want to do the same thing, they can re-use the code to do it.
For instance, if you want to create "an Order for a Customer with several OrderLines, containing ordered Products, agreed Prices, amount, etc", you'll need to do several things with your database: check if the customer already exists, check if all products already exist, check if there are enough items, etc.
These things are typically things that you should not implement in your DbContext, but in a separate class.
If you add a function like: CreateOrder, then several users can re-use this function. You'll only have to test this only once, and if you decide to change something in your order model, there is only one place where you'll have to change the creation of an Order.
Other advantages of separating the class that represents your database (DbContext)
from the class that handles this data is that will be easier to change the internal structure without having to change the users of your database. You can even decide to change from Dapper to Entity Framework without having to change usage. This makes it also easier to mock the database for test purposes.
Functions like CreateOrder, QueryOrder, UpdateOrder already indicate that they are not generic database actions, they are meant for an Ordering database, not for a School database.
This might have the effect that unit-of-work might not be a proper name for the functionality you want in the separate class. A few years ago, unit-of-work was mainly meant to represent actions on a database, not really a specific database, I'm not really sure about this, because I saw fairly soon that a real unit-of-work class would not enhance functionality of my DbContext.
Quite often you see the following:
A DbContext class that represents your Database: the database that you created, not any generic idea of databases
A Repository class that represent the idea of storing your data somewhere, how this is stored, it could be a DbContext, but also a CSV-file, or a collection of Dictionary classes created for Test purposes. This Repository has an IQueryable, and functions to Add / Update / Remove (as far as needed
A class that represents your problem: the ordering model: Add Order / Update Order / Get Order of a Customer: this class really knows everything about an Order, for instance that it has an OrderTotal, which is probably nowhere to be found in your Ordering database.
Outside DbContext you sometimes may need SQL, for instance to improve efficiency of a call. Outside Repository it should not be seen that you are using SQL
Consider to separate the concerns: how to save your data (DbContext), how to CRUD (create, fetch, update, etc) the data (Repository), how to use the data (combine the tables)
I think what you want to do in your unit-of-work should be done inside the repository. Your Ordering class should create the Repository (which creates the DbContext), query several items to check the data it has to Add / Update, do the Add and Update and save the changes. After that your ordering class should Dispose the Repository, which in turn will Dispose the DbContext.
The Repository class will look very similar to the DbContext class. It has several sets that represent the tables. Every set will implement IQueryable<...> and allow to Add / Update / Remove, whatever is needed.
Because of this similarity in functions you could omit the Repository class and let your Ordering class use the DbContext directly. However, keep in mind, that changes will be bigger if in future you decide that you don't want to use entity framework anymore but some newer concept, or maybe return back to Dapper, or even more low level. SQL will seep through into your Ordering class
What to choose
I think you should answer several questions for yourself:
Is there really only one database that should be represented by your DbContext, could it be that the same DbContext should be used in a 2nd database with the same layout. Think of a test database, or a development database. Wouldn't it be easer / better testable / better changeable, to let your program create the DbContext that is to be used?
Is there really one group of Users of your DbContext: should everyone have the possibility to Delete? to Create? Could it be that some programs only want to query data (the program that e-mails the orders), and that order programs need to add Customers. And maybe another program needs to Add and Update Products, and the amount of Products in the warehouse. Consider Creating different Repository classes for them. Each Repository gets the same DbContext, because they are all accessing the same database
Similarly: only one data processing class (the above mentioned ordering class): should the process that handles Orders, be able to change product prices and add items to the stock?
The reason that you need the factories, is because you don't want to let your "main" program decide what items it should create for the purpose it is running right now. Code would be much easier if you created the items yourself:
Creation sequence for an Ordering Process:
IJwtHelper jwtHelper = ...;
// The product database: all functionality to do everything with Products and Orders
ProductDbContext dbContext = new ProductDbContext(...)
{
JwtHelper = jwtHelper,
...
};
// The Ordering repository: everything to place Orders,
// It can't change ProductPrices, nor can it stock the wharehouse
// So no AddProduct, not AddProductCount,
// of course it has TakeNrOfProducts, to decrease Stock of ordered Products
OrderingRepository repository = new OrderingRepository(...) {DbContext = dbContext};
// The ordering process: Create Order, find Order, ...
// when an order is created, it checks if items are in stock
// the prices of the items, if the Customer exists, etc.
using (OrderingProcess process = new OrderingProcess(...) {Repository = repository})
{
... // check if Customer exists, check if all items in stock, create the Order
process.SaveChanges();
}
When the Process is Disposed, the Repository is Disposed, which in turns Disposes the DbContext.
Something similar for the process that e-mails the Orders: It does not have to check the products, nor create customers, it only has to fetch data, and maybe update that an order has been e-mailed, or that e-mailing failed.
IJwtHelper jwtHelper = ...;
// The product database: all functionality to do everything with Products and Orders
ProductDbContext dbContext = new ProductDbContext(...) {JwtHelper = jwtHelper};
// The E-mail order repository: everything to fetch order data
// It can't change ProductPrices, nor can it stock the wharehouse
// It can't add Customers, nor create Orders
// However it can query a lot: customers, orders, ...
EmailOrderRepository repository = new EmailOrderRepository(...){DbContext = dbContext};
// The e-mail order process: fetch non-emailed orders,
// e-mail them and update the e-mail result
using (EmailOrderProcess process = new EmailOrderProcess(...){Repository = repository}
{
... // fetch the orders that are not e-mailed yet
// email the orders
// warning about orders that can't be emailed
// update successfully logged orders
repository.SaveChanges();
See how much easier you make the creation process, how much more versatile you make it: give the DbContext a different JwtHelper, and the data is logged differently, give the Repository a different DbContext and the data is saved in a different database, give the Process a different Repository, and you'll use Dapper to execute your queries.
Testing will be easier: create a Repository that uses Lists to save the tables, and testing your process with test data will be easy
Changes in databases will be easier. If for instance you later decide to separate your databases into one for your stock and stock prices and one for Customers and Orders, only one Repository needs to change. None of the Processes will notice this change.
Conclusion
Don't let the classes decide which objects they need. Let the creator of the class say: "hey, you need a DbContext? Use this one!" This will omit the need of factories and such
Separate your actual database (DbContext) from the concept of storing and retrieving data (Repository), use a separate class that handles the data without knowing how this data is stored or retrieved (The process class)
Create several Repositories that can only access the data they need to perform the task (+data that can be foreseen in future after expected changed). Don't make too much Repositories, but also not one that can do everything.
Create process classes that do only what they are meant to do. Don't create one process class with 20 different tasks. It will only make it more difficult to describe what it should do, more difficult to test it and more difficult to change the task
If you want to reuse the existing implementation, EF Core 5 provides DbContext Factory out of the box now:
https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-5.0/whatsnew#dbcontextfactory
Make sure you dispose it correctly, as it's instances are not managed by the application's service provider.
See Microsoft documentation
Using a DbContext factory
I am currently trying to write an application in DDD allowing an entity to be created, updated and deleted. A change to an entity must be approved by another person. The application must also keep track of what changes were made to an entity. The simplified domain model looks like this:
The application has one bounded context containing ChangeSet, Enity and EntityHistory where ChangeSet is the aggregate root. I designed the aggregate this way because of an Entity should not be changed without a ChangeSet and furthermore a ChangeSet should be saved together with the edited entities in one transaction. On that account I designed a single aggregate.
The design work pretty good when creating new entities:
private void CreateChangeSet()
{
var repository = new ChangeSetRepository();
var entities = new List<Entity>
{
new Entity(Guid.NewGuid(), "Test1", new TagStatus(1, EntityState.Pending));
};
var changeSet = new ChangeSet("a user", "Added a new entity", DateTime.Now, ApprovalState.Submitted, entities);
repository.Insert(changeSet);
}
However, problems arise in my design occur when I am trying to edit an entity:
private void EditEnity()
{
var repository = new ChangeSetRepository();
var entity = repository.GetEntityByName("Test1");
entity.AssignName("a new name");
var entities = new List<Entity>{entity};
var cs = new ChangeSet("a user", "Added a new entity", DateTime.Now, ApprovalState.Submitted, entities);
repository.Insert(cs);
}
As far as I know an repository should return aggregates only, which would mean that in order to change an Entity I must first search for a ChangeSet which does not make sense. Is it a bad practice to return a sub-entity of an aggregate even if you perform changes only be the aggregate root?
I have searched the internet for an answer an many people are pointing out that this kind of query can point out a wrong design of aggregates. Which makes me think again if instead of one aggregate I need two aggregates one for the ChangeSet and one containing Entity and EntityHistory. Should I use two aggregates instead of one? If so how can I do this within a single transaction?
A further indication for two aggregates are user interface requirements like 'the user wants to see a change history for an entity' or 'show me all entities in a view'. On the one hand this indicates two aggregates on the other hand I have a feeling that ChangeSet and Entities should really belong together .
To sum up my questions:
Should I use one or two aggregates in my design?
If one aggregate: is it a bad practice to return a sub-entity of an aggregate even if you perform changes only through the aggregate root?
If two aggregates: how can I save the ChangeSet and the associated Entities in one transaction?
TL;DR:
You should use one entity.
Yes, it is bad practice as the behaviour should be exposed by the aggregate; also, reconstructing the Entity would require the Entity to know how to query ChangeSet; unless you orchestrate this at the service level, it is not great design.
You should not do it, as an aggregate root represents, IMHO, a transactional boundary.
Additional thoughts
If I understand correctly, you are trying to do what Event Sourcing does naturally, with the addition of the approval workflow. Events in an Event Store are approximately what you define with a ChangeSet.
If this is correct, you could model this elegantly in ES by:
Call an Edit Entity API that takes as input the bulk of the changes for an Entity
The API:
Builds a ChangeEntityCommand from the API input (command may fail validation);
Retrieves the Entity;
Invokes the corresponding Handler in the Entity aggregate, which in turn emits a ChangeQueuedForApprovalEvent.
Commits the Entity in the EventStore
An EventHandler will intercept the event above and take care of updating the approval view.
When the approver gives the green light, a similar flow will emit a ChangeApprovedEvent containing the same data of the former event. This event is the one that actually transforms the Entity.
Lastly, I do not believe that the ChangeSet modelling really suits DDD, as it fails to capture the intent of the change.
Hope this helps and good luck with your project.
I read much about DDD, but I can't steel see how use it in a real life. I make some examples (with C# and Entity Framework) for things which I can't understand.
Add money for User. User has just amount of money.
Ok, It's simple. It's example of model User
class User
{
public decimal Balance {get; private set; }
public void AddMoney(decimal sum)
{
if(sum>0)
Balance+=sum;
}
}
But how can I use it?
Fetch user from database
Update user - performed by domain model
Save changes
So, the first question is where I should perform fetching and saving data from database (repository)? I can't do this inside my domain model.
User has history of transaction, not just simple sum
class User
{
public ICollection<Transaction> Transactions {get; private set; }
public void AddMoney(decimal sum)
{
if(sum>0)
Transactions.Add(new Transaction(sum));
}
}
In this case I should fetch user from database and then EF will add new Entity to collection. But it isn't efficient, more efficient is do something like this:
transactionsRepository.Add(new Transaction(sum, userId));
but it isn't DDD-way.
Get money from one user and transfer to another
In this case operation affects multiple models. Where I should put logic which works with multiple models? (Maybe this example isn't good enough).
Get users' current balance
User's balance is a sum of all transactions
decimal Balance() => transactionsRepository.Get().Sum(x=>x.TransactionSum);
In this case query contains logic - how I should fetch data to do something, not simple fetch\save entities like in other examples. Where I should place queries with logic? Get total balance, get last unread messages etc.
So, the first question is where I should perform fetching and saving data from database (repository)? I can't do this inside my domain model.
You do this in an Application service in a Layered architecture or a command handler in a CQRS architecture .
But it isn't efficient, more efficient is do something like this
It is more efficient but indeed not the DDD way. Aggregates should have no dependency to repository. They work only with state that is stored in memory. An application service is responsible fir loading and storing an aggregate.
Where I should put logic which works with multiple models?
In Sagas/Process managers.
Where I should place queries with logic?
It depends on what do you query.
If not using CQRS:
If you query data from an Aggregate in an Aggregate's method.
If you query a specific lists of Aggregates them you put that logic in the repository.
If using CQRS then any query is done on a read-model/projection.
I'm using CQRS + ES and I have a modeling problem that can't find a solution for.
You can skip the below and answer the generic question in the title: Where would you query data needed for business logic?
Sorry of it turned out to be a complex question, my mind is twisted at the moment!!!
Here's the problem:
I have users that are members of teams. It's a many to many relationship. Each user has an availability status per team.
Teams receive tickets, each with a certain load factor, that should be assigned to one of the team's members depending on their availability and total load.
First Issue, I need to query the list of users that are available in a team and select the one with the least load since he's the eligible for assignment.(to note that this is one of the cases, it might be a different query to run)
Second Issue, load factor of a ticket might change so i have to take that into consideration when calculating the total load per user . Noting that although ticket can belong to 1 team, the assignment should be based on the user total load and not his load per that team.
Currently a TicketReceivedEvent is received by this bounded context and i should trigger a workflow to assign that ticket to a user.
Possible Solutions:
The easiest way would be to queue events and sequentially send a command AssignTicketToUser and have a service query the read model for the user id, get the user and user.assignTicket(Ticket). Once TicketAssignedEvent is received, send the next assignment command. But it seems to be a red flag to query the read model from within the command handler! and a hassle to queue all these tickets!
Have a process manager per user with his availability/team and tickets assigned to that user. In that case we replace the query to the read side by a "process manager lookup" query and the command handler would call Ticket.AssignTo(User). The con is that i think too much business logic leaked outside the domain model specifically that we're pulling all the info/model from the User aggregate to make it available for querying
I'm inclined to go with the first solution, it seems easier to maintain, modify/extend and locate in code but maybe there's something i'm missing.
Always (well, 99.99% of cases) in the business/domain layer i.e in your "Command" part of CQRS. This means that your repositories should have methods for the specific queries and your persistence model should be 'queryable' enough for this purpose. This means you have to know more about the use cases of your Domain before deciding how to implement persistence.
Using a document db (mongodb, raven db or postgres) might make work easier. If you're stuck with a rdbms or a key value store, create querying tables i.e a read model for the write model, acting as an index :) (this assumes you're serializing objects). If you're storing things relationally with specific table schema for each entity type (huge overhead, you're complicating your life) then the information is easily queryable automatically.
Why can't you query the aggregates involved?
I took the liberty to rewrite the objective:
Assign team-ticket to user with the lowest total load.
Here we have a Ticket which should be able to calculate a standard load factor, a Team which knows its users, and a User which knows its total load and can accept new tickets:
Update: If it doesn't feel right to pass a repository to an aggregate, it can be wrapped in a service, in this case a locator. Doing it this way makes it easier to enforce that only one aggregate is updated at a time.
public void AssignTicketToUser(int teamId, int ticketId)
{
var ticket = repository.Get<Ticket>(ticketId);
var team = repository.Get<Team>(teamId);
var users = new UserLocator(repository);
var tickets = new TicketLocator(repository);
var user = team.GetUserWithLowestLoad(users, tickets);
user.AssignTicket(ticket);
repository.Save(user);
}
The idea is that the User is the only aggregate we update.
The Team will know its users:
public User GetGetUserWithLowestLoad(ILocateUsers users, ILocateTickets tickets)
{
User lowest = null;
foreach(var id in userIds)
{
var user = users.GetById(id);
if(user.IsLoadedLowerThan(lowest, tickets))
{
lowest = user;
}
}
return lowest;
}
Update: As a ticket may change load over time, the User needs to calculate its current load.
public bool IsLoadedLowerThan(User other, ILocateTickets tickets)
{
var load = CalculateLoad(tickets);
var otherLoad = other.CalculateLoad(tickets);
return load < otherLoad;
}
public int CalculateLoad(ILocateTickets tickets)
{
return assignedTicketIds
.Select(id => tickets.GetById(id))
.Sum(ticket.CalculateLoad());
}
The User then accepts the Ticket:
public void AssignTicket(Ticket ticket)
{
if(ticketIds.Contains(ticket.Id)) return;
Publish(new TicketAssignedToUser
{
UserId = id,
Ticket = new TicketLoad
{
Id = ticket.Id,
Load = ticket.CalculateLoad()
}
});
}
public void When(TicketAssignedToUser e)
{
ticketIds.Add(e.Ticket.Id);
totalLoad += e.Ticket.Load;
}
I would use a process manager / saga to update any other aggregate.
You can query the data you need in your application service. This seems to be similar to your first solution.
Usually, you keep your aggregates cross-referenced, so I am not quite sure where the first issue comes from. Each user should have a list of teams it belongs to and each group has the list of users. You can complement this data with any attributes you want, including, for example, availability. So, when you read your aggregate, you have the data directly available. Surely, you will have lots of data duplication, but this is very common.
In the event sourced model never domain repositories are able to provide any querying ability. AggregateSource by Yves Reynhout is a good reference, here is the IRepository interface there. You can easily see there is no "Query" method in this interface whatsoever.
There is also a similar question Domain queries in CQRS
I have a somewhat complex permission system that uses six database tables in total and in order to speed it up, I would like to cache these tables in memory instead of having to hit the database every page load.
However, I'll need to update this cache when a new user is added or a permission is changed. I'm not sure how to go about having this in memory cache, and how to update it safely without causing problems if its accessed at the same time as updating
Does anyone have an example of how to do something like this or can point me in the right direction for research?
Without knowing more about the structure of the application, there are lots of possible options. One such option might be to abstract the data access behind a repository interface and handle in-memory caching within that repository. Something as simple as a private IEnumerable<T> on the repository object.
So, for example, say you have a User object which contains information about the user (name, permissions, etc.). You'd have a UserRepository with some basic fetch/save methods on it. Inside that repository, you could maintain a private static HashSet<User> which holds User objects which have already been retrieved from the database.
When you fetch a User from the repository, it first checks the HashSet for an object to return, and if it doesn't find out it gets it from the database, adds it to the HashSet, then returns it. When you save a User it updates both the HashSet and the database.
Again, without knowing the specifics of the codebase and overall design, it's hard to give a more specific answer. This should be a generic enough solution to work in any application, though.
I would cache items as you use it, which means on your data layer when you are getting you data back you check on your cache if it is available there otherwise you go to the database and cache the result after.
public AccessModel GetAccess(string accessCode)
{
if(cache.Get<string>(accessCode) != null)
return cache.Get<string>(accessCode);
return GetFromDatabase(accessCode);
}
Then I would think next on my cache invalidate strategy. You can follow two ways:
One would be set expire data to be 1 hour and then you just hit the database once in a hour.
Or invalidate the cache whenever you update the data. That is for sure the best but is a bit more complex.
Hope it helps.
Note: you can either use ASP.NET Cache or another solution like memcached depending on your infrastructure
Is it hitting the database every page load that's the problem or is it joining six tables that's the problem?
If it's just that the join is slow, why not create a database table that summarizes the data in a way that is much easier and faster to query?
This way, you just have to update your summary table each time you add a user or update a permission. If you group all of this into a single transaction, you shouldn't have issues with out-of-sync data.
You can take advantage of ASP.NET Caching and SqlCacheDependency Class. There is article on MSDN.
You can use the Cache object built in ASP.Net. Here is an article that explains how.
I can suggest cache such data in Application state object. For thread-safe usage, consider using lock operator. Your code would look something like this:
public void ClearTableCache(string tableName)
{
lock (System.Web.HttpContext.Current)
{
System.Web.HttpContext.Current.Application[tableName] = null;
}
}
public SomeDataType GetTableData(string tableName)
{
lock (System.Web.HttpContext.Current)
{
if (System.Web.HttpContext.Current.Application[tableName] == null)
{
//get data from DB then put it into application state
System.Web.HttpContext.Current.Application[tableName] = dataFromDb;
return dataFromDb;
}
return (SomeDataType)System.Web.HttpContext.Current.Application[tableName];
}
}