I'm using CQRS + ES and I have a modeling problem that can't find a solution for.
You can skip the below and answer the generic question in the title: Where would you query data needed for business logic?
Sorry of it turned out to be a complex question, my mind is twisted at the moment!!!
Here's the problem:
I have users that are members of teams. It's a many to many relationship. Each user has an availability status per team.
Teams receive tickets, each with a certain load factor, that should be assigned to one of the team's members depending on their availability and total load.
First Issue, I need to query the list of users that are available in a team and select the one with the least load since he's the eligible for assignment.(to note that this is one of the cases, it might be a different query to run)
Second Issue, load factor of a ticket might change so i have to take that into consideration when calculating the total load per user . Noting that although ticket can belong to 1 team, the assignment should be based on the user total load and not his load per that team.
Currently a TicketReceivedEvent is received by this bounded context and i should trigger a workflow to assign that ticket to a user.
Possible Solutions:
The easiest way would be to queue events and sequentially send a command AssignTicketToUser and have a service query the read model for the user id, get the user and user.assignTicket(Ticket). Once TicketAssignedEvent is received, send the next assignment command. But it seems to be a red flag to query the read model from within the command handler! and a hassle to queue all these tickets!
Have a process manager per user with his availability/team and tickets assigned to that user. In that case we replace the query to the read side by a "process manager lookup" query and the command handler would call Ticket.AssignTo(User). The con is that i think too much business logic leaked outside the domain model specifically that we're pulling all the info/model from the User aggregate to make it available for querying
I'm inclined to go with the first solution, it seems easier to maintain, modify/extend and locate in code but maybe there's something i'm missing.
Always (well, 99.99% of cases) in the business/domain layer i.e in your "Command" part of CQRS. This means that your repositories should have methods for the specific queries and your persistence model should be 'queryable' enough for this purpose. This means you have to know more about the use cases of your Domain before deciding how to implement persistence.
Using a document db (mongodb, raven db or postgres) might make work easier. If you're stuck with a rdbms or a key value store, create querying tables i.e a read model for the write model, acting as an index :) (this assumes you're serializing objects). If you're storing things relationally with specific table schema for each entity type (huge overhead, you're complicating your life) then the information is easily queryable automatically.
Why can't you query the aggregates involved?
I took the liberty to rewrite the objective:
Assign team-ticket to user with the lowest total load.
Here we have a Ticket which should be able to calculate a standard load factor, a Team which knows its users, and a User which knows its total load and can accept new tickets:
Update: If it doesn't feel right to pass a repository to an aggregate, it can be wrapped in a service, in this case a locator. Doing it this way makes it easier to enforce that only one aggregate is updated at a time.
public void AssignTicketToUser(int teamId, int ticketId)
{
var ticket = repository.Get<Ticket>(ticketId);
var team = repository.Get<Team>(teamId);
var users = new UserLocator(repository);
var tickets = new TicketLocator(repository);
var user = team.GetUserWithLowestLoad(users, tickets);
user.AssignTicket(ticket);
repository.Save(user);
}
The idea is that the User is the only aggregate we update.
The Team will know its users:
public User GetGetUserWithLowestLoad(ILocateUsers users, ILocateTickets tickets)
{
User lowest = null;
foreach(var id in userIds)
{
var user = users.GetById(id);
if(user.IsLoadedLowerThan(lowest, tickets))
{
lowest = user;
}
}
return lowest;
}
Update: As a ticket may change load over time, the User needs to calculate its current load.
public bool IsLoadedLowerThan(User other, ILocateTickets tickets)
{
var load = CalculateLoad(tickets);
var otherLoad = other.CalculateLoad(tickets);
return load < otherLoad;
}
public int CalculateLoad(ILocateTickets tickets)
{
return assignedTicketIds
.Select(id => tickets.GetById(id))
.Sum(ticket.CalculateLoad());
}
The User then accepts the Ticket:
public void AssignTicket(Ticket ticket)
{
if(ticketIds.Contains(ticket.Id)) return;
Publish(new TicketAssignedToUser
{
UserId = id,
Ticket = new TicketLoad
{
Id = ticket.Id,
Load = ticket.CalculateLoad()
}
});
}
public void When(TicketAssignedToUser e)
{
ticketIds.Add(e.Ticket.Id);
totalLoad += e.Ticket.Load;
}
I would use a process manager / saga to update any other aggregate.
You can query the data you need in your application service. This seems to be similar to your first solution.
Usually, you keep your aggregates cross-referenced, so I am not quite sure where the first issue comes from. Each user should have a list of teams it belongs to and each group has the list of users. You can complement this data with any attributes you want, including, for example, availability. So, when you read your aggregate, you have the data directly available. Surely, you will have lots of data duplication, but this is very common.
In the event sourced model never domain repositories are able to provide any querying ability. AggregateSource by Yves Reynhout is a good reference, here is the IRepository interface there. You can easily see there is no "Query" method in this interface whatsoever.
There is also a similar question Domain queries in CQRS
Related
I'm creating ASP.NET Core application using Dapper as ORM.
What is the proper flow to saves objects into multiple tables?
In my app architecture I got standard web api controllers that invoke command/query handlers that calculate/invoke other services/repositories etc.
My entities/db tables are User, Order, Product. One of my CommandHandler create user, then calculate products, orders etc. I just wonder how to save these objects into database. I see 2 solutions:
1.) I create some kind of DTO for all the calculated stuff during command handler:
public class TestDto
{
public User User;
public IList<Orders> Orders;
}
calculate all the stuff, fill the DTO one by one, and then at the end of command handler invoke all repositories:
...using (var ts = new Transaction)
{
_userRepository.Save(dto.User);
_ordersRepository.Save(dto.Orders);
ts.Complete
}
etc..
2.) Create transaction per whole command handler, and save user immediately after calculating him in memory, then calculate orders and save them immediately as well, and the same with orders.
You're going to need at least one transaction regardless. You don't want to save an incomplete order. I say at least one because it depends on if your user can exist separately from the order.
You could create and save the user first in one transaction, then save the order in a second transaction. If anything goes wrong while saving the order, you still have the user committed to the database. With option #2, you could perform the two transactions within your command handler.
With option #1, I'd be concerned about introducing an extra object tying users and orders together. You may find times when you need to make changes to just the user or an order separately from each other. The TestDto wouldn't be helpful.
I read much about DDD, but I can't steel see how use it in a real life. I make some examples (with C# and Entity Framework) for things which I can't understand.
Add money for User. User has just amount of money.
Ok, It's simple. It's example of model User
class User
{
public decimal Balance {get; private set; }
public void AddMoney(decimal sum)
{
if(sum>0)
Balance+=sum;
}
}
But how can I use it?
Fetch user from database
Update user - performed by domain model
Save changes
So, the first question is where I should perform fetching and saving data from database (repository)? I can't do this inside my domain model.
User has history of transaction, not just simple sum
class User
{
public ICollection<Transaction> Transactions {get; private set; }
public void AddMoney(decimal sum)
{
if(sum>0)
Transactions.Add(new Transaction(sum));
}
}
In this case I should fetch user from database and then EF will add new Entity to collection. But it isn't efficient, more efficient is do something like this:
transactionsRepository.Add(new Transaction(sum, userId));
but it isn't DDD-way.
Get money from one user and transfer to another
In this case operation affects multiple models. Where I should put logic which works with multiple models? (Maybe this example isn't good enough).
Get users' current balance
User's balance is a sum of all transactions
decimal Balance() => transactionsRepository.Get().Sum(x=>x.TransactionSum);
In this case query contains logic - how I should fetch data to do something, not simple fetch\save entities like in other examples. Where I should place queries with logic? Get total balance, get last unread messages etc.
So, the first question is where I should perform fetching and saving data from database (repository)? I can't do this inside my domain model.
You do this in an Application service in a Layered architecture or a command handler in a CQRS architecture .
But it isn't efficient, more efficient is do something like this
It is more efficient but indeed not the DDD way. Aggregates should have no dependency to repository. They work only with state that is stored in memory. An application service is responsible fir loading and storing an aggregate.
Where I should put logic which works with multiple models?
In Sagas/Process managers.
Where I should place queries with logic?
It depends on what do you query.
If not using CQRS:
If you query data from an Aggregate in an Aggregate's method.
If you query a specific lists of Aggregates them you put that logic in the repository.
If using CQRS then any query is done on a read-model/projection.
Vaughn Vernon has a series of pdfs that suggests to use indirect references to link related entities together. This way, Entity A would have a list of Entity B Ids instead of references to them. This avoids performance and scalability issues and also let's you focus on consistency boundaries.
My problem comes with traversing such a type of indirect reference to be able to perform domain logic.
Say I have Group, User and Authorizations. All three are entities and also their own aggregate. They can each be modified simultaneously from one another, and therefore are not part of a consistency boundary. They relate to each other in the way that Users are in Groups and Groups are Authorized for things.
I need domain logic that retrieves me the list of things a User is Authorized to. Ideally, I feel like this logic should be on the User as User.GetAuthorizations(); But I'm willing to move this somewhere else if it feels more appropriate.
The problem is that, no matter where I put this logic I will need to:
Get the Groups of a User.
For each of the user's groups, get
the given Authorizations.
Perform the merge logic of those
Authorization.
Return the Authorizations as they fit the User.
How should I therefore implement User.GetAuthorizations(); in a DDD context which uses Repositories for data access?
By this I mean that I want to know how I should retrieve the indirect reference and traverse it inside my method.
Here is an example:
public class UserApplicationService
{
public IEnumerable<Authorization> GetUserAuthorizations(string userId)
{
User user = _userRepo.Find(userId);
IEnumerable<Group> groups = _groupRepo.FindMany(user.GroupIds);
List<Tuple<Group, List<Authorization>>> groupAuths = new List<Tuple<Group, List<Authorization>>>()
foreach(Group group in groups)
{
List<Authorization> auths = _authorizationRepo.FindMany(group.AuthorizationIds).ToList();
Tuple<Group, List<Authorization>> groupAuth = new Tuple<Group, List<Authorization>>(group, auths);
groupAuths.Add(groupAuth);
}
return user.GetAuthorizations(groupAuths);
}
}
IEnumerable<Authorization> User.GetAuthorizations(List<Tuple<Group, List<Authorization>>> groupAuths)
{
// merge logic would be here
}
In this case, I retrieve in an Application Service the groups of the User and the Authorizations of all of those groups and I pass it to the User.GetAuthorizations method. I find this quite cumbersome, and if the relationship would be nested deeper, it would become even more cumbersome.
I wonder what other approach would there be that respect the DDD approach? How is this normally done?
Time has passed since asking this, and I've furthered my understanding to the point where I feel confident in answering this myself.
I need domain logic that retrieves me the list of things a User is Authorized to. Ideally, I feel like this logic should be on the User as User.GetAuthorizations(); But I'm willing to move this somewhere else if it feels more appropriate.
This is where lies the entire problem of my question, I want retrieval logic, and I'm mistakenly thinking it must go in my domain.
In DDD, you have a domain model that tries to encapsulate the ubiquitous language of the application, but only for purposes of data change. Those entities, value objects and aggregate roots should encapsulate transactional context.
When you want to get information, you bypass all that. Query your data directly, and get exactly the info you want in the form you need.
The problem is that, no matter where I put this logic I will need to:
Get the Groups of a User.
Perform the merge logic of those Authorization.
Return the Authorizations as they fit the User.
How should I therefore implement User.GetAuthorizations(); in a DDD context which uses Repositories for data access?For each of the user's groups, get the given Authorizations.
This was missing the point. The proper way to do this is have an application service that has a method GetAuthorizations, like in my example, but this method should never use the Repository and never use the Aggregate Roots User, Group and Authorization. Instead, it should use a second abstraction, which is a query layer for retrieval of persistence data, which returns it's own type of entities that better match the query. So GetAuthorizations could retrieve objects of type AuthorizedUser. It would implement the merge logic as part of the querying process.
First of all - make sure that customer actually needs grouping of users for authorization rules definition. If not, your approach is an overkill and consider defining authorization rules in static fashion.
Another approach would be to make UserAuthorization entity that gets properly mutated for changes like 'user group has changed', 'authorization deleted' etc. Then it would be simple as asking authorization rules list for a user.
I have a somewhat complex permission system that uses six database tables in total and in order to speed it up, I would like to cache these tables in memory instead of having to hit the database every page load.
However, I'll need to update this cache when a new user is added or a permission is changed. I'm not sure how to go about having this in memory cache, and how to update it safely without causing problems if its accessed at the same time as updating
Does anyone have an example of how to do something like this or can point me in the right direction for research?
Without knowing more about the structure of the application, there are lots of possible options. One such option might be to abstract the data access behind a repository interface and handle in-memory caching within that repository. Something as simple as a private IEnumerable<T> on the repository object.
So, for example, say you have a User object which contains information about the user (name, permissions, etc.). You'd have a UserRepository with some basic fetch/save methods on it. Inside that repository, you could maintain a private static HashSet<User> which holds User objects which have already been retrieved from the database.
When you fetch a User from the repository, it first checks the HashSet for an object to return, and if it doesn't find out it gets it from the database, adds it to the HashSet, then returns it. When you save a User it updates both the HashSet and the database.
Again, without knowing the specifics of the codebase and overall design, it's hard to give a more specific answer. This should be a generic enough solution to work in any application, though.
I would cache items as you use it, which means on your data layer when you are getting you data back you check on your cache if it is available there otherwise you go to the database and cache the result after.
public AccessModel GetAccess(string accessCode)
{
if(cache.Get<string>(accessCode) != null)
return cache.Get<string>(accessCode);
return GetFromDatabase(accessCode);
}
Then I would think next on my cache invalidate strategy. You can follow two ways:
One would be set expire data to be 1 hour and then you just hit the database once in a hour.
Or invalidate the cache whenever you update the data. That is for sure the best but is a bit more complex.
Hope it helps.
Note: you can either use ASP.NET Cache or another solution like memcached depending on your infrastructure
Is it hitting the database every page load that's the problem or is it joining six tables that's the problem?
If it's just that the join is slow, why not create a database table that summarizes the data in a way that is much easier and faster to query?
This way, you just have to update your summary table each time you add a user or update a permission. If you group all of this into a single transaction, you shouldn't have issues with out-of-sync data.
You can take advantage of ASP.NET Caching and SqlCacheDependency Class. There is article on MSDN.
You can use the Cache object built in ASP.Net. Here is an article that explains how.
I can suggest cache such data in Application state object. For thread-safe usage, consider using lock operator. Your code would look something like this:
public void ClearTableCache(string tableName)
{
lock (System.Web.HttpContext.Current)
{
System.Web.HttpContext.Current.Application[tableName] = null;
}
}
public SomeDataType GetTableData(string tableName)
{
lock (System.Web.HttpContext.Current)
{
if (System.Web.HttpContext.Current.Application[tableName] == null)
{
//get data from DB then put it into application state
System.Web.HttpContext.Current.Application[tableName] = dataFromDb;
return dataFromDb;
}
return (SomeDataType)System.Web.HttpContext.Current.Application[tableName];
}
}
I currently have a repository for just about every table in the database and would like to further align myself with DDD by reducing them to aggregate roots only.
Let’s assume that I have the following tables, User and Phone. Each user might have one or more phones. Without the notion of aggregate root I might do something like this:
//assuming I have the userId in session for example and I want to update a phone number
List<Phone> phones = PhoneRepository.GetPhoneNumberByUserId(userId);
phones[0].Number = “911”;
PhoneRepository.Update(phones[0]);
The concept of aggregate roots is easier to understand on paper than in practice. I will never have phone numbers that do not belong to a User, so would it make sense to do away with the PhoneRepository and incorporate phone related methods into the UserRepository? Assuming the answer is yes, I’m going to rewrite the prior code sample.
Am I allowed to have a method on the UserRepository that returns phone numbers? Or should it always return a reference to a User, and then traverse the relationship through the User to get to the phone numbers:
List<Phone> phones = UserRepository.GetPhoneNumbers(userId);
// Or
User user = UserRepository.GetUserWithPhoneNumbers(userId); //this method will join to Phone
Regardless of which way I acquire the phones, assuming I modified one of them, how do I go about updating them? My limited understanding is that objects under the root should be updated through the root, which would steer me towards choice #1 below. Although this will work perfectly well with Entity Framework, this seems extremely un-descriptive, because reading the code I have no idea what I’m actually updating, even though Entity Framework is keeping tab on changed objects within the graph.
UserRepository.Update(user);
// Or
UserRepository.UpdatePhone(phone);
Lastly, assuming I have several lookup tables that are not really tied to anything, such as CountryCodes, ColorsCodes, SomethingElseCodes. I might use them to populate drop downs or for whatever other reason. Are these standalone repositories? Can they be combined into some sort of logical grouping/repository such as CodesRepository? Or is that against best practices.
You are allowed to have any method you want in your repository :) In both of the cases you mention, it makes sense to return the user with phone list populated. Normally user object would not be fully populated with all the sub information (say all addresses, phone numbers) and we may have different methods for getting the user object populated with different kind of information. This is referred to as lazy loading.
User GetUserDetailsWithPhones()
{
// Populate User along with Phones
}
For updating, in this case, the user is being updated, not the phone number itself. Storage model may store the phones in different table and that way you may think that just the phones are being updated but that is not the case if you think from DDD perspective. As far as readability is concerned, while the line
UserRepository.Update(user)
alone doesn't convey what is being updated, the code above it would make it clear what is being updated. Also it would most likely be part of a front end method call that may signifiy what is being updated.
For the lookup tables, and actually even otherwise, it is useful to have GenericRepository and use that. The custom repository can inherit from the GenericRepository.
public class UserRepository : GenericRepository<User>
{
IEnumerable<User> GetUserByCustomCriteria()
{
}
User GetUserDetailsWithPhones()
{
// Populate User along with Phones
}
User GetUserDetailsWithAllSubInfo()
{
// Populate User along with all sub information e.g. phones, addresses etc.
}
}
Search for Generic Repository Entity Framework and you would fine many nice implementation. Use one of those or write your own.
Your example on the Aggregate Root repository is perfectly fine i.e any entity that cannot reasonably exist without dependency on another shouldn't have its own repository (in your case Phone). Without this consideration you can quickly find yourself with an explosion of Repositories in a 1-1 mapping to db tables.
You should look at using the Unit of Work pattern for data changes rather than the repositories themselves as I think they're causing you some confusion around intent when it comes to persisting changes back to the db. In an EF solution the Unit of Work is essentially an interface wrapper around your EF Context.
With regards to your repository for lookup data we simply create a ReferenceDataRepository that becomes responsible for data that doesn't specifically belong to a domain entity (Countries, Colours etc).
If phone makes no sense w/o user, it's an entity (if You care about it's identity) or value object and should always be modified through user and retrieved/updated together.
Think about aggregate roots as context definers - they draw local contexts but are in global context (Your application) themselves.
If You follow domain driven design, repositories are supposed to be 1:1 per aggregate roots.
No excuses.
I bet these are problems You are facing:
technical difficulties - object relation impedance mismatch. You are struggling with persisting whole object graphs with ease and entity framework kind a fails to help.
domain model is data centric (as opposed to behavior centric). because of that - You lose knowledge about object hierarchy (previously mentioned contexts) and magically everything becomes an aggregate root.
I'm not sure how to fix first problem, but I've noticed that fixing second one fixes first good enough. To understand what I mean with behavior centric, give this paper a try.
P.s. Reducing repository to aggregate root makes no sense.
P.p.s. Avoid "CodeRepositories". That leads to data centric -> procedural code.
P.p.p.s Avoid unit of work pattern. Aggregate roots should define transaction boundaries.
This is an old question, but thought worth posting a simple solution.
EF Context is already giving you both Unit of Work (tracks changes) and Repositories (in-memory reference to stuff from DB). Further abstraction is not mandatory.
Remove the DBSet from your context class, as Phone is not an aggregate root.
Use the 'Phones' navigation property on User instead.
static void updateNumber(int userId, string oldNumber, string newNumber)
static void updateNumber(int userId, string oldNumber, string newNumber)
{
using (MyContext uow = new MyContext()) // Unit of Work
{
DbSet<User> repo = uow.Users; // Repository
User user = repo.Find(userId);
Phone oldPhone = user.Phones.Where(x => x.Number.Trim() == oldNumber).SingleOrDefault();
oldPhone.Number = newNumber;
uow.SaveChanges();
}
}
If a Phone entity only makes sense together with an aggregate root User, then I would also think it makes sense that the operation for adding a new Phone record is the responsibility of the User domain object throught a specific method (DDD behavior) and that could make perfectly sense for several reasons, the immidiate reason is we should check the User object exists since the Phone entity depends on it existence and perhaps keep a transaction lock on it while doing more validation checks to ensure no other process have deleted the root aggregate before we are done validating the operation. In other cases with other kinds of root aggregates you might want to aggregate or calculate some value and persist it on column properties of the root aggregate for more efficient processing by other operations later on. Note though I suggest the User domain object have a method that adds the Phone it doesn't mean it should know about the existence of the database or EF, one of the great feature of EM and Hibernate is that they can track changes made to entity classes transparently and that also means adding of new related entities by their navigation collection properties.
Also if you want to use methods that retrieve all phones regardless of the users owning them you could still though it through the User repository you only need one method returns all users as IQueryable then you can map them to get all user phones and do a refined query with that. So you don't even need a PhoneRepository in this case. Beside I would rather use a class with extensions method for IQueryable that I can use anywhere not just from a Repository class if I wanted to abstract queries behind methods.
Just one caveat for being able to delete Phone entities by only using the domain object and not a Phone repository you need to make sure the UserId is part of the Phone primary key or in other words the primary key of a Phone record is a composite key made up of UserId and some other property (I suggest an auto generated identity) in the Phone entity. This makes sense intuively as the Phone record is "owned" by the User record and it's removal from the User navigation collection would equal its complete removal from the database.