I am new to eventsourcing, so this might be a dreadfully incompetent question, so please bear with me:
We have an eventsourced, cqrs system with cassandra for persistance. We have a sequence/version number to handle conflicting modifications on an aggregate.
We need a readmodel for an administrative interface, that needs to display quite a few details from several bounded contexts and make it available for editing via a rest api.
What is the best practice handling concurrency in this readmodel. Thougts are the following:
1)
It would be nice to have a clean readmodel including all relevant data, that we can get with one request. This raises the problem: When multiple fields can be independently be edited, how do we actually create this readmodel guaranteeing that we handle the sequence for all fields? Could we add a sequence number per field and handle that somehow, but that will totally clutter our readmodel.
2)
We could have a readmodel per field making everything easy in theory, but creating a lot of requests, which would be generally stupid, but easy to manage.
3)
We could create a sequence-table seperate to the readmodel and keep track that way having both a general sequence and a per-field-sequence, and the use that to write a new readmodel when necessary.
Any thoughts.
We use strict event ordering, each event having an aggregate version inside it as metadata. We also have one, single-threaded projection. This guarantees that there will be no overrides.
The question arises about the scalability of such system and so far we have not hit the limits with this approach but the day might come. Daniel's suggestion to keep the aggregate version together with the read model makes sense but again, it will fail when scaling, assuming you will need competing consumers that will try updating your model simultaneously.
As we know, ordering is an issue with competing consumers, so I don't really have a ready-made answer for this. If the chance of concurrent update for the same field will be real (not just hypothetical), I would also consider field-level read models and UI composition.
The assumption in a CQRS ES based system is that you/the user doesn't edit read models directly. Read models are the result of all relevant events in the event stream, ie a projection. Therefore, read models change as a result of events which were caused as a result of commands.
In order to handle concurrency in that scenario, I favour using a version number stored against read models. When you form a command it should include the version of the read model table that was used to supply the context for the command.
You can then check that the events being raised have the right number. If not you can throw a concurrency exception. OR you can create a concurrency resolver service. This would check to see if the events which happened since your command was issues actually conflict or not.
If you want to know more about concurrency management in an event sourced system you can check out this post:
Handling Concurrency Conflicts in a CQRS and Event Sourced System
Hope that helps.
Related
I am trying my hands at DDD for the first time, and got stuck in the middle.
Current Design:
Our current design, not elegant but simple and works. Views bind to ViewModels, and ViewModels have reference to the ServiceClients. When user requests something on the View, it changes the state of ViewModels, and ViewModels in-turn requests ServiceClients. ServiceClients can respond to the requests either synchronously, or asynchronously.
Pros: Very simple design, and works.
Cons:
Tightly coupled to everything, everyone needs reference to everything to get things done.
Violating SOLID largely.
God objects every-where
Not testable at all!
DDD:
I decided to take refuge in DDD, to get rid of all the cons mentioned above. The idea was to carve-out Models from ViewModels, and break dependency between ViewModels and ServiceClients. So far so good.
The major issue I am facing here is the project is becoming highly event driven now. I have to fire a lot of events to get a simple operation executed.
For instance: If ViewModel has a request for ServiceClients
ViewModel will invoke an operation on the Model object,
Model fires an event, which ServiceClients handle.
ServiceClients will invoke an operation on Model to send response
Model will fire an event, which is handled by ViewModel to receive the response
The question is:
1. Is the architecture laid out correctly?
2. Is eventing necessary to get simple things done?
Thank you for your time.
Without knowing sufficient details about the actual use-case, I would venture and say you are facing this problem because you are only applying the tactical patterns of DDD without spending enough time on the strategic side.
Let me explain.
Even when there is an existing codebase like you do, you want to start thinking about high-level boundaries in the system, key actors, their behavior, and interactions between key actors. You should identify the different bounded contexts in your domain and the connections between them.
More importantly, you should try to capture the domain functionality as-it-is in the codebase. By that, I mean representing concepts in the application to mirror what a domain expert would say.
If you take your existing codebase and directly try to apply the domain modeling concepts of Aggregates/Value Objects/Domain Events etc., what often results is spaghetti code where everybody references everybody else, and dependencies are left unchecked. This interdependency leads to a situation where an update in one part of the system triggers an application-wide change and needs the extensive use of Domain Events.
Further reading:
The Two Sides of Domain-Driven Design (DDD)
DDD Strategic Patterns: How To Define Bounded Contexts
Vladimir Khorikov's Pluralsight Course (Uses the same code structure as yours)
This question stems from this other question I had asked about too many interfaces, QCRS and Mediatr library (request/response)
Mediatr: reducing number of DI'ed objects
I have created bunch of commands and queries and I have bunch of behaviors and one of them being is a Cache behaviour that for every query, cache is checked for the value before the query is actually executed against the db. So far this is working great, but the delima comes in when I have an UpdateSomethingCommand, once I update the underlying object in the db, I would like to refresh the cache with what was successfully saved to the db.
My question is specifically when to actually update the cache:
In the UpdateSomethingCommandHandler (this might be breaking the SOLID principal)
Call another command in UpdateSomethingCommandHanlder that is specifically designed to update caches (not sure this is a good design principal)
Introduce another behavior that is specifically designed for updating caches (not sure how to go about this yet)
Is there a better solution?
We had a similar need on a project that uses MediatR and ended up incorporating caching into the mediator pipeline, including cache invalidation as you describe.
The basic premise is that we have two different behaviors inserted into the pipeline, one for caching a response from a request, and one for invalidating a cached request response from a different request.
There is a little bit of interplay between the two behaviors in the fact that they need to exchange a cache key in order to invalidate the correct request.
I've recently pulled some of this work into a stand-alone library that in theory can be dropped in as-is to any project using MediatR. In your case, you may just want to look at the techniques we've used here and recreate them as needed.
Rather than repeat everything here and now, I'll point you at the project page where there is some documentation under the Getting Started link on the homepage:
https://github.com/Imprise/Imprise.MediatR.Extensions.Caching
In my opinion, the cache invalidation makes the whole process extremely simple and straightforward, but there are cases where we needed finer control over when the invalidation occurs. In these cases the other approach we have taken is to inject an ICache<TRequest, TResponse> cache into INotificationHandlers and then call _cache.Remove(key); manually as needed. Then, from any requests you know should invalidate just raise a notification that is handled by the INotificationHandler e.g. _mediator.Publish(SomethingUpdated);
My suggestion is to use a cache behavior that acts on requests that implement some sort of ICacheableRequest marker interface and invalidating the cache as a step in the corresponding Update/Delete command handlers (like you mentioned in point 1).
If you choose to create an invalidator behavior there are a few problems.
First, it's unclear that the command is invalidating the cache. Whenever I need to check what's going on when an entity is updated/deleted, I just follow the command handler, there are no side effects (harder to follow) by creating a separate cache invalidator.
Second, even if putting the invalidation code in a separate file follows better the SRP, you will have to choose where to put the cache invalidator class. Does it go next to the cached query or next to the command handler that invalidates the cache?
Third, in many scenarios you won't have enough information about the key used to cache the request in the associated command, you'll only get that and any other extra invalidation condition in the CommandHandler.
First of all, sorry could not find a good title :(
Problem
So let's say we have an object and we can apply different actions to manipulate that object and i want to persist this in the database.
As an example, we can define the following classes.
interface IAction{ object transform(object input); }
class Stretch : IAction { }
class Shrink : IAction {}
And an object can have a list of IAction. Also each action can have a type that we defined.
enum ActionType { Stretch, Shrink }
When we retrieve the actions from the database then we need to do logics to checkh what is the type of the action to know which class to create.
If i just change the order of the items in the enum, then there is nothing that tell me my database has also to change and this can introduce breaking changes.
I'm trying to find a design pattern that will avoid this situation. Is there any?
Personally, I do not understand the problem as described. Normally, I would not store/register actions or changes to an object itself and just let the objects maintain their current state.
When I want to store/register actions or create an undo/redo buffer, I would set up a logging or auditing mechanism. But I would design it completely separate from the basic storage/modeling structure, since I do not want to introduce such tight functional coupling in an application.
As with all problems, there might be many possible solutions. It is quite difficult to give an out-of-the-box solution for your problem.
You may want to look into somewhat more traditional solutions, like implementing a logging mechanism, an auditing mechanism and/or an undo/redo buffer in your application.
Choosing the best strategy depends on way more factors than described here and should be based on the overall design and architecture of your application (and the quality requirements that should be met).
By the way: In my experience, the future will certainly bring (unexpected) change requests that will break your current code logic and/or data structures. For that reason, it is important to keep everything in your application as simple and as elegant as possible while regarding all the required specifications. That will make the code more maintainable and it might also simplify future code and data migrations.
Getting started with all this MVVM stuff, I was following this post by Josh Smith that talks about an approach to validation while using MVVM. The example is simple, and I began to wonder how I would use it in my own application.
I have, in my BLL, a BookInfo class that implements IDataErrorInfo to report invalid values such as "published date can't be in the future" or "number of pages can't be negative". Then my AddBookViewModel would check the state of the newly created BookInfo, see that errors exist, and the AddBookView will display a red blob next to the appropriate TextBox. That stuff is straightforward, just like in the example in the post.
Now my BookInfo class also holds a list of author IDs. On addition of a new BookInfo to my database, I need to check if those author IDs already exist.
Should this check be done in my BookInfo class? If so, then I would have to pass in my BLL's AuthorManager object to BookInfo's constructor, since the former would contain methods such as CheckIfExists(int authorID).
Is this the recommended approach? What if there are a lot of records in the DB? Dynamically checking would affect performance?
On the other hand, it would seem a bit messy to perform some checks in the BookInfo class and some elsewhere... especially when all those checks can be categorized into the same group... ie. making sure a newly created BookInfo object is valid. Or maybe I'm wrong since I don't really have experience to make proper judgement.
Some guidance?
I wouldn't do this. I would keep the validations done 'inside' IDataErrorInfo simple and contextless. Any validations that depend on context, such as cross-entity validations and validations that are dependent on a database, do that validation when you save the changes.
Trying these more complex context based validations in IDataErrorInfo will be error prone and often impossible. It is often impossible to do this reliably without a context.
I've written a blog post about this. While it is written in the context (no pun intended) of the Validation Application Block, it talks about the general problem of context based validation. It might be helpful. Here it is.
I agree with Steven that you should perform the server-side validations when attempting to save the data.
Another reason for this is network latency. Since WPF's support for IDataErrorInfo uses input events to determine when properties are validated, and results in a blocking/synchronous call to your VM object, the use of IDataErrorInfo has a direct impact on the responsiveness of the UI. You cannot begin an async call to your database to perform the validation and then dispatch validation errors to the UI thread when your network call completes. You would have to make a blocking call to your DB to get the result, which could wreak havoc on the UI thread while it waits for the call to return.
I hope someday WPF gets Silverlight's fancy new INotifyDataErrorInfo interface, which allows for the async validation scenario I described above.
In some projects I see that a dummy record is needed to create in Db in order to keep the business logic go on without breaking the Db constraints.
So far I have seen its usage in 2 ways:
By adding a field like IsDummy
By adding a field something called ObjectType which points a type: Dummy
Ok, it helps on what needs to be achieved.
But what makes me feel alert on such solutions is sometimes you have to keep in mind that some dummy records exist in the application which needs to be handled in some processes. If not, you face some problems until you realize their existence or until someone in the team tells you "Aha! You have forgotten the dummy records. You should also do..."
So the question is:
Is it a good idea to create dummy records to keep business logic as it is without making the Db complain? If yes, what is the best practice to prevent developers from skipping their existence? If not, what do you do to prevent yourself from falling in a situation where you end up with an only option of creating a dummy record?
Thanks!
Using dummy records is inferior to getting the constraints right.
There's often a temptation to use them because using dummy records can seem like the fastest way to deliver a new feature (and maybe sometimes it is), but they are never part of a good design, because they hide differences between your domain logic and data model.
Dummy records are only required when the modeller cannot easily change the Database Definition, which means the definition and/or the data model is pretty bad. One should never end up in a situation where there has to be special code in the app layer to handle special cases in the database. That is a guaranteed maintenance nightmare.
Any good definition or model will allow changes easily, without "affecting existing code".
All business logic [that is defined in the Database] should be implemented using ANSI SQL Constraints, Checks, and Rules. (Of course Lower level structures are already constrained via Domains/Datatypes, etc., but I would not classify them as "business rules".) I ensure that I don't end up having to implement dummies, simply by doing that.
If that cannot be done, then the modeller lacks knowledge and experience. Or higher level requirements such as Normalisation, have been broken, and that presents obstacles to implementing Constraints which are dependent on them; also meaning the modeller failed.
I have never needed to break such Constraints, or add dummy records (and I have worked on an awful lot of databases). I have removed dummy records (and duplicates) when I have reworked databases created by others.
I've never run across having to do this. If you need to do this, there's something wrong with your data structure, and it's going to cause problems further down the line for reporting...
Using Dummies is dumb.
In general you should aim to get your logic right without them. I have seen them used too, but only as an emergency solution. Your description sounds way too much like making it a standard practice. That would cause more problems than it solves.
The only reason I can see for adding "dummy" records is when you have a seriously bad app and database design.
It is most definitely not common practice.
If your business logic depends on a record existing then you need to do one of two things: Either make sure that a CORRECT record is created prior to executing that logic; or, change the logic to take missing information into account.
I think any situation where something isn't very easily distinguishable as "business-logic" is a cause for trying to think of a better way.
The fact that you mention "which points a type: Dummy" leads me to believe you are using some kind of ORM for handling your data access. A very good checkpoint (though not the only) for ORM solutions like NHibernate is that your source code VERY EXPLICITLY describes your data structures driving your application. This not only allows your data access to easily be managed under source control, but it also allows for easier debugging down the line should a problem occur (and let's face it, it's not a matter of IF a problem will occur, but WHEN).
When you introduce some kind of "crutch" like a dummy record, you are ignoring the point of a database. A database is there to enforce rules against your data, in an effort to ELIMINATE the need for this kind of thing. I recommend you take a look at your application logic FIRST, before resorting to this kind of technique. Think about your fellow dev's, or a new hire. What if they need to add a feature and forget your little "dummy record" logic?
You mention yourself in your question feeling apprehension. Go with your gut. Get rid of the dummy records.
I have to go with the common feeling here and argue against dummy records.
What will happen is that a new developer will not know about them and not code to handle them, or delete a table and forget to add in a new dummy record.
I have experienced them in legacy databases and have seen both of the above mentioned happen.
Also the longer they exist the harder it is to take them out and the more code you have to write to take into account these dummy records which could probably have been removed if you just did the original design without them.
The correct solution would be to update your business logic.
To quote your expanded explanation:
Assume that you have a Package object and you have implement a business logic that a Package without any content cannot be created. YOu created some business layer rules and designed your Db with relevant constraints. But after some years a new feature is requested and to accomplish that you have to be able to create a package without a contnent. To overcome this, you decide to create a dummy content which is not visible on UI but lets you to create an empty package.
So the at one time to a package w/o content was invalid thus business layer enforced existence of content in a package object. That makes sense. Now if the real world scenario has changed such there is now a need VALID reason to create Package objects without content it is the business logic layer which needs to be changed.
Almost universally using "dummy" anything anywhere is a bad idea and usually indicates an issue in implementation. In this instance you are using dummy data to allow "compliance" with a business layer which is no longer accurately representing the real world constraints of the business.
If package without content is not valid then dummy data to allow "compliance" with business layer is a foolish hack. In essence you wrote rules to protect your own system and then how are attempting to circumvent your own protection. On the other hand if package without content is valid then business layer shouldn't be enforcing bogus constraints. In neither instance is dummy data valid.