I have two legacy enterprise application that have a few similar features. I need to build a system that responds to data change events from those systems, processes that data and exposes the combined results through an API in multiple formats. I would like to use an Event Source/DDD style architecture but I'm not sure if it makes sense. Given the simplified model below, how could I design the system?
Note - The following has been edited to clarify the question:
Both systems have Products that contain different prices based on the date. When the price changes, each system can emit an event, PriceChanged, that contains the identifier of the legacy system, that system's primary key, a date and the new price. The product ID is unique to the system, but may not be unique between both systems so a system ID will also be included.
PriceUpdated {
int systemId,
int productId,
Date date,
Float amount
}
Within the bounded context of my new system there would be a service that receives this event and need to look up my aggregate by systemId, productId, and date and then emit a command to update the corresponding price in the aggregate. My aggregate would be defined as:
class ProductPriceAggregate
{
Guid Id,
int systemId,
int productId,
Date date,
Float amount
Apply(CreateProductPriceCommand e){
Id = e.Id;
systemId = e.systemId;
productId = e.productId;
date = e.date;
RaiseEvent(new ProductPriceCreatedEvent(this))
}
Apply(UpdateProductPriceCommand d){
amount = e.amount;
RaiseEvent(new ProductPriceUpdatedEvent(this));
}
}
If I am using NEventStore which stores streams using a GUID, then each aggreateId will be represented by a GUID. My service would need to query for the GUID using the systemId, productId and date to emit a command with the correct ID.
The service might look like this:
class PriceUpdateService : ISubscribeTo<PriceUpdated>{
Handle<PriceUpdated>(PriceUpdated e)
{
var aggregateId = RetrieveId(e.systemId, e.productId, e.date);
if (aggregateId == null)
Raise(new CreateProductPriceCommand(e))
else
Raise(new UpdateProductPriceCommand(aggregateId, e.amount);
}
RetrieveId(int systemId, int productId, DateTime date)
{
// ???
}
}
The question is what is the best way to look up the aggregate's ID? The legacy systems emitting the PriceUpdated event will have no knowledge of this new system. Could I use a read model that is updated in response to ProductPriceCreatedEvent that contains enough information to query for the ID? Would I need another aggregate who's responsibility is to index ProductPrices? As posted as an answer by VoiceOfUnreason, I could use a repeatable convention for generating the ID by systemId, productId and date. Is this the recommended option from a DDD perspective?
Do you control your own IDs?
PriceUpdated {
int systemId,
int productId,
Date date,
Float amount
}
An alternative to trying to lookup an aggregateId is to calculate what that aggregateId must be. The basic idea is that the different points that need to find an aggregate from this event share an instance of a Domain Service that encapsulates the calculation.
The signature looks like the query you wrote in your question
// Just a query, we aren't changing state anywhere.
var aggregateId = idGenerator.getId(e.systemId, e.productId, e.date);
Any given implementation takes it's own salt, the arguments you pass to it, and generates a hash that is the common id used everywhere to map this combination of arguments to an aggregate.
You can, of course, produce identifiers for different aggregates using the same event data by passing in an idGenerator with a different salt.
For the particular case where your IDs are UUID, you can use a Name-Based UUID.
Related
So I am making an inventory system as a school project and I want to generate a Unique IDs for the items so that it will be the identifier for that certain Item.
Is it possible to generate an ID which is based on certain fields such as the item name, the price, the expiration.
Is there some existing library I can use for this?
EDIT: It is okay for my system to have duplicate ids because it will mean that the item already exists in the system and does not need to be added again.
Is it possible to generate an ID which is based on certain fields such as the item name, the price, the expiration.
Yes: a hash.
For example:
// It's very important to use `InvariantCulture ` and "o" to ensure consistent formatting on all computers
String infoText = item.Name + item.Price.ToString( "C2", CultureInfo.InvariantCulture ) + item.Expiration.ToString("o");
Byte[] infoBytes = Encoding.UTF8.GetBytes( infoText );
using( SHA256 sha = new SHA256Cng() )
{
Byte[] hash = sha.ComputeHash( infoBytes );
String hashText = Convert.ToBase64String( hash );
Console.WriteLine( "{0} = {1}", infoText, hashText )
}
A hash (also known as a "digest") will always be the same for the same input, and will be different for different input.
So if you have an SKU containing $10 worth of apples that expires on 2019-10-09, and you feed that in to the code above, then it will generate a unique code you can use without needing to store the mapping between that SKU and that unique code (also known as "content-based addressing").
...so if you come across another $10 worth of apples that also expire on 2019-10-09, then it will have the same unique code, even though it's a different object, and you didn't need to memorize that unique code you generated earlier.
But if you come across $10 worth of pears that expires on 2019-10-09, or $20 worth of apples that also expires on the same day, the'll have a different code.
You could create the ID by concatenating all of the information together as a big string or even hashing that string.
However this can still give you duplicates if all of that information is that same several times. There doesn't seem to be any need on using the existing information to generate the ID. If you are basing it on existing data then there will always be a chance of duplicates. Unless there is some sort of constraint on one of the fields requiring that to be unique. But if that's unique then you could just use that as an ID in the first place.
If you are storing this data in a database I would suggest creating an ID field and make that the primary key and give it an identity. This will automatically increment the ID so it will always be unique.
You could also just generate a GUID by using Guid.NewGuid() and use that as the ID.
Is it possible to generate an ID which is based on certain fields such as the item name, the price, the expiration.
Joining these three strings or even hashing them still have the chance of being duplicate.
Simple question needs simple solution, why not using a running number? Since you have an inventory system, for the first item in your inventory system that should be ID#1 (or you can start with any numbers you like).
But if you insist to use certain fields information i would suggest that adding all below information together:
combination of fields information
timestamp
user id(person who perform the insert)
maybe your favorite colors
hashing all information above
I've a situation where I want to store count of messages of a particular user in a chat in a particular day.
To make the Id unique, I thought I should combine these ids together and it became ~20 characters length consisted of ChatId + DDMMYY + UserId
public class UserContributions
{
[BsonId]
public string ChatIdDateUserId { get; set; }
public int Count { get; set; }
// the rest
}
But I guess an ID with this length is not good regarding performance. Is that How I should make a complex ID?
Thanks.
The length should not be too much of a problem. Furthermore, you can have a compound _id
{
_id:{
ChatID: "someId",
Date: ISODate("2017-10-30T00:00:00.000Z"),
UserID: "someUID"
}
}
Some notes first: Do NOT use strings to denote a date. First of all, ISODates are stored as 64bit unsigned integer. While with a date stored as 6 characters, you save some space, but you loose all capabilities for date operations in aggregations as well as normal date comparisons. Bad idea.
Second, your model is prone to collision. The same user at the same date could only post one message to a specific chat. The second message of the day would have exactly the same values and hence would violate the constraint of uniqueness. So you actually have to use the full ISODate, down to the millisecond.
And still then, there is a small chance of collision (say you have a date generated on two application servers which are slightly of time wise). There is a reason why there is an additional counter in ObjectId.
Here is how I would model it
{
_id: new ObjectId(),
ChatId: "someChat",
UserId: "someUser"
}
Reason: the ObjectId contains a timestamp by which you can query (I do not know C# well enough to give an example, hence I will make this a wiki answer to give others the opportunity to do so), eliminates unneeded complexity and with indices on both ChatId and UserId it is fast enough.
I have data in IBM DB2 and what I am trying to do is to get that data using EntityFramework.
I have one column that has TIMESTAMP values in this format: dd.MM.yyyy hh:mm:ss.ffffff and it is primary key in that table.
When I am getting data from DB2 database time part of timestamp is lost.
On the other side I have entity that has string property for that Id:
public string Id { get; set; }
This is a method that will return specific merchant. It is expected only one, because timestamps aka Ids are unique. But, when data gets pulled and when time part is lost I get 9 merchants with same Id and of course exception).
MerchantEntity IMerchantRepository.GetMerchantByRegistrationNumber(string companyRegistrationNumber)
{
var db = merchantDataContextProvider.GetMerchantContext();
var merchant = db.Merchants.Include(x => x.MerchantProperty).
SingleOrDefault(x => x.MerchantProperty.CompanyRegistrationNumber == companyRegistrationNumber);
return merchant;
}
I have tried to use DateTime type for Id, the only difference was that it was cutting last 3 numbers, instead whole time part.
Did anyone had same or similar problem? Who to get accurate data when using DB2?
Timestamps as primary key.. not a great idea. If you do however want to use time as your basis for creating an ID of the merchant, you could do this:
var newId = string.format("{0:D19}", DateTime.Now.Ticks)
which will give you a 19-digit long number based on the current time. The problem of course is that if the clock on the PC resets, or you run the program on a different computer, different timezones etc, this strategy will fail.
If you need something that uniquely identifies a row in a table, use GUID's. They're very unique, and the chance of two different processes making the same one is very close to zero
Can we implement a custom GUID in C# with string values and again fetch the custom string data from GUID parsing? It may not sound practical, but just to know the process and its de-merits.
Suppose we have a database table CUSTOMER with Cust_Data column as uniqueidentifier datatype. From C#, we created method to generate GUID and put in DB table. for example
public Guid GetCustomGuid(Guid guidValue , string customerData)
{
// Method implementation
// Create New GUID as ==> guidValue + customerData;
return //Guid created
}
Once the GUID is generated , we again parse the GUID from DB and fetch the customer data.
public string GetCustomData(Guid GuidFromDB)
{
// Method implementation for parsing the guid.
// fetch the custom data from GuidFromDB.
// Every GUID received is always unique with respect to customer data.
return //CustomerData;
}
I didn't know whether, I put the question exact context, please edit it, if required.
A column of type uniqueidentifier can store 16 bytes. This is in general not enough to leave room for extra data and is certainly not in anyway suitable to ensure that the column is still unique once you start dropping bytes from the Guid to make room. There's no GU in guid anymore when you do that, it is no longer guaranteed to be globally unique. The random index collisions you get are of course very unpleasant.
There's no point, just add an extra column to the table. Maybe you want a clustered index, it isn't clear from the question.
One of this biggest reasons NOT to do this is because if the GUID is relative to some customer data, and you change that data, you change the Guid. This would break referential integrity in the database. Row Identifers are not support to, in and of themselves, contain data about row, only how the row is related to other rows.
I have an object whose fields I would like to store in a database. I will be using SQL Server Compact Edition (with Visual C# Express 2010). For the record, I'm fairly new to programming with databases (and databases in general). This program will be used every day to read emails, process the orders inside them, store them, and access them when necessary to help with completing the orders. The list of orders is going to become much to large to store in a List, write to a file, create the List from a file, etc. The problem is that each order contains a list of the items purchased. I am aware that I can serialize the list in binary or XML, and use that data as the field. However, this prevents me from searching/selecting based on that list. For instance, if I wanted to find an order based on what items are in it, or see how many times a particular item has been purchased. Since the list will be of arbitrary size, I can't just create a bunch of fields and fill only the ones I need (which in my opinion, is a bad idea anyway).
While writing this I realized a mistake. If I serialize the list again, I could compare the serialized data to find the same list again (though, this assumes that the same data is serialized the same way each time). However, I'm still prevented from finding any particular item.
So, is there any way to store the list of items, in a fixed number of fields (preferably 1) and still be able to search its contents with a query (I will most likely be using LINQ)?
Edit:
To address what I've gotten so far: first, thanks! I'm starting to piece together what I have to do, but I'm not quite there. The consensus seems to be to have a table for each set of items. Does that mean I'd be creating thousands of tables each month?
After re-reading my question I realize I have to be more clear. As the order comes in, I parse the data and store it in an Order object, which consists of the customer's information, and the list of items. Here is a simplified version of what I'm trying to store:
class Order{
private DateTime date;
private String orderNumber;
private String buyerName;
private List<Item> items;
private String address;
}
class Item{
private String itemCode;
private String description;
private int quantity;
}
So would I have to create a new table for each List I create, or am I missing something?
Update:
This may be a helpful reference for One-to-many relationships if you are new to the subject (especially check out the second and most upvoted answer): Create a one to many relationship using SQL Server
You'll want to create a new table in the database for your line items then create a foreign key relationship to the order table. It will be a one to many relationship. Some thing like the following - obviously you'll need to create a FK relationship, but you get the gist.
Existing
CREATE TABLE Order (
OrderID INT,
PONumber VARCHAR,
ItemsList VARCHAR
)
New
CREATE TABLE Order (
OrderID INT,
PONumber VARCHAR
)
CREATE TABLE LineItem(
LineItemID INT,
Description VARCHAR,
Quantity INT,
SequenceNumber INT,
OrderID INT -- <--- Important one here
)
Note that if you want to create just a simple lookup, the relationship would go the other way.
Instead of storing the list as a field, you can create a separate table that hold it's items. And records in this ListTable will have a field pointing to record ID in your original table. That way you can write various queries afterwards.
This can be solved in several ways, but if you want to be able to query the data directly, you should redesign your database with a seperate table where you store your list data with a reference ID. After all, databases are all about data lists.