We Keep Coding

Should I create separate tables for each category?

So, I have recently been tasked with transferring an old Flat File System Scheduling system into a SQL Database using C#.
The main issue I am finding is the fact that the tasks (See snippet below) use a List of strings (Created using GUIDs) has made me unsure of how to structure the database.
public class Task
{
private string TaskID;
private string TaskName;
private string TaskDescription;
private bool IsComplete;
private DateTime EstimatedStartDate;
private DateTime ActualStartDate;
private DateTime EstimatedCompletionDate;
private DateTime ActualCompletionDate;
private string TeamLead;
private List<string> TeamMembers = new List<string>();
private TaskType TaskType;
private string ParentID;
private List<string> ChildIDs = new List<string>();
}
When it comes to SQL I know that using list that can only be contained in a single Cell are generally a nono.
The real question is: Should I be having this in a list where the query will only have to query the taskID or parentID to find the requested task OR to have it split into different tables for each Category in the system (This works in 4 different Categories) and then dependant on the task's type and taskID to choose the correct table it will need to query for its children.

It helps if you define the problem domain more clearly, using a semi-formal syntax. Interpreting your code snippet, I think it boils down to the following.
A task is identified by TaskID
A task has attributes name, description etc.
A task has exactly one person, in the role "TeamLead".
A task has 0 or more persons, in the role "team member".
A task has exactly one type, selected from a collection of valid types.
A task may or may not have a relationship to another task, in the role ParentTask
A task has a relationship with 0 or more other tasks, in the relationship "childTask".
If this is true, you can see the relational model emerging.
In general, any relationship where you have "x..n" connections leads to a bridging table. In your case, that's "TeamMembers", with TaskID and PersonID as foreign keys. ChildTasks is a similar relationship.
In the case where there's "has exactly one", or "may have one", it's a foreign key. TeamLead and TaskType are examples.
There is absolutely no reason to create different tables for task type - the relational model encourages you to group similar things together, and distinguish them by data, rather than by structure.

Having multiple tables with the same structure and the same meaning is a strong antipattern: you have to modify all your queries to access the correct table (which gets especially complex if you want to summarize data from multiple categories), and you would have to modify the database structure whenever the set of possible categories changes. There is unlikely to be any measurable (let alone noticeable) performance difference.
In other words, never put data into the table name.

EF virtual collection with custom get method

Is it possible in EF to have a virtual collection that has a custom query to the data it pulls?
For instance, if I have a class person.cs and I have a property in that class: public ICollection<job> jobs, but I want that property to be defined by a custom query that I write, for instance, _context.jobs.where(j => j.backup_person_id == id).select(j); rather than defaulting to look for just person_id on the jobs table.
Is that possible? If so, how is it achieved?
[Edit]
More specifically: let's say the jobs table has the two columns manager and main_lead. I then want my person.cs class to have the properties: ICollection<job> managerJobs which contains all of the <job> entries where the manager field matches the person_code field of that person and ICollection<job> main_leadJobs which contains all of the <job> entries where the main_lead field matches the person_code field of that person,
In my head, it seems pretty simple: ICollection<job> managerJobs = select * from tbl_jobs where manager = person_code and ICollection<job> main_leadJobs = select * from tbl_jobs where main_lead = person_code
I hope that makes more sense than what I had above

No. It is not possible. However, what you want to achieve, more generally, is possible, just in a different way. See the MSDN article: https://msdn.microsoft.com/en-us/data/jj574232.aspx#explicitFilter
In a situation where you a dealing with a specific person, rather than including jobs generally, you can do:
var person = _context.People.Find(personId);
_context.Entry(person)
.Collection(b => b.jobs)
.Query()
.Where(j => j.backup_person_id == backupId)
.Load();
Then, when you accessed the person.jobs collection, it would only be those filtered items. This means you've got two queries, though, whereas if you just eargerly loaded all jobs via Include, it would be done in a single query.
If you have multiple people you're working with though, this explicit load would have to be done multiple times, meaning a query for each person (N+1). In that situation, you're better off querying each entity individually:
var people = db.People.Where(...).ToList();
var jobs = db.Jobs.Where(m => people.Select(m => m.Id).Contains(m.person_id) && m.backup_person_id == backupId).ToList();
In other words, you're querying just jobs belonging to the specific selected people and then further limiting that by your criteria, which in this case is the id of the backup person. Then, as you're iterating through the people, you can filter jobs in memory to just the jobs that belong to the current person in context.

CQRS + ES - Where to query Data needed for business logic?

I'm using CQRS + ES and I have a modeling problem that can't find a solution for.
You can skip the below and answer the generic question in the title: Where would you query data needed for business logic?
Sorry of it turned out to be a complex question, my mind is twisted at the moment!!!
Here's the problem:
I have users that are members of teams. It's a many to many relationship. Each user has an availability status per team.
Teams receive tickets, each with a certain load factor, that should be assigned to one of the team's members depending on their availability and total load.
First Issue, I need to query the list of users that are available in a team and select the one with the least load since he's the eligible for assignment.(to note that this is one of the cases, it might be a different query to run)
Second Issue, load factor of a ticket might change so i have to take that into consideration when calculating the total load per user . Noting that although ticket can belong to 1 team, the assignment should be based on the user total load and not his load per that team.
Currently a TicketReceivedEvent is received by this bounded context and i should trigger a workflow to assign that ticket to a user.
Possible Solutions:
The easiest way would be to queue events and sequentially send a command AssignTicketToUser and have a service query the read model for the user id, get the user and user.assignTicket(Ticket). Once TicketAssignedEvent is received, send the next assignment command. But it seems to be a red flag to query the read model from within the command handler! and a hassle to queue all these tickets!
Have a process manager per user with his availability/team and tickets assigned to that user. In that case we replace the query to the read side by a "process manager lookup" query and the command handler would call Ticket.AssignTo(User). The con is that i think too much business logic leaked outside the domain model specifically that we're pulling all the info/model from the User aggregate to make it available for querying
I'm inclined to go with the first solution, it seems easier to maintain, modify/extend and locate in code but maybe there's something i'm missing.

Always (well, 99.99% of cases) in the business/domain layer i.e in your "Command" part of CQRS. This means that your repositories should have methods for the specific queries and your persistence model should be 'queryable' enough for this purpose. This means you have to know more about the use cases of your Domain before deciding how to implement persistence.
Using a document db (mongodb, raven db or postgres) might make work easier. If you're stuck with a rdbms or a key value store, create querying tables i.e a read model for the write model, acting as an index :) (this assumes you're serializing objects). If you're storing things relationally with specific table schema for each entity type (huge overhead, you're complicating your life) then the information is easily queryable automatically.

Why can't you query the aggregates involved?
I took the liberty to rewrite the objective:
Assign team-ticket to user with the lowest total load.
Here we have a Ticket which should be able to calculate a standard load factor, a Team which knows its users, and a User which knows its total load and can accept new tickets:
Update: If it doesn't feel right to pass a repository to an aggregate, it can be wrapped in a service, in this case a locator. Doing it this way makes it easier to enforce that only one aggregate is updated at a time.
public void AssignTicketToUser(int teamId, int ticketId)
{
var ticket = repository.Get<Ticket>(ticketId);
var team = repository.Get<Team>(teamId);
var users = new UserLocator(repository);
var tickets = new TicketLocator(repository);
var user = team.GetUserWithLowestLoad(users, tickets);
user.AssignTicket(ticket);
repository.Save(user);
}
The idea is that the User is the only aggregate we update.
The Team will know its users:
public User GetGetUserWithLowestLoad(ILocateUsers users, ILocateTickets tickets)
{
User lowest = null;
foreach(var id in userIds)
{
var user = users.GetById(id);
if(user.IsLoadedLowerThan(lowest, tickets))
{
lowest = user;
}
}
return lowest;
}
Update: As a ticket may change load over time, the User needs to calculate its current load.
public bool IsLoadedLowerThan(User other, ILocateTickets tickets)
{
var load = CalculateLoad(tickets);
var otherLoad = other.CalculateLoad(tickets);
return load < otherLoad;
}
public int CalculateLoad(ILocateTickets tickets)
{
return assignedTicketIds
.Select(id => tickets.GetById(id))
.Sum(ticket.CalculateLoad());
}
The User then accepts the Ticket:
public void AssignTicket(Ticket ticket)
{
if(ticketIds.Contains(ticket.Id)) return;
Publish(new TicketAssignedToUser
{
UserId = id,
Ticket = new TicketLoad
{
Id = ticket.Id,
Load = ticket.CalculateLoad()
}
});
}
public void When(TicketAssignedToUser e)
{
ticketIds.Add(e.Ticket.Id);
totalLoad += e.Ticket.Load;
}
I would use a process manager / saga to update any other aggregate.

You can query the data you need in your application service. This seems to be similar to your first solution.
Usually, you keep your aggregates cross-referenced, so I am not quite sure where the first issue comes from. Each user should have a list of teams it belongs to and each group has the list of users. You can complement this data with any attributes you want, including, for example, availability. So, when you read your aggregate, you have the data directly available. Surely, you will have lots of data duplication, but this is very common.
In the event sourced model never domain repositories are able to provide any querying ability. AggregateSource by Yves Reynhout is a good reference, here is the IRepository interface there. You can easily see there is no "Query" method in this interface whatsoever.
There is also a similar question Domain queries in CQRS

Entity Framework 5 Updating a Record

I have been exploring different methods of editing/updating a record within Entity Framework 5 in an ASP.NET MVC3 environment, but so far none of them tick all of the boxes I need. I'll explain why.
I have found three methods to which I'll mention the pros and cons:
Method 1 - Load original record, update each property
var original = db.Users.Find(updatedUser.UserId);
if (original != null)
{
original.BusinessEntityId = updatedUser.BusinessEntityId;
original.Email = updatedUser.Email;
original.EmployeeId = updatedUser.EmployeeId;
original.Forename = updatedUser.Forename;
original.Surname = updatedUser.Surname;
original.Telephone = updatedUser.Telephone;
original.Title = updatedUser.Title;
original.Fax = updatedUser.Fax;
original.ASPNetUserId = updatedUser.ASPNetUserId;
db.SaveChanges();
}
Pros
Can specify which properties change
Views don't need to contain every property
Cons
2 x queries on database to load original then update it
Method 2 - Load original record, set changed values
var original = db.Users.Find(updatedUser.UserId);
if (original != null)
{
db.Entry(original).CurrentValues.SetValues(updatedUser);
db.SaveChanges();
}
Pros
Only modified properties are sent to database
Cons
Views need to contain every property
2 x queries on database to load original then update it
Method 3 - Attach updated record and set state to EntityState.Modified
db.Users.Attach(updatedUser);
db.Entry(updatedUser).State = EntityState.Modified;
db.SaveChanges();
Pros
1 x query on database to update
Cons
Can't specify which properties change
Views must contain every property
Question
My question to you guys; is there a clean way that I can achieve this set of goals?
Can specify which properties change
Views don't need to contain every property (such as password!)
1 x query on database to update
I understand this is quite a minor thing to point out but I may be missing a simple solution to this. If not method one will prevail ;-)

You are looking for:
db.Users.Attach(updatedUser);
var entry = db.Entry(updatedUser);
entry.Property(e => e.Email).IsModified = true;
// other changed properties
db.SaveChanges();

I really like the accepted answer. I believe there is yet another way to approach this as well. Let's say you have a very short list of properties that you wouldn't want to ever include in a View, so when updating the entity, those would be omitted. Let's say that those two fields are Password and SSN.
db.Users.Attach(updatedUser);
var entry = db.Entry(updatedUser);
entry.State = EntityState.Modified;
entry.Property(e => e.Password).IsModified = false;
entry.Property(e => e.SSN).IsModified = false;
db.SaveChanges();
This example allows you to essentially leave your business logic alone after adding a new field to your Users table and to your View.

foreach(PropertyInfo propertyInfo in original.GetType().GetProperties()) {
if (propertyInfo.GetValue(updatedUser, null) == null)
propertyInfo.SetValue(updatedUser, propertyInfo.GetValue(original, null), null);
}
db.Entry(original).CurrentValues.SetValues(updatedUser);
db.SaveChanges();

I have added an extra update method onto my repository base class that's similar to the update method generated by Scaffolding. Instead of setting the entire object to "modified", it sets a set of individual properties. (T is a class generic parameter.)
public void Update(T obj, params Expression<Func<T, object>>[] propertiesToUpdate)
{
Context.Set<T>().Attach(obj);
foreach (var p in propertiesToUpdate)
{
Context.Entry(obj).Property(p).IsModified = true;
}
}
And then to call, for example:
public void UpdatePasswordAndEmail(long userId, string password, string email)
{
var user = new User {UserId = userId, Password = password, Email = email};
Update(user, u => u.Password, u => u.Email);
Save();
}
I like one trip to the database. Its probably better to do this with view models, though, in order to avoid repeating sets of properties. I haven't done that yet because I don't know how to avoid bringing the validation messages on my view model validators into my domain project.

public interface IRepository
{
void Update<T>(T obj, params Expression<Func<T, object>>[] propertiesToUpdate) where T : class;
}
public class Repository : DbContext, IRepository
{
public void Update<T>(T obj, params Expression<Func<T, object>>[] propertiesToUpdate) where T : class
{
Set<T>().Attach(obj);
propertiesToUpdate.ToList().ForEach(p => Entry(obj).Property(p).IsModified = true);
SaveChanges();
}
}

Just to add to the list of options. You can also grab the object from the database, and use an auto mapping tool like Auto Mapper to update the parts of the record you want to change..

Depending on your use case, all the above solutions apply. This is how i usually do it however :
For server side code (e.g. a batch process) I usually load the entities and work with dynamic proxies. Usually in batch processes you need to load the data anyways at the time the service runs. I try to batch load the data instead of using the find method to save some time. Depending on the process I use optimistic or pessimistic concurrency control (I always use optimistic except for parallel execution scenarios where I need to lock some records with plain sql statements, this is rare though). Depending on the code and scenario the impact can be reduced to almost zero.
For client side scenarios, you have a few options
Use view models. The models should have a property UpdateStatus(unmodified-inserted-updated-deleted). It is the responsibility of the client to set the correct value to this column depending on the user actions (insert-update-delete). The server can either query the db for the original values or the client should send the original values to the server along with the changed rows. The server should attach the original values and use the UpdateStatus column for each row to decide how to handle the new values. In this scenario I always use optimistic concurrency. This will only do the insert - update - delete statements and not any selects, but it might need some clever code to walk the graph and update the entities (depends on your scenario - application). A mapper can help but does not handle the CRUD logic
Use a library like breeze.js that hides most of this complexity (as described in 1) and try to fit it to your use case.
Hope it helps

EF Core 7.0 new feature: ExecuteUpdate
Finally! After a long wait, EF Core 7.0 now has a natively supported way to run UPDATE (and also DELETE) statements while also allowing you to use arbitrary LINQ queries (.Where(u => ...)), without having to first retrieve the relevant entities from the database: The new built-in method called ExecuteUpdate — see "What's new in EF Core 7.0?".
ExecuteUpdate is precisely meant for these kinds of scenarios, it can operate on any IQueryable instance, and lets you update specific columns on any number of rows, while always issuing a single UPDATE statement behind the scenes, making it as efficient as possible.
Usage:
Imagine you want to update a specific user's email and display name:
dbContext.Users
.Where(u => u.Id == someId)
.ExecuteUpdate(b => b
.SetProperty(u => u.Email, "NewEmail#gmail.com")
.SetProperty(u => u.DisplayName, "New Display Name")
);
As you can see, ExecuteUpdate requires you to make one or more calls to the SetProperty method, to specify which property to update, and also what new value to assign to it.
EF Core will translate this into the following UPDATE statement:
UPDATE [u]
SET [u].[Email] = "NewEmail#gmail.com",
[u].[DisplayName] = "New Display Name"
FROM [Users] AS [u]
WHERE [u].[Id] = someId
Also, ExecuteDelete for deleting rows:
There's also a counterpart to ExecuteUpdate called ExecuteDelete, which, as the name implies, can be used to delete a single or multiple rows at once without first fetching them.
Usage:
// Delete users that haven't been active in 2022:
dbContext.Users
.Where(u => u.LastActiveAt.Year < 2022)
.ExecuteDelete();
Similar to ExecuteUpdate, ExecuteDelete will generate DELETE SQL statements behind the scenes — in this case, the following one:
DELETE FROM [u]
FROM [Users] AS [u]
WHERE DATEPART(year, [u].[LastActiveAt]) < 2022
Other notes:
Keep in mind that both ExecuteUpdate and ExecuteDelete are "terminating", meaning that the update/delete operation will take place as soon as you call the method. You're not supposed to call dbContext.SaveChanges() afterwards.
If you're curious about the SetProperty method, and you're confused as to why ExectueUpdate doesn't instead receive a member initialization expression (e.g. .ExecuteUpdate(new User { Email = "..." }), then refer to this comment (and the surrounding ones) on the GitHub issue for this feature.
Furthermore, if you're curious about the rationale behind the naming, and why the prefix Execute was picked (there were also other candidates), refer to this comment, and the preceding (rather long) conversation.
Both methods also have async equivalents, named ExecuteUpdateAsync, and ExecuteDeleteAsync respectively.

There are some really good answers given already, but I wanted to throw in my two cents. Here is a very simple way to convert a view object into a entity. The simple idea is that only the properties that exist in the view model get written to the entity. This is similar to #Anik Islam Abhi's answer, but has null propagation.
public static T MapVMUpdate<T>(object updatedVM, T original)
{
PropertyInfo[] originalProps = original.GetType().GetProperties();
PropertyInfo[] vmProps = updatedVM.GetType().GetProperties();
foreach (PropertyInfo prop in vmProps)
{
PropertyInfo projectProp = originalProps.FirstOrDefault(x => x.Name == prop.Name);
if (projectProp != null)
{
projectProp.SetValue(original, prop.GetValue(updatedVM));
}
}
return original;
}
Pros
Views don't need to have all the properties of the entity.
You never have to update code when you add remove a property to a view.
Completely generic
Cons
2 hits on the database, one to load the original entity, and one to save it.
To me the simplicity and low maintenance requirements of this approach outweigh the added database call.

How does ServiceStack Redis function in retrieving data

Not sure if it's the best title for the question... maybe someone could rename it for me?
My question is regarding performance of reading and combining data in c# ServiceStack wrapper for Redis and how the calls work internally.
I will explain two scenarios that will hopefully yield in a final result. One scenario has the list of category id's attached to the Transaction so that the Category can be stored independently.
Question: My end goal is to retrieve all transactions that have category 'food'.
I have tried to number other points where clarity would help my understanding. Consider there being 10,000 transactions and each transaction had on average 3 categories.
Note: There is a related question at ServiceStack.Net Redis: Storing Related Objects vs. Related Object Ids however doesn't explain the efficiency.
Example A
public class Transaction
{
public List<string> CategoryIds;
}
Example B
public class Transaction
{
public List<string> CategoryNames;
}
Code
var transactionClient = redisClient.GetTypedClient<Transaction>();
//1. is this inefficient returning all transactions?
// is there any filtering available at this part?
var allTransactions = transactionClient.GetAll();
//2. In the case of Example A where the categories are stored as id's
// how would I map the categories to a transaction?
// maybe I have a List that has a container with the Transaction associated with a
// list of Categories, however this seems inefficient as I would have to loop
// through all transactions make a call to get their Categories and then
// populate the container datatype.
//3. If we are taking Example B how can I efficiently just retrieve the transactions
// where they have a category of food.

The efficiency is less network calls vs more data. Data in Redis just gets blobbed, most of the time a single API call maps 1:1 with a redis server operation. Which means you can think about the perf implications as simply downloading a json dataset blob from a remote server's memory and deserializing it on the client - which is effectively all that happens.
In some APIs such as GetAll() it requires 2 calls, 1 to fetch all the ids in the Entity set, and the other to fetch all the records with those ids. The source code of the Redis Client is quite approachable so I recommend having a look to see exactly what's happening.
Because you've only got 3 categories, it's not that much extra data you're saving by trying to filter on the server.
So your options are basically:
Download the entire entity dataset and filter on the client
Maintain a custom index mapping from Category > Ids
More Advanced: Use a server-side LUA operation to apply server side filtering (requires Redis 2.6)