We Keep Coding

DynamoDB - How to implement Optimistic Locking using ServiceStack.Aws

Currently, I am using ServiceStack.Aws v5.9.0 to communicate with DynamoDB. I have used PutItem for both creating and updating an item without anticipating data loss in case of concurrency handling.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
}
public class CustomerDynamo
{
private readonly IPocoDynamo db;
//Constructor
public CustomerDynamo()
{
var dynamoClient = new AmazonDynamoDBClient(_region);
var entityType = typeof(Customer);
var tableName = entityType.Name;
entityType.AddAttributes(new AliasAttribute(name: tableName));
db = new PocoDynamo(dynamoClient) { ConsistentRead = true }.RegisterTable(tableType: entityType);
}
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
db.PutItem(customer);
return customer;
}
}
The above Update method is called in every service/async task that needs to update the data of the customer.
Refer to this article of AWS I decided to implement the Optimistic Locking to save my life from the issue of concurrency requests.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBContext.VersionSupport.html
Assume that the VersionNumber will be the key for Optimistic Locking. So I added the VersionNumber into the Customer model.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
[DynamoDBVersion]
public int? VersionNumber { get; set; }
}
The result is VersionNumber not updated while it should be automatically incremented. I think it is just because the PutItem will override the whole existing item. Is this correct?
I think I need to change from PutItem to UpdateItem in the Update method. The question is how can I generate the expression dynamically to be used with the UpdateItem?
Thanks in advance for any help!
Updates:
Thanks #mythz for the useful information about DynamoDBVersion attribute. Then I tried to remove the DynamoDBVersion and using the UpdateExpression of PocoDynamo as below
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
var expression = db.UpdateExpression<Customer>(customer.CustomerId).Set(() => customer);
expression.ExpressionAttributeNames = new Dictionary<string, string>()
{
{ "#Version", "VersionNumber" }
};
expression.ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{ ":incr", new AttributeValue { N = "1" } },
{ ":zero", new AttributeValue { N = "0" } }
};
expression.UpdateExpression = "SET #Version = if_not_exists(#Version, :zero) + :incr";
if (customer.VersionNumber.HasValue)
{
expression.Condition(c => c.VersionNumber == customer.VersionNumber);
}
var success = db.UpdateItem(expression);
}
But the changes are not saved except the VersionNumber

The [DynamoDBVersion] is an AWS Object Persistence Model attribute for usage with AWS's DynamoDBContext not for PocoDynamo. i.e. the only [DynamoDB*] attributes PocoDynamo utilizes are [DynamoDBHashKey] and [DynamoDBRangeKey] all other [DynamoDB*] attributes are intended for AWS's Object Persistence Model libraries.
When needed you can access AWS's IAmazonDynamoDB with:
var db = new PocoDynamo(awsDb);
var awsDb = db.DynamoDb;
Here are docs on PocoDynamo's UpdateItem APIs that may be relevant.

Copying data between models and saving children without entities duplicating themselves in Entity Framework

I am having trouble saving children entities via Entity Framework / ASP Identity. It seems to be adding duplicates of everything that is added.
I have tried using a detached graph of the DrivingLicenceModel by TeamMember.DrivingLicence = null in the TeamMemberModel and then working with a detached graph by looking if there is new or old DrivingLicenceCategories but because DrivingLicence links back to TeamMember it causes TeamMember.DrivingLicenceId to be null as it cannot link back to TeamMember.
I have tried Manually adding the EntityState to the DrivingLicence and DrivingLicenceCategories but when I do that it complains that it cannot save two entities with the same primary key.
I assume this is because they way I am copying the entities but I after a lot of looking I am drawing a blank.
If there anyway to copy from TeamMemberRequestModel to TeamMemberModel and then save without the children trying to create clone copies of themselves?
Models
public class TeamMemberModel : IdentityUser
{
public virtual DrivingLicenceModel DrivingLicence { get; set; }
public void ShallowCopy(TeamMemberRequestModel src)
{
this.DateOfBirth = src.DateOfBirth;
if (src.DrivingLicence != null)
{
if (this.DrivingLicence == null)
{
this.DrivingLicence = new DrivingLicenceModel(src.DrivingLicence);
}
else
{
this.DrivingLicence.ShallowCopy(src.DrivingLicence);
}
}
}
public TeamMemberModel() { }
}
public class DrivingLicenceModel
{
[Key]
public int Id { get; set; }
[ForeignKey("TeamMember")]
public string TeamMemberId { get; set; }
[JsonIgnore]
public TeamMemberModel TeamMember { get; set; }
public virtual List<DrivingLicenceCategoryModel> DrivingLicenceCategories { get; set; }
public DrivingLicenceModel() { }
public DrivingLicenceModel(DrivingLicenceModel src)
{
this.ShallowCopy(src);
}
public void ShallowCopy(DrivingLicenceModel src)
{
this.Id = src.Id;
this.IsFullLicence = src.IsFullLicence;
this.IssueDate = src.IssueDate;
this.ExpiryDate = src.ExpiryDate;
this.IssuingAuthority = src.IssuingAuthority;
this.LicenceNumber = src.LicenceNumber;
this.DrivingLicenceCategories = src.DrivingLicenceCategories;
this.DrivingLicencePoints = src.DrivingLicencePoints;
}
}
public class DrivingLicenceCategoryModel
{
[Key]
public int Id { get; set; }
[ForeignKey("DrivingLicence")]
public int DrivingLicenceId { get; set; }
[JsonIgnore]
public DrivingLicenceModel DrivingLicence { get; set; }
}
public class TeamMemberRequestModel
{
public string Id { get; set; }
public virtual DrivingLicenceModel DrivingLicence { get; set; }
}
Context
public class TIERDBContext : IdentityDbContext<TeamMemberModel, RoleModel, string, IdentityUserLogin, IdentityUserRole, IdentityUserClaim>
{
public TIERDBContext() : base("SARDBConnection") { }
public DbSet<DrivingLicenceModel> DrivingLicences { get; set; }
public DbSet<DrivingLicenceCategoryModel> DrivingLicenceCategories { get; set; }
}
Controller
public async Task<IHttpActionResult> Put(string id, TeamMemberRequestModel teamMember)
{
TeamMemberModel CurrentTeamMember = await this.TIERUserManager.FindByIdAsync(id);
CurrentTeamMember.ShallowCopy(teamMember);
await this.TIERUserManager.UpdateAsync(CurrentTeamMember);
}

you have to create clone property into context class
.
In the context clases you could to use clone method that retiran the entity you send by parameters this duplicarse any entity you pass. Sorry for my english
hope you help

After far to many hours working over this. I have come to an answer. The best way to deal with this is to simply deal with it is to add or attach all entities down the tree.
The controller now attaches all children unless they have an ID of 0, therefore new and uses add instead. Then I use this very useful extension I found here http://yassershaikh.com/c-exceptby-extension-method/ to compare lists to see added and deleted entities in the list. While I don't need the added part as the entity will already be marked to an add state as I use add() it does not harm and I want to use it later with add and delete state changing.
Controller
public async Task<IHttpActionResult> Put(string id, TeamMemberRequestModel teamMember)
{
TIERDBContext IdentityContext = (TIERDBContext)this.TIERUserManager.UserStore().Context;
foreach (DrivingLicenceCategoryModel DrivingLicenceCategory in teamMember.DrivingLicence.DrivingLicenceCategories)
{
if (DrivingLicenceCategory.Id == 0)
{
IdentityContext.DrivingLicenceCategories.Add(DrivingLicenceCategory);
}
else
{
IdentityContext.DrivingLicenceCategories.Attach(DrivingLicenceCategory);
}
}
foreach (DrivingLicencePointModel DrivingLicencePoint in teamMember.DrivingLicence.DrivingLicencePoints)
{
if (DrivingLicencePoint.Id == 0)
{
IdentityContext.DrivingLicencePoints.Add(DrivingLicencePoint);
}
else
{
IdentityContext.DrivingLicencePoints.Attach(DrivingLicencePoint);
}
}
this.DetectAddedOrRemoveAndSetEntityState(CurrentTeamMember.DrivingLicence.DrivingLicenceCategories.AsQueryable(),teamMember.DrivingLicence.DrivingLicenceCategories, IdentityContext);
this.DetectAddedOrRemoveAndSetEntityState(CurrentTeamMember.DrivingLicence.DrivingLicencePoints.AsQueryable(),teamMember.DrivingLicence.DrivingLicencePoints, IdentityContext);
CurrentTeamMember.ShallowCopy(teamMember);
await this.TIERUserManager.UpdateAsync(CurrentTeamMember);
}
I then use a generic that uses ExceptBy to work out what is added and delete from the old team member model to the new team member model.
protected void DetectAddedOrRemoveAndSetEntityState<T>(IQueryable<T> old, List<T> current, TIERDBContext context) where T : class, IHasIntID
{
List<T> OldList = old.ToList();
List<T> Added = current.ExceptBy(OldList, x => x.Id).ToList();
List<T> Deleted = OldList.ExceptBy(current, x => x.Id).ToList();
Added.ForEach(x => context.Entry(x).State = EntityState.Added);
Deleted.ForEach(x => context.Entry(x).State = EntityState.Deleted);
}
It works but it is far from great. It takes two DB queries, getting the original and updating. I just cannot think of any better way to do this.

Creating constant objects in C# EF CodeFirst?

I have a class "Stage" that contains an Id, Name, and TimeSpan. I need to create some constant Stages that can be referenced throughout my entire application. The Stage table and Stages should be read-only once these constants are defined.
The Stage class:
public class Stage
{
[Key]
public virtual int StageId { get; set; }
public string Name { get; set; }
public TimeSpan Span { get; set; }
}
I've tried the following (defining two constants):
public class Stage
{
[Key]
public virtual int StageId { get; set; }
public string Name { get; set; }
public TimeSpan Span { get; set; }
public static class Values
{
public static readonly Stage ONE = new Stage()
{
StageId = 0,
Name = "ONE",
Span = new TimeSpan(0, 0, 0)
};
public static readonly Stage TWO = new Stage()
{
StageId = 1,
Name = "TWO",
Span = new TimeSpan(0, 0, 10)
};
}
But whenever I create a new instance of an entity that has a Stage, a new Stage is added to the db. I just need a few constant stages.
Related entity constructor that's creating new instances when it should just be a reference to an existing stage (as defined above):
public class Side
{
public Side()
{
Stage = Stage.Values.ONE;
}
public virtual Stage Stage { get; set; }
}
How can I create a handful of constant objects and reference them via Stage.Values.One syntax?

You must attach Stage.Values.ONE and Stage.Values.TWO to the context instances where you are working with entities that reference those two constant entities in order to prevent that new Stage objects are created in the database, like
context.Stages.Attach(Stage.Values.ONE)
I would prefer though to have "context-local" constant entities instead of global static objects to avoid possible problems that can occur when they are attached to two different context instances at the same time. Something like:
public class MyContext : DbContext
{
//...
public Stage StageONE
{
get
{
var stage = this.Stages.Local.SingleOrDefault(s => s.StageId == 0);
if (stage == null)
{
stage = new Stage()
{
StageId = stageId,
Name = "ONE",
Span = new TimeSpan(0, 0, 0)
};
this.Stages.Attach(stage)
}
return stage;
}
}
}
To be used like so:
using (var context = new MyContext())
{
var side = new Side { Stage = context.StageONE };
context.Sides.Add(side);
context.SaveChanges();
}
But it doesn't seem to make much sense to me to store something in the database that you never read from there and only use hard-coded in your application. You could just make Side.Stage a byte property and use 0 and 1 for it.

How to work with multiple DbSets with common base class in EF 5?

I have 2 POCO classes:
class Email: Base
{
public int SomeProperty { get; set; }
}
class Photo: Base
{
public int SomeOtherProperty { get; set; }
}
and a base class
abstract class Base
{
public DateTime DateCreated { get; set; }
}
here is my context definition:
public class EntitiesContext : DbContext
{
public DbSet<Email> Emails { get; set; }
public DbSet<Photo> Photos { get; set; }
}
of course these classes here are just for the sake of example, things are quite more complicated.
base class is only intended to have common properties for each table - date modified, state, etc. I believe I use Table-Per-Type approach.
PROBLEM: I have some common business logic which I need to run against each table (for example, count non-processed items of each type). I need a way to iterate through a set of tables with common base class. I was hoping to do something like this:
private void GoThroughAllTables(Action<DbSet<Base>> fnProcess, bool needSave)
{
using (var db = new EntitiesContext())
{
fnProcess(db.Emails);
fnProcess(db.Photos);
if (needSave == true)
{
db.SaveChanges();
}
}
}
public IEnumerable<QueueStatus> GetQueueStatus()
{
var res = new List<QueueStatus>();
GoThroughAllTables((set) =>
{
res.Add(new QueueStatus
{
Count = set.Cast<Base>().Count(x => x.DateCreated > someDate),
});
}, false);
return res;
}
public void DeleteFailedItems()
{
GoThroughAllTables((set) =>
{
set.Cast<Base>().Remove(x => x.DateCreated > someDate);
}, true);
return res;
}
this will not compile:
fnProcess(db.Emails);
Argument 1: cannot convert from 'System.Data.Entity.DbSet|Email|' to
'System.Data.Entity.DbSet|Base|'
passing non-typed DbSet will not work because Cast will fail.
so I'm not sure what else can I try. Any suggestions?

Include derived property into a linq to entity query

Here is a simple project based on a Poco class named Task:
class Program
{
static void Main(string[] args)
{
using (MyDbContext ctx = new MyDbContext())
{
// first query
DateTime compareDate = DateTime.Now + TimeSpan.FromDays(3);
var res = ctx.Tasks.Where(t => t.LastUpdate < compareDate).ToList();
// second query
res = ctx.Tasks.Where(t => t.ShouldUpdate).ToList();
}
}
}
public class MyDbContext : DbContext
{
public DbSet<Task> Tasks { get; set; }
}
public class Task
{
public int ID { get; set; }
public DateTime LastUpdate { get; set; }
public bool ShouldUpdate
{
get
{
return LastUpdate < DateTime.Now + TimeSpan.FromDays(3);
}
}
}
What I want to do is to query the context dbset including in the where clause the ShouldUpdate derived property.
The "first query works fine" (I can't write it in a single line but it doesn't matter).
As you know, we get a NotSupportedException on the "second query", with the following message:
The specified type member 'ShouldUpdate' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.
That's right and I can understand why it happen, but I need to encapsulate the derived information inside the Task object so I can display the property in a grid or use it in every other place, without duplicating the logic behind it.
Is there a smart technique to do this?
NB: What is the technical name of the ShouldUplate property? derived? calculated? computed?

Finally i found THE solution..
You can store the partial queries (Expressions) in static fileds and use them like this:
class Program
{
static void Main(string[] args)
{
using (MyDbContext ctx = new MyDbContext())
{
res = ctx.Tasks.Where(Task.ShouldUpdateExpression).ToList();
}
}
}
public class MyDbContext : DbContext
{
public DbSet<Task> Tasks { get; set; }
}
public class Task
{
public int ID { get; set; }
public DateTime LastUpdate { get; set; }
public bool ShouldUpdate
{
get
{
return ShouldUpdateExpression.Compile()(this);
}
}
public static Expression<Func<Task, bool>> ShouldUpdateExpression
{
get
{
return t => t.LastUpdate < EntityFunctions.AddDays(DateTime.Now, 3);
}
}
}

Repository pattern would provide a better abstraction in this case. You can centralize the logic as follows. Define a new property TaskSource in your context.
public class MyDbContext : DbContext
{
public DbSet<Task> Tasks { get; set; }
public IQueryable<Task> TaskSource
{
get
{
return Tasks.Where(t => t.LastUpdate < EntityFunctions.AddDays(DateTime.Now, 3));
}
}
}

You need to put the ShouldUpdate logic inside the linq-to-enitites query. You can use EntityFunctions.AddDays to help you out like so
res = ctx.Tasks.Where(t => t.LastUpdate < EntityFunctions.AddDays(DateTime.Now, 3)).ToList();