We Keep Coding

DynamoDB - How to implement Optimistic Locking using ServiceStack.Aws

Currently, I am using ServiceStack.Aws v5.9.0 to communicate with DynamoDB. I have used PutItem for both creating and updating an item without anticipating data loss in case of concurrency handling.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
}
public class CustomerDynamo
{
private readonly IPocoDynamo db;
//Constructor
public CustomerDynamo()
{
var dynamoClient = new AmazonDynamoDBClient(_region);
var entityType = typeof(Customer);
var tableName = entityType.Name;
entityType.AddAttributes(new AliasAttribute(name: tableName));
db = new PocoDynamo(dynamoClient) { ConsistentRead = true }.RegisterTable(tableType: entityType);
}
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
db.PutItem(customer);
return customer;
}
}
The above Update method is called in every service/async task that needs to update the data of the customer.
Refer to this article of AWS I decided to implement the Optimistic Locking to save my life from the issue of concurrency requests.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBContext.VersionSupport.html
Assume that the VersionNumber will be the key for Optimistic Locking. So I added the VersionNumber into the Customer model.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
[DynamoDBVersion]
public int? VersionNumber { get; set; }
}
The result is VersionNumber not updated while it should be automatically incremented. I think it is just because the PutItem will override the whole existing item. Is this correct?
I think I need to change from PutItem to UpdateItem in the Update method. The question is how can I generate the expression dynamically to be used with the UpdateItem?
Thanks in advance for any help!
Updates:
Thanks #mythz for the useful information about DynamoDBVersion attribute. Then I tried to remove the DynamoDBVersion and using the UpdateExpression of PocoDynamo as below
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
var expression = db.UpdateExpression<Customer>(customer.CustomerId).Set(() => customer);
expression.ExpressionAttributeNames = new Dictionary<string, string>()
{
{ "#Version", "VersionNumber" }
};
expression.ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{ ":incr", new AttributeValue { N = "1" } },
{ ":zero", new AttributeValue { N = "0" } }
};
expression.UpdateExpression = "SET #Version = if_not_exists(#Version, :zero) + :incr";
if (customer.VersionNumber.HasValue)
{
expression.Condition(c => c.VersionNumber == customer.VersionNumber);
}
var success = db.UpdateItem(expression);
}
But the changes are not saved except the VersionNumber

The [DynamoDBVersion] is an AWS Object Persistence Model attribute for usage with AWS's DynamoDBContext not for PocoDynamo. i.e. the only [DynamoDB*] attributes PocoDynamo utilizes are [DynamoDBHashKey] and [DynamoDBRangeKey] all other [DynamoDB*] attributes are intended for AWS's Object Persistence Model libraries.
When needed you can access AWS's IAmazonDynamoDB with:
var db = new PocoDynamo(awsDb);
var awsDb = db.DynamoDb;
Here are docs on PocoDynamo's UpdateItem APIs that may be relevant.

Entity Framework adds duplicate foreign keys

I have two entities
public class CandlestickData
{
[Key]
public int Id { get; set; }
public virtual Symbol Symbol { get; set; }
[Column(TypeName = "datetime2")]
public DateTime Time { get; set; }
public decimal Open { get; set; }
public decimal High { get; set; }
public decimal Low { get; set; }
public decimal Close { get; set; }
}
and
public class Symbol
{
[Key]
public int Id { get; set; }
public string Name { get; set; }
}
Because I add lots of 'CandlestickData' in the db per minute and the Symbols are usually the same and the don't get changed, I want to avoid unuseful calls to the DB so I made an extension method that keeps the symbols that were already retrieved from the db in a List so I re-use them.
public static class Extension
{
static List<Symbol> ExistingSymbols = new List<Symbol>();
public static Symbol GetSymbolIfExistsOrCreateItInTheDb(this string name, Repository repository)
{
if (ExistingSymbols.Any(x => x.Name == name))
{
return ExistingSymbols.First(x => x.Name == name);
}
if (repository.SymbolExists(name))
{
var symbol = repository.GetSymbol(name);
ExistingSymbols.Add(symbol);
return symbol;
}
else
{
Symbol symbol = new Symbol { Name = name };
repository.AddSymbol(symbol);
symbol = repository.GetSymbol(name);
ExistingSymbols.Add(symbol);
return symbol;
}
}
}
This is code code that adds multiple 'CandlestickData' in the db
using (var repository = new Repository())
{
var candlesticks = new List<CandlestickData>();
foreach (var symbol in AllSymbolsTradeData[GetNotUsedIndex()])
{
candlesticks.Add(new CandlestickData
{
Close = symbol.Value.Close,
Symbol = symbol.Key.GetSymbolIfExistsOrCreateItInTheDb(repository),
High = symbol.Value.High,
Low = symbol.Value.Low,
Open = symbol.Value.Open,
Time = symbol.Value.Time
});
}
repository.AddCandlesticksData(candlesticks);
repository.CommitChanges();
}
Also if you want to see how the Repository class looks:
public class Repository : IDisposable
{
private Db context;
public Repository()
{
context = new Db();
}
public bool SymbolExists(string name)
{
return context.Symbols.Where(x => x.Name == name).Any();
}
public Symbol GetSymbol(string name)
{
return context.Symbols.First(x => x.Name == name);
}
public void AddSymbol(Symbol symbol)
{
context.Symbols.Add(symbol);
context.SaveChanges();
}
public void AddCandlestickData(CandlestickData candlestickData)
{
context.Candlesticks.Add(candlestickData);
}
public void AddCandlesticksData(List<CandlestickData> candlesticks)
{
context.Candlesticks.AddRange(candlesticks);
}
public void CommitChanges()
{
context.SaveChanges();
}
public void Dispose()
{
context.Dispose();
}
}
Now my problem is that every time my Extension method gets the symbol from the memory list(with ID and Name) after the CommitChanges method is called the reference Symbol that was used in from my list gets another ID and in the database a new Symbol is inserted with the same name but with a new ID.
My code worked correctly(without creating duplicates in the Db) when I was getting the symbols from the db every time and I can't find what I'm doing wrong since the Symbols stored in the list are the ones from the db with the correct ID...

The problem is ExistingSymbols was loaded in a different DbContext than is the context that saves the candlestick.
That is because the ExistingSymbols is static it is used for multiple operations saving candlesticks.
You might solve it by changing thr model of CandlestickData to add a foreign key:
public class CandlestickData
{
public int SymbolId { get; set; }
[ForeignKey(nameof(SymbolId))]
public virtual Symbol Symbol { get; set; }
and then use just the foreign key when creating a new object using the symbol:
candlesticks.Add(new CandlestickData
{
Close = symbol.Value.Close,
SymbolId = symbol.Key.GetSymbolIfExistsOrCreateItInTheDb(repository).Id,
High = symbol.Value.High,
Low = symbol.Value.Low,
Open = symbol.Value.Open,
Time = symbol.Value.Time
});

Copying data between models and saving children without entities duplicating themselves in Entity Framework

I am having trouble saving children entities via Entity Framework / ASP Identity. It seems to be adding duplicates of everything that is added.
I have tried using a detached graph of the DrivingLicenceModel by TeamMember.DrivingLicence = null in the TeamMemberModel and then working with a detached graph by looking if there is new or old DrivingLicenceCategories but because DrivingLicence links back to TeamMember it causes TeamMember.DrivingLicenceId to be null as it cannot link back to TeamMember.
I have tried Manually adding the EntityState to the DrivingLicence and DrivingLicenceCategories but when I do that it complains that it cannot save two entities with the same primary key.
I assume this is because they way I am copying the entities but I after a lot of looking I am drawing a blank.
If there anyway to copy from TeamMemberRequestModel to TeamMemberModel and then save without the children trying to create clone copies of themselves?
Models
public class TeamMemberModel : IdentityUser
{
public virtual DrivingLicenceModel DrivingLicence { get; set; }
public void ShallowCopy(TeamMemberRequestModel src)
{
this.DateOfBirth = src.DateOfBirth;
if (src.DrivingLicence != null)
{
if (this.DrivingLicence == null)
{
this.DrivingLicence = new DrivingLicenceModel(src.DrivingLicence);
}
else
{
this.DrivingLicence.ShallowCopy(src.DrivingLicence);
}
}
}
public TeamMemberModel() { }
}
public class DrivingLicenceModel
{
[Key]
public int Id { get; set; }
[ForeignKey("TeamMember")]
public string TeamMemberId { get; set; }
[JsonIgnore]
public TeamMemberModel TeamMember { get; set; }
public virtual List<DrivingLicenceCategoryModel> DrivingLicenceCategories { get; set; }
public DrivingLicenceModel() { }
public DrivingLicenceModel(DrivingLicenceModel src)
{
this.ShallowCopy(src);
}
public void ShallowCopy(DrivingLicenceModel src)
{
this.Id = src.Id;
this.IsFullLicence = src.IsFullLicence;
this.IssueDate = src.IssueDate;
this.ExpiryDate = src.ExpiryDate;
this.IssuingAuthority = src.IssuingAuthority;
this.LicenceNumber = src.LicenceNumber;
this.DrivingLicenceCategories = src.DrivingLicenceCategories;
this.DrivingLicencePoints = src.DrivingLicencePoints;
}
}
public class DrivingLicenceCategoryModel
{
[Key]
public int Id { get; set; }
[ForeignKey("DrivingLicence")]
public int DrivingLicenceId { get; set; }
[JsonIgnore]
public DrivingLicenceModel DrivingLicence { get; set; }
}
public class TeamMemberRequestModel
{
public string Id { get; set; }
public virtual DrivingLicenceModel DrivingLicence { get; set; }
}
Context
public class TIERDBContext : IdentityDbContext<TeamMemberModel, RoleModel, string, IdentityUserLogin, IdentityUserRole, IdentityUserClaim>
{
public TIERDBContext() : base("SARDBConnection") { }
public DbSet<DrivingLicenceModel> DrivingLicences { get; set; }
public DbSet<DrivingLicenceCategoryModel> DrivingLicenceCategories { get; set; }
}
Controller
public async Task<IHttpActionResult> Put(string id, TeamMemberRequestModel teamMember)
{
TeamMemberModel CurrentTeamMember = await this.TIERUserManager.FindByIdAsync(id);
CurrentTeamMember.ShallowCopy(teamMember);
await this.TIERUserManager.UpdateAsync(CurrentTeamMember);
}

you have to create clone property into context class
.
In the context clases you could to use clone method that retiran the entity you send by parameters this duplicarse any entity you pass. Sorry for my english
hope you help

After far to many hours working over this. I have come to an answer. The best way to deal with this is to simply deal with it is to add or attach all entities down the tree.
The controller now attaches all children unless they have an ID of 0, therefore new and uses add instead. Then I use this very useful extension I found here http://yassershaikh.com/c-exceptby-extension-method/ to compare lists to see added and deleted entities in the list. While I don't need the added part as the entity will already be marked to an add state as I use add() it does not harm and I want to use it later with add and delete state changing.
Controller
public async Task<IHttpActionResult> Put(string id, TeamMemberRequestModel teamMember)
{
TIERDBContext IdentityContext = (TIERDBContext)this.TIERUserManager.UserStore().Context;
foreach (DrivingLicenceCategoryModel DrivingLicenceCategory in teamMember.DrivingLicence.DrivingLicenceCategories)
{
if (DrivingLicenceCategory.Id == 0)
{
IdentityContext.DrivingLicenceCategories.Add(DrivingLicenceCategory);
}
else
{
IdentityContext.DrivingLicenceCategories.Attach(DrivingLicenceCategory);
}
}
foreach (DrivingLicencePointModel DrivingLicencePoint in teamMember.DrivingLicence.DrivingLicencePoints)
{
if (DrivingLicencePoint.Id == 0)
{
IdentityContext.DrivingLicencePoints.Add(DrivingLicencePoint);
}
else
{
IdentityContext.DrivingLicencePoints.Attach(DrivingLicencePoint);
}
}
this.DetectAddedOrRemoveAndSetEntityState(CurrentTeamMember.DrivingLicence.DrivingLicenceCategories.AsQueryable(),teamMember.DrivingLicence.DrivingLicenceCategories, IdentityContext);
this.DetectAddedOrRemoveAndSetEntityState(CurrentTeamMember.DrivingLicence.DrivingLicencePoints.AsQueryable(),teamMember.DrivingLicence.DrivingLicencePoints, IdentityContext);
CurrentTeamMember.ShallowCopy(teamMember);
await this.TIERUserManager.UpdateAsync(CurrentTeamMember);
}
I then use a generic that uses ExceptBy to work out what is added and delete from the old team member model to the new team member model.
protected void DetectAddedOrRemoveAndSetEntityState<T>(IQueryable<T> old, List<T> current, TIERDBContext context) where T : class, IHasIntID
{
List<T> OldList = old.ToList();
List<T> Added = current.ExceptBy(OldList, x => x.Id).ToList();
List<T> Deleted = OldList.ExceptBy(current, x => x.Id).ToList();
Added.ForEach(x => context.Entry(x).State = EntityState.Added);
Deleted.ForEach(x => context.Entry(x).State = EntityState.Deleted);
}
It works but it is far from great. It takes two DB queries, getting the original and updating. I just cannot think of any better way to do this.

How to convert data from two tables get from Linq query to List witch can be populated in wpf form

I don't know how to convert LINQ query to List type of Owner with data from Transport table and pass it to WPF form (using MVVM)
DB structure :
Owner has many cars, so I described relation like this:
public partial class Transport
{
public Transport()
{
TransportOwners = new List<TransportOwner>();
}
[Key]
public int TransportID { get; set; }
public string PlateNo { get; set; }
public string Brand { get; set; }
public string Model { get; set; }
public virtual ICollection<TransportOwner> TransportOwners { get; set; }
}
public partial class Owner
{
[Key]
public int OwnerID { get; set; }
public int TransportID { get; set; }
[ForeignKey("TransportID")]
public virtual Transport Transport { get; set; }
[NotMapped]
public string PlateNo { get; set; }
[NotMapped]
public string Brand { get; set; }
[NotMapped]
public string Model { get; set; }
}
In ViewModel I created list type of Owner :
private List<Owner> _haveList;
public List<Owner> HaveList
{
get { return _haveList; }
set
{
if (value != _haveList)
{
_haveList = value;
RaisePropertiesChanged("HaveList");
}
}
}
Now I am trying to get the data :
using (var dbContext = new DataModelContext())
{
var query = dbContext.Owners.AsQueryable();
query = query.Where(o => o.OwnerId.Equal(OwnerParameter));
query = query.Select(t => new
{
Model = t.Transport.Model,
Brand = t.Transport.Brand,
PlateNo = t.Transport.PlateNo
}).ToList();
// Here I see data I need (list of Transport by Owner)
HaveList = query;
'System.Collections.Generic.List<<anonymous type: ... >>' to 'System.Collections.Generic.List<DataModels.Owner>'

In Linq-to-Entities you can only project to an anonymous type or a regular class. You can't project to an existing entity type
var result = (from o in query
where o.OwnerID==OwnerParameter
select new OwnerModel
{
Model=o.Transport.Model,
Brand=o.Transport.Brand
}).ToList();

1 - You should try to use a named object
HaveList= query.Select(t => new OwnerModel
{
Model = t.Transport.Model,
Brand = t.Transport.Brand,
PlateNo = t.Transport.PlateNo
}).ToList();
2 - Your query object is created as IQuerible, then you try to assign it as a List
query = query should not work I think.
Note that OwnerModel should fire INotificationEvent when one of the property is modified :)
private List<OwnerModel> _haveList;
public List<OwnerModel> HaveList
{
get { return _haveList; }
set
{
if (value != _haveList)
{
_haveList = value;
RaisePropertiesChanged("HaveList");
}
}
}

Finally I have what I need, thank You for Your help
List<Owner> list = DBContext.Owners.Where(to => to.OwnerID == ownerParameter).ToList();
HaveList = list.Select(t => new Owner()
{
Model = t.Transport.Model,
Brand = t.Transport.Brand,
PlateNo = t.Transport.PlateNo
}).ToList();

EF4.1 : How to deal with items being added to an Object's collection

I'm using EF4.1 for the first time (so be patient with me) but I just cant get to grips with how I can add new items to a sub collection of an object and then save the object.
For example, with the classes below, I can initially save the TravelTicket (containing multiple People) into my database, but as soon as I add a new person and then try to save the TravelTicket again I get:
An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.
Can anyone help?
public class TravelTicket
{
public int Id { get; set; }
public string Destination { get; set; }
public virtual List<Person> Members { get; set; }
}
public class Person
{
public int Id { get; set; }
public string Name{ get; set; }
}
EDITED: All relevant code added as requested:
Domain Models:
public class TravelTicket
{
public int Id { get; set; }
public string Destination { get; set; }
public virtual ICollection<Person> Members { get; set; }
}
public class Person
{
public int Id { get; set; }
public string Name { get; set; }
}
The DB Context:
public class TicketContext : DbContext
{
public TicketContext()
: base("TicketStore")
{ }
public DbSet<TravelTicket> TravelTickets { get; set; }
public DbSet<Person> People { get; set; }
}
The Repository (relevant methods only):
public class TicketRepository : ITicketRepository
{
TicketContext context = new TicketContext();
public void InsertOrUpdate(TravelTicket quoteContainer)
{
if (quoteContainer.Id == default(int))
{
// New entity
context.TravelTickets.Add(quoteContainer);
}
else
{
// Existing entity
context.Entry(quoteContainer).State = EntityState.Modified;
}
}
public void Save()
{
try
{
context.SaveChanges();
}
catch (DbEntityValidationException dbEx)
{
foreach (var validationErrors in dbEx.EntityValidationErrors)
{
foreach (var validationError in validationErrors.ValidationErrors)
{
Trace.TraceInformation("Property: {0} Error: {1}", validationError.PropertyName, validationError.ErrorMessage);
}
}
}
}
}
public interface ITicketRepository
{
void InsertOrUpdate(TravelTicket travelTicket);
void Save();
}
The consuming (example) MVC Controller code:
public class TicketSaleController : Controller
{
private readonly ITicketRepository ticketRepository;
public TicketSaleController()
: this(new TicketRepository())
{
}
public TicketSaleController(ITicketRepository ticketRepository)
{
this.ticketRepository = ticketRepository;
}
public ActionResult Index()
{
TravelTicket ticket = new TravelTicket();
ticket.Destination = "USA";
List<Person> travellers = new List<Person>();
travellers.Add(new Person { Name = "Tom" });
travellers.Add(new Person { Name = "Dick" });
travellers.Add(new Person { Name = "Harry" });
ticket.Members = travellers;
ticketRepository.InsertOrUpdate(ticket);
ticketRepository.Save();
Session["Ticket"] = ticket;
return RedirectToAction("Next");
}
public ActionResult Next()
{
TravelTicket ticket = (TravelTicket)Session["Ticket"];
ticket.Members.Add(new Person { Name = "Peter" });
ticket.Members.Add(new Person { Name = "Paul" });
ticketRepository.InsertOrUpdate(ticket);
ticketRepository.Save();
return View();
}
}
The call "ticketRepository.InsertOrUpdate(ticket);" on the "Next" method causes the exception:
An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.
FURTHER EDIT: If I pull the object back from the database after its been saved instead of pulling the object from the session, adding the 2 new persons works OK:
Works:
TravelTicket ticket = ticketRepository.Find(ticketId);
ticket.Members.Add(new Person { Name = "Peter" });
ticket.Members.Add(new Person { Name = "Paul" });
ticketRepository.InsertOrUpdate(ticket);
ticketRepository.Save();
Doesn't Work:
TravelTicket ticket = (TravelTicket)Session["Ticket"];
ticket.Members.Add(new Person { Name = "Peter" });
ticket.Members.Add(new Person { Name = "Paul" });
ticketRepository.InsertOrUpdate(ticket);
ticketRepository.Save();

I'd need to see the code you are using to add items and then persist them. Until that a few generic advice.
It seems like you're using a long-living context to do your stuff. It's a good practice to use short living context, like this:
Instance context
Do a single operation
Dispose the context
Rinse and repeat for every operation you have to do. While following this good practice, you could be indirectly solving your problem.
Again, for more specific help, please post the code you're using ;)

In your mapping class for person, you may need do something like this
Property(p => p.Id)
.StoreGeneratedPattern = StoreGeneratedPattern.Identity;