Currently, I am using ServiceStack.Aws v5.9.0 to communicate with DynamoDB. I have used PutItem for both creating and updating an item without anticipating data loss in case of concurrency handling.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
}
public class CustomerDynamo
{
private readonly IPocoDynamo db;
//Constructor
public CustomerDynamo()
{
var dynamoClient = new AmazonDynamoDBClient(_region);
var entityType = typeof(Customer);
var tableName = entityType.Name;
entityType.AddAttributes(new AliasAttribute(name: tableName));
db = new PocoDynamo(dynamoClient) { ConsistentRead = true }.RegisterTable(tableType: entityType);
}
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
db.PutItem(customer);
return customer;
}
}
The above Update method is called in every service/async task that needs to update the data of the customer.
Refer to this article of AWS I decided to implement the Optimistic Locking to save my life from the issue of concurrency requests.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBContext.VersionSupport.html
Assume that the VersionNumber will be the key for Optimistic Locking. So I added the VersionNumber into the Customer model.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
[DynamoDBVersion]
public int? VersionNumber { get; set; }
}
The result is VersionNumber not updated while it should be automatically incremented. I think it is just because the PutItem will override the whole existing item. Is this correct?
I think I need to change from PutItem to UpdateItem in the Update method. The question is how can I generate the expression dynamically to be used with the UpdateItem?
Thanks in advance for any help!
Updates:
Thanks #mythz for the useful information about DynamoDBVersion attribute. Then I tried to remove the DynamoDBVersion and using the UpdateExpression of PocoDynamo as below
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
var expression = db.UpdateExpression<Customer>(customer.CustomerId).Set(() => customer);
expression.ExpressionAttributeNames = new Dictionary<string, string>()
{
{ "#Version", "VersionNumber" }
};
expression.ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{ ":incr", new AttributeValue { N = "1" } },
{ ":zero", new AttributeValue { N = "0" } }
};
expression.UpdateExpression = "SET #Version = if_not_exists(#Version, :zero) + :incr";
if (customer.VersionNumber.HasValue)
{
expression.Condition(c => c.VersionNumber == customer.VersionNumber);
}
var success = db.UpdateItem(expression);
}
But the changes are not saved except the VersionNumber
The [DynamoDBVersion] is an AWS Object Persistence Model attribute for usage with AWS's DynamoDBContext not for PocoDynamo. i.e. the only [DynamoDB*] attributes PocoDynamo utilizes are [DynamoDBHashKey] and [DynamoDBRangeKey] all other [DynamoDB*] attributes are intended for AWS's Object Persistence Model libraries.
When needed you can access AWS's IAmazonDynamoDB with:
var db = new PocoDynamo(awsDb);
var awsDb = db.DynamoDb;
Here are docs on PocoDynamo's UpdateItem APIs that may be relevant.
Related
I'm trying to create several documents within a transaction, and one of the documents needs to be inserted as a BsonDocument as I have to be able to mutate a couple of values prior to insertion.
The two documents that I don't need to map .ToBsonDocument() are properly getting a new ID generated, but the main document is not.
Here is a basic class structure that I'm using.
public abstract class BaseDto
{
[JsonProperty("_id")] public string Id { get; init; }
[JsonProperty("_partitionKey")] public string PartitionKey { get; init; } = Constants.MasterDataPartitionKey;
}
public class ProductDto :BaseDto
{
public string Name { get; init; }
public string BrandId { get; init; }
public BrandDto Brand { get; init; }
public string ProducerId { get; init; }
public ProducerDto Producer { get; init; }
}
public class BrandDto :BaseDto
{
public string Name { get; init; }
}
public class ProducerDto :BaseDto
{
public string Name { get; init; }
}
And I'm configuring my mappings without the MongoDb specific attributes.
BsonClassMap.RegisterClassMap<BaseDto>(map =>
{
map.MapIdProperty(c => c.Id)
.SetElementName("_id")
.SetIdGenerator(CannectIdGenerator.Instance)
.SetSerializer(new StringSerializer(BsonType.String));
map.MapProperty(x => x.PartitionKey).SetElementName("_partitionKey");
});
BsonClassMap.RegisterClassMap<ProducerDto>(map => map.AutoMap());
BsonClassMap.RegisterClassMap<BrandDto>(map => map.AutoMap());
BsonClassMap.RegisterClassMap<ProductDto>(map => map.AutoMap());
The trouble I'm running into is that my Product isn't getting an auto-generated ID (the others are). Here's my transaction
[HttpPost]
public async Task<ActionResult<string>> Add(ProductDto product)
{
using var session = await _database.Client.StartSessionAsync();
try
{
session.StartTransaction();
// NOTICE THIS IS A `BsonDocument`
var products = _database.GetCollection<BsonDocument>(MasterCollection.Products.DisplayName);
var brands = _database.GetCollection<BrandDto>(MasterCollection.Brands.DisplayName);
var producers = _database.GetCollection<ProducerDto>(MasterCollection.Producers.DisplayName);
// THIS INSERT GETS A PROPER ID
await producers.InsertOneAsync(session, product.Producer);
// THIS INSERT GETS A PROPER ID
await brands.InsertOneAsync(session, product.Brand);
var insert = product.ToBsonDocument();
insert[nameof(ProductDto.Brand)] = insert[nameof(ProductDto.BrandId)] = product.Brand.Id;
insert[nameof(ProductDto.Producer)] = insert[nameof(ProductDto.ProducerId)] = product.Producer.Id;
// THIS INSERT DOES NOT GET AN ID, INSTEAD IT JUST STAYS `null`
await products.InsertOneAsync(session, insert);
await session.CommitTransactionAsync();
return Ok(insert["_id"]);
}
catch
{
await session.AbortTransactionAsync();
return Problem("Transaction Aborted");
}
}
What I'm trying to avoid is to have to manually initialize the ID field, I'd much prefer it to happen as a part of an insert.
I keep getting this error when I try to submit to the database:
A dependent property in a ReferentialConstraint is mapped to a store-generated column. Column: 'NPPRProvId'
Here's where it breaks:
public static bool Save(NPPR_Provider provider, State state, string filename, bool validateBeforeSave = true )
{
using (var db = new NPPRContext(state))
{
var prov = new NPPR_Provider()
{
First = provider.First,
Middle = provider.Middle,
Last = provider.Last,
DateOfBirth = provider.DateOfBirth,
DateOfDeath = provider.DateOfDeath,
Gender = provider.Gender,
SSN = provider.SSN,
DegreeCode = provider.DegreeCode,
BusinessName = provider.BusinessName,
DbaName = provider.DbaName,
Action = "A",
EffectiveDate = "20121212",
EndDate = "99991231",
NPPR_ServLocation = new NPPR_ServLocation()
{
EnrollmentType = provider.NPPR_ServLocation.EnrollmentType,
OrganizationType = provider.NPPR_ServLocation.OrganizationType,
ProviderTypeCode = provider.NPPR_ServLocation.ProviderTypeCode,
IRSTaxAssociations = provider.NPPR_ServLocation.IRSTaxAssociations,
NPIAssociations = provider.NPPR_ServLocation.NPIAssociations,
Address = provider.NPPR_ServLocation.Address
},
NPPR_Header = new NPPR_Header()
{
FileName = filename,
TransactionDate = Utilities.DateTimeToPRNDate(DateTime.Now),
FileLoadDate = DateTime.Now,
SubmitterId = "M00000503",
Purpose = "A",
Action = "A"
}
};
foreach(var npi in prov.NPPR_ServLocation.NPIAssociations)
{
npi.NPIType = prov.NPPR_ServLocation.OrganizationType == "I" ? "1" : "2";
}
prov.NPPR_ServLocation.Licenses = SegmentOrNull<NPPR_Licenses>(provider.NPPR_ServLocation, "Licenses", "LicenseNumber");
prov.NPPR_ServLocation.Certifications = SegmentOrNull<NPPR_Certifications>(provider.NPPR_ServLocation, "Certifications", "CertificationNumber");
prov.NPPR_ServLocation.Specialties = SegmentOrNull<NPPR_Specialties>(provider.NPPR_ServLocation, "Specialties", "SpecialtyCode");
prov.NPPR_ServLocation.Taxonomies = SegmentOrNull<NPPR_Taxonomies>(provider.NPPR_ServLocation, "Taxonomies", "TaxonomyCode");
prov.NPPR_ServLocation.OtherIds = SegmentOrNull<NPPR_OtherIds>(provider.NPPR_ServLocation, "OtherIds", "IdentifierTypeId");
prov.NPPR_ServLocation.GroupAssociations = SegmentOrNull<NPPR_GroupAssociations>(provider.NPPR_ServLocation, "GroupAssociations", "ProviderLocationId");
db.NPPR_Provider.Add(prov);
if (validateBeforeSave)
db.SaveChangesWithValidation();
else
db.SaveChanges();
return true;
}
}
db.SaveChanges() is where, according to the stacktrace, the exception is thrown. This function was originally written for EF Core, but due to issues with the server, I was forced to turn everything to EF6.4. Under Core, this method worked fine, but under EF6, it throws an exception. I've tried a few things I've read on other SO questions but so far no luck.
Specifically the exception throws on the NPPR_Provider primary key, NPPRProvId, which at no point in my code is ever read or written, except to be defined in the model.
In SQL, the NPPRProvId is PK, int, not null
The model involved:
public class NPPR_Provider : Provider
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int NPPRProvId { get; set; }
//[ForeignKey("NPPR_Header")]
public int? NPPRFileId { get; set; }
[ForeignKey("NPPRFileId")]
public NPPR_Header NPPR_Header { get; set; }
[ForeignKey("NPPRProvId")]
public NPPR_ServLocation NPPR_ServLocation { get; set; }
}
What am I doing wrong here and how might I fix it? I'm still fairly new to EF and this is my first major project in EF and MVC, and in the .NET framework in general.
In EF Core the NPPR_ServLocation could be an owned object but to my knowledge in EF 6 there are not owned objects; you need to define them explicitly in a separate table and give them keys.
public class NPPR_Provider : Provider
{
public int NPPR_ProviderId { get; set; }
public NPPR_Header NPPR_Header { get; set; }
public NPPR_ServLocation NPPR_ServLocation { get; set; }
… // data properties here
}
public class NPPR_Header {
public int NPPR_HeaderId {get;set;}
public int NPPR_ProviderId {get;set;}
public NPPR_Provider {get;set;}
… // data properties here
}
public class NPPR_ServLocation {
public int NPPR_ServLocationId {get;set;}
public int NPPR_ProviderId {get;set;}
public NPPR_Provider {get;set;}
… // data properties here.
}
I am using Entity Framework 5.0 and I created my database from model. The below is the screenshot of the edmx diagram.
I am working towards to a below structure of data:
On given Client ID give me list of Theader which belongs to that ClientID and its TReports so I modeled my models as below:
public class TReportHeaderModel
{
public int ID { get; set; }
public int ClientID { get; set; }
public string THeaderTitle { get; set; }
public int RowNumber { get; set; }
public IList<TReportModel> TReports { get; set; }
}
public class TReportModel
{
public int ID { get; set; }
public string TReportName { get; set; }
public string URL { get; set; }
public int RowNumber { get; set; }
}
So when I query to get Theaders and its each report for given clientID:
I am listing the headers first for given clientID:
public IList<TReportHeaderModel> GetHeadersByClient(int ClientID)
{
using (var connection = new TReportEntitiesConnection())
{
var clientHeaders= (from st in connection.THeaders
where ClientID == st.ClientID
select new TReportHeaderModel
{
ID=st.ID,
THeaderTitle=st.THeaderTitle,
RowNumber=st.RowNumber
}).ToList();
return (clientHeaders);
}
}
And then to get the list of reports for each title and this is where I am stuck--->
public IList<TReportModel> GetChildReportsByHeader(int THeaderID)
{
using (var connection = new TReportEntitiesConnection())
{
// ....
}
}
Instead of separating it by get the headers by client first and then get the report by header id, is it possible to combine it in one method? sorry for the confusing explanation but I am new to LINQ Query so please understand.
The below is the ideal structure for the UI implemetation:
Client ID =2
Header 1
TReportName
URL
Header 2
TReportName
URL
is it possible to combine it in one method?
If I understand you correctly, this is what you're looking for:
using (var connection = new TReportEntitiesConnection())
{
var clientHeaders = (
from st in connection.THeaders
where ClientID == st.ClientID
select new TReportHeaderModel
{
ID=st.ID,
THeaderTitle = st.THeaderTitle,
RowNumber = st.RowNumber,
Reports = from r in st.TReports
select new TReportModel
{
ID = r.ID,
TReportName = r.TReportName,
URL = r.URL,
RowNumber = r.RowNumber,
}
}
).ToList();
}
return clientHeaders;
Note that for this to work, TReportHeaderModel.TReports should be IEnumerable<TReportModel>.
Normally I would suggest you separate the methods for getting your data and transforming your data into DTOs like this (And usually I have the connection defined at the class level, not at the method level because I will reuse the connection many times, and I prefer keeping my data accesses as lazy as possible):
TReportEntitiesConnection conn = new TReportEntitiesConnection();
Then I will create extension methods like so:
public static class MyExtensions
{
public IQueryable<THeader> ByClientId(this IQuerable<THeader> conn, int ClientID)
{
return conn
.Include(h=>h.Reports)
.Where(h=>h.ClientID==ClientID);
}
public TReportHeaderModel ToDto(this THeader t)
{
return new TReportHeaderModel
{
ID=t.ID,
ClientID=t.ClientID,
THeaderTitle=t.THeaderTitle,
RowNumber=t.RowNumber,
Reports=t.Reports.ToDto()
};
}
public TReportModel ToDto(this TReport r)
{
return new TReportModel
{
ID=r.ID,
TReportName=r.TReportName,
URL=r.URL,
RowNumber=r.RowNumber
};
}
public IEnumerable<TReportHeaderModel> ToDto(this IEnumerable<THeader> h)
{
return h.Select(x=>x.ToDto());
}
public IEnumerable<TReportModel> ToDto(this IEnumerable<TReport> r)
{
return r.Select(x=>x.ToDto());
}
}
Then you can use it like so:
var result=conn.THeaders.ByClientId(200).ToDto();
If you prefer not having your connection at the module level, that is easy too:
using(var connection = new TReportEntitiesConnection())
{
var result=connection.THeaders.ByClientId(200).ToDto();
}
(or use AutoMapper and skip all the manual Dto conversions)
Error: Violation of PRIMARY KEY constraint 'PK_dbo.CompanyDtoes'.
Cannot insert duplicate key in object 'dbo.CompanyDtoes'. The
duplicate key value is (b20a140d-440b-4a41-b2c3-6763fa752246). The
statement has been terminated.
PersonDto
public class PersonDto : PartnerDto, IPartner
{
public string FirstName { get; set; }
public string LastName { get; set; }
public DateTime BirthDate { get; set; }
public string BirthPlace { get; set; }
public string MothersName { get; set; }
public string TaxId { get; set; }
public List<CompanyDto> OwnedCompanies { get; set; }
/// <summary>
/// Partner címe(i)
/// </summary>
public List<PersonAddressDto> Addresses { get; set; }
public PersonDto()
{
OwnedCompanies = new List<CompanyDto>();
Addresses = new List<PersonAddressDto>();
}
}
CompanyDto
public class CompanyDto : PartnerDto, IPartner
{
public string TaxNumber { get; set; }
public int CompanyValue { get; set; }
public List<PersonDto> Owners { get; set; }
/// <summary>
/// Partner címe(i)
/// </summary>
public List<CompanyAddressDto> Addresses { get; set; }
public CompanyDto()
{
Owners = new List<PersonDto>();
Addresses = new List<CompanyAddressDto>();
}
}
My DBContext:
public class PartnerDBContext : DbContext
{
public DbSet<PersonDto> Persons { get; set; }
public DbSet<CompanyDto> Companies { get; set; }
public DbSet<AddressDto> Addresses { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<PersonDto>()
.HasKey(k => k.PartnerId);
modelBuilder.Entity<CompanyDto>()
.HasKey(k => k.PartnerId);
modelBuilder.Entity<AddressDto>()
.HasKey(k => k.ID);
}
}
I try to insert a new person, that contains some references for already exists companies:
public bool InsertPerson(PersonDto personToInsert)
{
try
{
using (var db = new PartnerDBContext())
{
db.Persons.Add(personToInsert);
db.SaveChanges();
}
}
catch (Exception ex)
{
return false;
}
return true;
}
My problem that, I can't insert it, cos it writes Violation of primary key for CompanyDto. I know it's already exists and I don't want to add a new one, but how should I add it? I use it in a WCF service, that called from UWP. Unfortunately can't use DataAnnonations using from UWP (it's a bug), so I use ModelBuilder...
public bool InsertPerson(PersonDto personToInsert)
{
try
{
using (var db = new PartnerDBContext())
{
var companies = personToInsert.OwnedCompanies;
personToInsert.OwnedCompanies = new List<CompanyDto>();
foreach (var company in companies)
{
var companyInDb = db.Companies.Find(company.PartnerId);
personToInsert.OwnedCompanies.Add(companyInDb);
}
db.Persons.Add(personToInsert);
db.SaveChanges();
}
}
catch (Exception ex)
{
return false;
}
return true;
}
I found that solution if I get the companies from the db it saves everything as I wanted.
The problem is that when you add the person entity to the context, it also adds the related entities and marks them as Added (i.e. new). The SaveChanges() in turn tries to insert them in the database and you get the duplicate PK exception.
The solution you posted works, but involves unnecessary database trips for retrieving the related objects. Since you know they are existing, you can avoid that by simply attaching them to the context in advance, which will mark them as Unchanged (i.e. existing). Then SaveChanges will insert only the person record and the links.
using (var db = new PartnerDBContext())
{
foreach (var company in personToInsert.OwnedCompanies)
db.Companies.Attach(company);
db.Persons.Add(personToInsert);
db.SaveChanges();
}
Alternatively you can mark them as Unchanged after adding the person to the context:
using (var db = new PartnerDBContext())
{
db.Persons.Add(personToInsert);
foreach (var company in personToInsert.OwnedCompanies)
db.Entry(company).State = EntityState.Unchanged;
db.SaveChanges();
}
In my test (latest EF6.1.3, short lived new DbContext as in the posted sample) both approaches work.
I'd like to fetch data from a database by using Entity Framework into a list, and send it to client (WCF). I want to add all rows from the database to response.Offers.
This part of my code look like that:
private TestEntities bp = new TestEntities();
public TAResponse GetInformation(TARequest request)
{
List<OfferDB> result = bp.OfferDB.ToList();
TAResponse response = new TAResponse();
response.Offers = new List<DataTransferObjects.OfferDto>();
response.Offers.Add(new DataTransferObjects.OfferDto()
{
//???
});
return response; //?result but how?
}
OfferDB is a database imported from SQL Server. Maybe should I use result? But how can I return that?
TAResponse:
[DataContract]
public class TAResponse
{
[DataMember]
public List<OfferDto> Offers { get; set; }
[DataMember]
public OfferDto ThisOffer { get; set; }
}
and simplified TARequest:
[DataContract]
public class TARequest
{
[DataMember]
public int OfferID { get; set; }
[DataMember]
public string KindOfAccommodation { get; set; }
[DataMember]
public string Country { get; set; }
}
You can iterate through your db list and add to DTO list. Effective if you don't have many properties on your Offer object.
List<OfferDB> result = bp.OfferDB.ToList();
TAResponse response = new TAResponse();
response.Offers = new List<DataTransferObjects.OfferDto>();
foreach(OfferDB objCurrentOffer in result)
{
response.Offers.Add(new DataTransferObjects.OfferDto()
{
Prop1 = objCurrentOffer.Prop1,
Prop2 = objCurrentOffer.Prop2
});
}
Alternatively you can use AutoMapper to map EF object to your DTO object. https://github.com/AutoMapper/AutoMapper