We Keep Coding

MongoDb Driver autogenerate _id for BsonDocument

I'm trying to create several documents within a transaction, and one of the documents needs to be inserted as a BsonDocument as I have to be able to mutate a couple of values prior to insertion.
The two documents that I don't need to map .ToBsonDocument() are properly getting a new ID generated, but the main document is not.
Here is a basic class structure that I'm using.
public abstract class BaseDto
{
[JsonProperty("_id")] public string Id { get; init; }
[JsonProperty("_partitionKey")] public string PartitionKey { get; init; } = Constants.MasterDataPartitionKey;
}
public class ProductDto :BaseDto
{
public string Name { get; init; }
public string BrandId { get; init; }
public BrandDto Brand { get; init; }
public string ProducerId { get; init; }
public ProducerDto Producer { get; init; }
}
public class BrandDto :BaseDto
{
public string Name { get; init; }
}
public class ProducerDto :BaseDto
{
public string Name { get; init; }
}
And I'm configuring my mappings without the MongoDb specific attributes.
BsonClassMap.RegisterClassMap<BaseDto>(map =>
{
map.MapIdProperty(c => c.Id)
.SetElementName("_id")
.SetIdGenerator(CannectIdGenerator.Instance)
.SetSerializer(new StringSerializer(BsonType.String));
map.MapProperty(x => x.PartitionKey).SetElementName("_partitionKey");
});
BsonClassMap.RegisterClassMap<ProducerDto>(map => map.AutoMap());
BsonClassMap.RegisterClassMap<BrandDto>(map => map.AutoMap());
BsonClassMap.RegisterClassMap<ProductDto>(map => map.AutoMap());
The trouble I'm running into is that my Product isn't getting an auto-generated ID (the others are). Here's my transaction
[HttpPost]
public async Task<ActionResult<string>> Add(ProductDto product)
{
using var session = await _database.Client.StartSessionAsync();
try
{
session.StartTransaction();
// NOTICE THIS IS A `BsonDocument`
var products = _database.GetCollection<BsonDocument>(MasterCollection.Products.DisplayName);
var brands = _database.GetCollection<BrandDto>(MasterCollection.Brands.DisplayName);
var producers = _database.GetCollection<ProducerDto>(MasterCollection.Producers.DisplayName);
// THIS INSERT GETS A PROPER ID
await producers.InsertOneAsync(session, product.Producer);
// THIS INSERT GETS A PROPER ID
await brands.InsertOneAsync(session, product.Brand);
var insert = product.ToBsonDocument();
insert[nameof(ProductDto.Brand)] = insert[nameof(ProductDto.BrandId)] = product.Brand.Id;
insert[nameof(ProductDto.Producer)] = insert[nameof(ProductDto.ProducerId)] = product.Producer.Id;
// THIS INSERT DOES NOT GET AN ID, INSTEAD IT JUST STAYS `null`
await products.InsertOneAsync(session, insert);
await session.CommitTransactionAsync();
return Ok(insert["_id"]);
}
catch
{
await session.AbortTransactionAsync();
return Problem("Transaction Aborted");
}
}
What I'm trying to avoid is to have to manually initialize the ID field, I'd much prefer it to happen as a part of an insert.

DynamoDB - How to implement Optimistic Locking using ServiceStack.Aws

Currently, I am using ServiceStack.Aws v5.9.0 to communicate with DynamoDB. I have used PutItem for both creating and updating an item without anticipating data loss in case of concurrency handling.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
}
public class CustomerDynamo
{
private readonly IPocoDynamo db;
//Constructor
public CustomerDynamo()
{
var dynamoClient = new AmazonDynamoDBClient(_region);
var entityType = typeof(Customer);
var tableName = entityType.Name;
entityType.AddAttributes(new AliasAttribute(name: tableName));
db = new PocoDynamo(dynamoClient) { ConsistentRead = true }.RegisterTable(tableType: entityType);
}
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
db.PutItem(customer);
return customer;
}
}
The above Update method is called in every service/async task that needs to update the data of the customer.
Refer to this article of AWS I decided to implement the Optimistic Locking to save my life from the issue of concurrency requests.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBContext.VersionSupport.html
Assume that the VersionNumber will be the key for Optimistic Locking. So I added the VersionNumber into the Customer model.
public class Customer
{
[HashKey]
public int CustomerId { get; set; }
[AutoIncrement]
public int SubId { get; set; }
public string CustomerType { get; set; }
public string LastName { get; set; }
public string FirstName { get; set; }
...//and hundreds of fields here
[DynamoDBVersion]
public int? VersionNumber { get; set; }
}
The result is VersionNumber not updated while it should be automatically incremented. I think it is just because the PutItem will override the whole existing item. Is this correct?
I think I need to change from PutItem to UpdateItem in the Update method. The question is how can I generate the expression dynamically to be used with the UpdateItem?
Thanks in advance for any help!
Updates:
Thanks #mythz for the useful information about DynamoDBVersion attribute. Then I tried to remove the DynamoDBVersion and using the UpdateExpression of PocoDynamo as below
public Customer Update(Customer customer)
{
customer.ModifiedDate = DateTime.UtcNow;
var expression = db.UpdateExpression<Customer>(customer.CustomerId).Set(() => customer);
expression.ExpressionAttributeNames = new Dictionary<string, string>()
{
{ "#Version", "VersionNumber" }
};
expression.ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{ ":incr", new AttributeValue { N = "1" } },
{ ":zero", new AttributeValue { N = "0" } }
};
expression.UpdateExpression = "SET #Version = if_not_exists(#Version, :zero) + :incr";
if (customer.VersionNumber.HasValue)
{
expression.Condition(c => c.VersionNumber == customer.VersionNumber);
}
var success = db.UpdateItem(expression);
}
But the changes are not saved except the VersionNumber

The [DynamoDBVersion] is an AWS Object Persistence Model attribute for usage with AWS's DynamoDBContext not for PocoDynamo. i.e. the only [DynamoDB*] attributes PocoDynamo utilizes are [DynamoDBHashKey] and [DynamoDBRangeKey] all other [DynamoDB*] attributes are intended for AWS's Object Persistence Model libraries.
When needed you can access AWS's IAmazonDynamoDB with:
var db = new PocoDynamo(awsDb);
var awsDb = db.DynamoDb;
Here are docs on PocoDynamo's UpdateItem APIs that may be relevant.

Web API HTTPGet for multiple attributes?

We have a Web API written in DotNet Core 3.1.402 (I am new to DotNet Core and WebAPI).
We use SqlKata for Database processing.
We have an Account model that has AccountID, AccountName, AccountNumber, etc.
We would like to get an Account by different attributes, for ex: by AccountID, by AccountName, by AccountNumber.
How can we do that so that we don't need a separate HttpGet for each attribute (so we don't have to repeat the same code for different attributes) ?
This is our HttpGet in the AccountsController to get the account by AccountID
public class AccountsController : ControllerBase
{
private readonly IAccountRepository _accountRepository;
[HttpGet("{AccountID}")]
public Account GetAccount(int AccountID)
{
var result = _accountRepository.GetAccount(AccountID);
return result;
}
This is the code in the AccountRepository.cs
public Account GetAccount(int accountID)
{
var result = _db.Query("MyAccountTable").Where("AccountID", accountID).FirstOrDefault<Account>();
return result;
}
This is the Account class
namespace MyApi.Models
{
public class Account
{
public string AccountID { get; set; }
public string AccountName { get; set; }
public string AccountNumber { get; set; }
// other attributes
}
}
Thank you.

Doing it with GET can be a pain, there are ways to pass on the path/query arrays and complex objects but are ugly, the best you can do is to use POST instead of GET and pass an object with the filters that you want.
//In the controller...
[HttpPost]
public Account GetAccount([FromBody]Filter[] DesiredFilters)
{
var result = _accountRepository.GetAccount(DesiredFilters);
return result;
}
//Somewhere else, in a shared model...
public class Filter
{
public string PropertyName { get; set; }
public string Value { get; set; }
}
//In the repository...
public Account GetAccount(Filter[] Filters)
{
var query = _db.Query("MyAccountTable");
foreach(var filter in Filters)
query = query.Where(filter.PropertyName, filter.Value);
return query.FirstOrDefault<Account>();
}
Now you can send a JSON array on the request body with any filters that you want, per example:
[
{ "PropertyName": "AccountID", "Value": "3" },
{ "PropertyName": "AccountName", "Value": "Whatever" }
]

Split tags from input field ASP.NET

I have a input field where a user can insert a comma separated list, then on the save method, it tries to split the string and save into tags table.
When I save the post, I get this in my tags table: (GalleryId is the post its related to)
Each separated value should go into its own row, but instead, I get that.
This is what I do on the Save method:
[HttpPost]
public async Task<IActionResult> Save(Gallery gallery) {
var tags = gallery.Tags.ToString();
gallery.Tags = ParseTags(tags);
_context.Galleries.Add(gallery);
await _context.SaveChangesAsync();
return RedirectToAction("UploadForm", "Gallery");
}
public List<Tag> ParseTags(string tags)
{
return tags.Split(",").Select(tag => new Tag
{
Name = tag
}).ToList();
}
And this is my Gallery Model:
namespace SimpleImageGallery.Models
{
public class Gallery
{
public int Id { get; set; }
public string Title { get; set; }
public virtual IEnumerable<Tag> Tags { get; set; }
}
Tag class:
public class Tag
{
public int Id { get; set; }
public string Name { get; set; }
}
What can I do to get them to split?

I got it working, I just had to add the string tags into my parameter like this:
public async Task<IActionResult> Save(Gallery gallery, string tags) {
if (tags != null)
{
gallery.Tags = ParseTags(tags);
}
_context.Galleries.Add(gallery);
await _context.SaveChangesAsync();
return RedirectToAction("UploadForm", "Gallery");
}
Also, check to see if any tags passed in before handling it.

WebApi field filtering without using EF

I am building WebApi2 project to expose some RESTful service. Let's say I have following model objects.
public class Person
{
public string Name { get; set; }
public DateTime? Birthdate { get; set; }
public int Status { get; set; }
public List<Account> Accounts { get; set; }
}
public class Account
{
public decimal Amount { get; set; }
public string Code { get; set; }
public DateTime Expiry { get; set; }
}
In my service I have to go to 2 different systems to retrieve data for Person and the account info of the Person. Obviously the service implementation looks like
[HttpGet]
[Route("Person/{id:int}")]
public IHttpActionResult Get(string id)
{
var person = new Person();
person = GetPersonFromSystemA(id);
if (person.Status == 2)
{
person.Accounts = GetPersonAccountsFromSystemB(id);
}
return this.Ok(person);
}
I cannot use EF at all in this case, so OData is very tricky.
I have some requirement that I need to provide the filtering capability to the service client. The client can decide which fields of the objects to return, it also means that if the client does not like to include Accounts info of the person I should skip the second call to system B to avoid entire child object.
I did some quick search but I could not find some similar solution yet. I know I can implement my own filtering syntax, and have all custom codes to use the filtering (by having lots of if/else).
I am looking for some ideas of more elegant solution.

Entity Framework is not required for building an OData Service. If you do not use OData, you will probably have to implement your own IQueryable which is what OData does out of the box.
Some sample code.
Model classes with some added properties
public class Person
{
[Key]
public String Id { get; set; }
[Required]
public string Name { get; set; }
public DateTime? Birthdate { get; set; }
public int Status { get; set; }
public List<Account> Accounts { get; set; }
}
public class Account
{
[Key]
public String Id { get; set; }
[Required]
public decimal Amount { get; set; }
public string Code { get; set; }
public DateTime Expiry { get; set; }
}
WebApiConfig.cs
public static class WebApiConfig
{
public static void Register(HttpConfiguration config)
{
config.MapODataServiceRoute("odata", null, GetEdmModel(), new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer));
config.EnsureInitialized();
}
private static IEdmModel GetEdmModel()
{
ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
builder.Namespace = "YourNamespace";
builder.ContainerName = "DefaultContainer";
builder.EntitySet<Person>("People");
builder.EntitySet<Account>("Accounts");
var edmModel = builder.GetEdmModel();
return edmModel;
}
}
Controller method
[EnableQuery]
public class PeopleController : ODataController
{
public IHttpActionResult Get()
{
return Ok(SomeDataSource.Instance.People.AsQueryable());
}
}
You will need to include the Microsoft.AspNet.OData Nuget package.
Refer to the following for more guidance. It uses an in memory data source, but the concept is the same regardless.
http://www.odata.org/blog/how-to-use-web-api-odata-to-build-an-odata-v4-service-without-entity-framework/

When building a web api you would often want to filter your response and get only certain fields. You could do it in many ways, one of which as suggested above. Another way, you could approach it is using data shaping in your web api.
If you had a controller action as such:
public IHttpActionResult Get(string fields="all")
{
try
{
var results = _tripRepository.Get();
if (results == null)
return NotFound();
// Getting the fields is an expensive operation, so the default is all,
// in which case we will just return the results
if (!string.Equals(fields, "all", StringComparison.OrdinalIgnoreCase))
{
var shapedResults = results.Select(x => GetShapedObject(x, fields));
return Ok(shapedResults);
}
return Ok(results);
}
catch (Exception)
{
return InternalServerError();
}
}
And then your GetShapedData method can do the filtering as such:
public object GetShapedObject<TParameter>(TParameter entity, string fields)
{
if (string.IsNullOrEmpty(fields))
return entity;
Regex regex = new Regex(#"[^,()]+(\([^()]*\))?");
var requestedFields = regex.Matches(fields).Cast<Match>().Select(m => m.Value).Distinct();
ExpandoObject expando = new ExpandoObject();
foreach (var field in requestedFields)
{
if (field.Contains("("))
{
var navField = field.Substring(0, field.IndexOf('('));
IList navFieldValue = entity.GetType()
?.GetProperty(navField, BindingFlags.IgnoreCase | BindingFlags.Instance | BindingFlags.Public)
?.GetValue(entity, null) as IList;
var regexMatch = Regex.Matches(field, #"\((.+?)\)");
if (regexMatch?.Count > 0)
{
var propertiesString = regexMatch[0].Value?.Replace("(", string.Empty).Replace(")", string.Empty);
if (!string.IsNullOrEmpty(propertiesString))
{
string[] navigationObjectProperties = propertiesString.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
List<object> list = new List<object>();
foreach (var item in navFieldValue)
{
list.Add(GetShapedObject(item, navigationObjectProperties));
}
((IDictionary<string, object>)expando).Add(navField, list);
}
}
}
else
{
var value = entity.GetType()
?.GetProperty(field, BindingFlags.IgnoreCase | BindingFlags.Instance | BindingFlags.Public)
?.GetValue(entity, null);
((IDictionary<string, object>)expando).Add(field, value);
}
}
return expando;
}
Check my blog for a detailed post: https://jinishbhardwaj.wordpress.com/2016/12/03/web-api-supporting-data-shaping/