We Keep Coding

DynamoDb converting Attribute Values to Types

Is there a more efficient way of converting dynamo db data into concrete types? For example, when I query the data everything is in:
List<Dictionary<string, AttributeValue>>
Is it possible to easily convert the type without having to loop through each item and doing this all manually?
For example I am doing:
return items.Select(item => new Connection
{
ConnectionId = Guid.Parse(item["connectionId"].S),
ClientId = item["clientId"].S,
ProviderId = item["providerId"].S,
Scopes = item["scopes"].SS.ToArray(),
CredentialsId = item["credentialsId"].S,
Evidences = ToEvidences(item["consentEvidences"].L)
})
.ToList();
This then returns a list of my type Connection however I am explicitly mapping each field. Is there an easier way or a helper library that can do the mapping?

I think you'll have luck with the higher-level .NET Document model. It presents more natural data types.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DotNetSDKMidLevel.html

The easiest way I have found is to use the Document.FromAttributeMap function to convert it to a Document object and then convert it again to the .NET type using the DynamoDBContext.FromDocument method as shown below.
public async Task<IEnumerable<WeatherForecast>> GetAll(string cityName)
{
var queryRequest = new QueryRequest()
{
TableName = nameof(WeatherForecast),
KeyConditionExpression = "CityName = :cityName",
ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{":cityName", new AttributeValue(cityName)},
}
};
var response = await _dynamoDbClient.QueryAsync(queryRequest);
return response.Items.Select(a =>
{
var doc = Document.FromAttributeMap(a);
return _dynamoDbContext.FromDocument<WeatherForecast>(doc);
});
}

Replace property values in a class from List<Dictionary> values

I have a method that takes a List<Dictionary<string,object>> as a parameter. The plan is to use that parameter, but only update the values held in a particular class. Here is the (partially written) method
public async Task<Errors> UpdatePageForProject(Guid projectId, List<Dictionary<string, object>> data)
{
if (!IsValidUserIdForProject(projectId))
return new Errors { ErrorMessage = "Project does not exist", Success = false };
if (data.Count == 0)
return new Errors { ErrorMessage = "No data passed to change", Success = false };
var page = await _context.FlowPages.FirstOrDefaultAsync(t => t.ProjectId == projectId);
foreach (var d in data)
{
}
return new Errors { Success = true };
}
My original plan is to take each dictionary, check if the key and the property in page match and then alter the value (so I can pass in 1 dictionary or 8 dictionaries in the list and then alter page to save back to my entity database).
I'd rather not use reflection due to the speed hit (though C#9 is really fast, I'd still rather not use it), but I'm not sure how else this can be done. I did consider using AutoMapper to do this, but for now would rather not (it's a PoC, so it is possibly overkill)

If you want to do this without Reflection (which I agree is a good idea, not just for performance reasons) then you could use a "map" or lookup table with actions for each property.
var map = new Dictionary<string,Action<Page,object>>()
{
{ "Title", (p,o) => p.Title = (string)o },
{ "Header", (p,o) => p.Field1 = (string)o },
{ "DOB", (p,o) => p.DateOfBirth = (DateTime)o }
};
You can then iterate over your list of dictionaries and use the map to execute actions that update the page.
foreach (var dictionary in data)
{
foreach (entry in dictionary)
{
var action = map[entry.Key];
action(page, entry.Value);
}
}

RavenDB: PatchRequest to update/merge property that may not exist in document

I have a document with the following domain model:
class Entity
{
...
Dictionary<string, string> Settings { get; set; }
...
}
And there's a need to update specified collection. But not override - merge with incoming updates. As I need to process thousands of documents in that manner, I choosed the PatchCommand for better performance. Got following:
new PatchCommandData
{
Key = upd.EntityId,
Patches = new[]
{
// Foreach incoming Setting remove existing value (if any) and add the new one
new PatchRequest
{
Type = PatchCommandType.Modify,
Name = nameof(Entity.Settings),
Nested = upd.UpdatedSettings.Keys
.Select(x => new PatchRequest
{
Type = PatchCommandType.Unset,
Name = x
})
.ToArray()
.Union(upd.UpdatedSettings.Keys
.Select(x => new PatchRequest
{
Type = PatchCommandType.Set,
Name = x,
Value = upd.UpdatedSettings[x]
})
.ToList())
.ToArray(),
Value = RavenJToken.FromObject(upd.UpdatedSettings)
}
}
}
This way next update is performed:
Before: { Setting1 = "Value1", Setting2 = "Value2" }
Update request: { Setting2 = "NewValue2", Setting3 = "Value3" }
After: { Setting1 = "Value1", Setting2 = "NewValue2", Setting3 = "Value3" }
But.. There's always a "but". If there is a document without Settings property in db, provided patch will raise an error saying "Cannot modify value from Settings because it was not found".
I can't find any option to switch patch mode to Set vs. Modify on the fly. And there is no option to load all documents, apply the update on application's side and update thousands of documents.
The only reasonable option I can see is to create Dictionary instance for Settings property in the class constructor.
Folks, can you advice some other options?
P.S. RavenDB version is limited to 3.5

Done. You can find the final version below. Actually, the ScriptedPatchCommandData was used. It contains JavaScript body, that Raven will evaluate and execute against documents. Potentially, it isn't the best option from the performance point of view. But nothing critical as it turned out. Hope it may be helpful for someone.
const string fieldParameterName = "fieldName";
const string valueParameterName = "value";
var patch = updates
.Select(update => new ScriptedPatchCommandData
{
Key = update.EntityId,
Patch = new ScriptedPatchRequest
{
Script = $"this[{fieldParameterName}] = _.assign({{}}, this[{fieldParameterName}], JSON.parse({valueParameterName}));",
Values = new Dictionary<string, object>
{
{
fieldParameterName,
nameof(Entity.Settings)
},
{
valueParameterName,
JsonConvert.SerializeObject(update.UpdatedSettings)
}
}
}
})
.ToList();
return await DocumentStore.AsyncDatabaseCommands.BatchAsync(patch);

Cannot Convert from Http.QueryString to System.collections.IEqualityComparer [duplicate]

I am given an absolute URI that contains a query string. I'm looking to safely append a value to the query string, and change an existing parameter.
I would prefer not to tack on &foo=bar, or use regular expressions, URI escaping is tricky. Rather I want to use a built-in mechanism that I know will do this correctly and handle the escaping.
I've found a ton of answers that all use HttpUtility. However this being ASP.NET Core, there is no more System.Web assembly anymore, thus no more HttpUtility.
What is the appropriate way to do this in ASP.NET Core while targeting the core runtime?

If you are using ASP.NET Core 1 or 2, you can do this with Microsoft.AspNetCore.WebUtilities.QueryHelpers in the Microsoft.AspNetCore.WebUtilities package.
If you are using ASP.NET Core 3.0 or greater, WebUtilities is now part of the ASP.NET SDK and does not require a separate nuget package reference.
To parse it into a dictionary:
var uri = new Uri(context.RedirectUri);
var queryDictionary = Microsoft.AspNetCore.WebUtilities.QueryHelpers.ParseQuery(uri.Query);
Note that unlike ParseQueryString in System.Web, this returns a dictionary of type IDictionary<string, string[]> in ASP.NET Core 1.x, or IDictionary<string, StringValues> in ASP.NET Core 2.x or greater, so the value is a collection of strings. This is how the dictionary handles multiple query string parameters with the same name.
If you want to add a parameter on to the query string, you can use another method on QueryHelpers:
var parametersToAdd = new System.Collections.Generic.Dictionary<string, string> { { "resource", "foo" } };
var someUrl = "http://www.google.com";
var newUri = Microsoft.AspNetCore.WebUtilities.QueryHelpers.AddQueryString(someUrl, parametersToAdd);
Using .net core 2.2 you can get the query string using
var request = HttpContext.Request;
var query = request.Query;
foreach (var item in query){
Debug.WriteLine(item)
}
You will get a collection of key:value pairs - like this
[0] {[companyName, ]}
[1] {[shop, ]}
[2] {[breath, ]}
[3] {[hand, ]}
[4] {[eye, ]}
[5] {[firstAid, ]}
[6] {[eyeCleaner, ]}

The easiest and most intuitive way to take an absolute URI and manipulate it's query string using ASP.NET Core packages only, can be done in a few easy steps:
Install Packages
PM> Install-Package Microsoft.AspNetCore.WebUtilities
PM> Install-Package Microsoft.AspNetCore.Http.Extensions
Important Classes
Just to point them out, here are the two important classes we'll be using: QueryHelpers, StringValues, QueryBuilder.
The Code
// Raw URI including query string with multiple parameters
var rawurl = "https://bencull.com/some/path?key1=val1&key2=val2&key2=valdouble&key3=";
// Parse URI, and grab everything except the query string.
var uri = new Uri(rawurl);
var baseUri = uri.GetComponents(UriComponents.Scheme | UriComponents.Host | UriComponents.Port | UriComponents.Path, UriFormat.UriEscaped);
// Grab just the query string part
var query = QueryHelpers.ParseQuery(uri.Query);
// Convert the StringValues into a list of KeyValue Pairs to make it easier to manipulate
var items = query.SelectMany(x => x.Value, (col, value) => new KeyValuePair<string, string>(col.Key, value)).ToList();
// At this point you can remove items if you want
items.RemoveAll(x => x.Key == "key3"); // Remove all values for key
items.RemoveAll(x => x.Key == "key2" && x.Value == "val2"); // Remove specific value for key
// Use the QueryBuilder to add in new items in a safe way (handles multiples and empty values)
var qb = new QueryBuilder(items);
qb.Add("nonce", "testingnonce");
qb.Add("payerId", "pyr_");
// Reconstruct the original URL with new query string
var fullUri = baseUri + qb.ToQueryString();
To keep up to date with any changes, you can check out my blog post about this here: http://benjii.me/2017/04/parse-modify-query-strings-asp-net-core/

HttpRequest has a Query property which exposes the parsed query string via the IReadableStringCollection interface:
/// <summary>
/// Gets the query value collection parsed from owin.RequestQueryString.
/// </summary>
/// <returns>The query value collection parsed from owin.RequestQueryString.</returns>
public abstract IReadableStringCollection Query { get; }
This discussion on GitHub points to it as well.

This function return Dictionary<string, string> and does not use Microsoft.xxx for compatibility
Accepts parameter encoding in both sides
Accepts duplicate keys (return last value)
var rawurl = "https://emp.com/some/path?key1.name=a%20line%20with%3D&key2=val2&key2=valdouble&key3=&key%204=44#book1";
var uri = new Uri(rawurl);
Dictionary<string, string> queryString = ParseQueryString(uri.Query);
// queryString return:
// key1.name, a line with=
// key2, valdouble
// key3,
// key 4, 44
public Dictionary<string, string> ParseQueryString(string requestQueryString)
{
Dictionary<string, string> rc = new Dictionary<string, string>();
string[] ar1 = requestQueryString.Split(new char[] { '&', '?' });
foreach (string row in ar1)
{
if (string.IsNullOrEmpty(row)) continue;
int index = row.IndexOf('=');
if (index < 0) continue;
rc[Uri.UnescapeDataString(row.Substring(0, index))] = Uri.UnescapeDataString(row.Substring(index + 1)); // use Unescape only parts
}
return rc;
}

It's important to note that in the time since the top answer has been flagged as correct that Microsoft.AspNetCore.WebUtilities has had a major version update (from 1.x.x to 2.x.x).
That said, if you're building against netcoreapp1.1 you will need to run the following, which installs the latest supported version 1.1.2:
Install-Package Microsoft.AspNetCore.WebUtilities -Version 1.1.2

I use this as extention method, works with any number of params:
public static string AddOrReplaceQueryParameter(this HttpContext c, params string[] nameValues)
{
if (nameValues.Length%2!=0)
{
throw new Exception("nameValues: has more parameters then values or more values then parameters");
}
var qps = new Dictionary<string, StringValues>();
for (int i = 0; i < nameValues.Length; i+=2)
{
qps.Add(nameValues[i], nameValues[i + 1]);
}
return c.AddOrReplaceQueryParameters(qps);
}
public static string AddOrReplaceQueryParameters(this HttpContext c, Dictionary<string,StringValues> pvs)
{
var request = c.Request;
UriBuilder uriBuilder = new UriBuilder
{
Scheme = request.Scheme,
Host = request.Host.Host,
Port = request.Host.Port ?? 0,
Path = request.Path.ToString(),
Query = request.QueryString.ToString()
};
var queryParams = QueryHelpers.ParseQuery(uriBuilder.Query);
foreach (var (p,v) in pvs)
{
queryParams.Remove(p);
queryParams.Add(p, v);
}
uriBuilder.Query = "";
var allQPs = queryParams.ToDictionary(k => k.Key, k => k.Value.ToString());
var url = QueryHelpers.AddQueryString(uriBuilder.ToString(),allQPs);
return url;
}
Next and prev links for example in a view:
var next = Context.Request.HttpContext.AddOrReplaceQueryParameter("page",Model.PageIndex+1+"");
var prev = Context.Request.HttpContext.AddOrReplaceQueryParameter("page",Model.PageIndex-1+"");

I'm not sure at what point it was added but as early as .NET Core 3.1 HttpUtility.ParseQueryString is available and built into the standard .NET Microsoft.NETCore.App framework. I was able to access it from a class library with no additional NuGet packages or special dll references required.
I've tested it with .NET Core 3.1, .NET 6, and .NET 7.
The benefit of this approach is that you do not have to reference any web libraries which will add bloat to your project if you are building a library that may be used outside of the context of ASP.NET.
... of course, if you are needing this within an ASP.NET Core app, the Microsoft.AspNetCore.WebUtilities class suggested by others is perfectly valid.
EDIT: #KTCO pointed out in his comment on the OP back in 2017 that this class was available in .NET Core 2.0 as well.

Couchbase Lite 2 + JsonConvert

The following code sample writes a simple object to a couchbase lite (version 2) database and reads all objects afterwards. This is what you can find in the official documentation here
This is quite a lot of manual typing since every property of every object must be transferred to the MutableObject.
class Program
{
static void Main(string[] args)
{
Couchbase.Lite.Support.NetDesktop.Activate();
const string DbName = "MyDb";
var db = new Database(DbName);
var item = new Item { Name = "test", Value = 5 };
// Serialization HERE
var doc = new MutableDocument();
doc.SetString("Name", item.Name);
doc.SetInt("Value", item.Value);
db.Save(doc);
using (var qry = QueryBuilder.Select(SelectResult.All())
.From(DataSource.Database(db)))
{
foreach (var result in qry.Execute())
{
var resultItem = new Item
{
// Deserialization HERE
Name = result[DbName].Dictionary.GetString("Name"),
Value = result[DbName].Dictionary.GetInt("Value")
};
Console.WriteLine(resultItem.Name);
}
}
Console.ReadKey();
}
class Item
{
public string Name { get; set; }
public int Value { get; set; }
}
}
From my research Couchbase lite uses JsonConvert internally, so there might be a way to simplify all that with the help of JsonConvert.
Anything like:
var json = JsonConvert.SerializeObject(item);
var doc = new MutableDocument(json); // No overload to provide raw JSON
or maybe
var data = JsonConvert.SerializeToDict(item); // JsonConvert does not provide this
var doc = new MutableDocument(data);
Is there or is this some kind of optimization and the preferred approach is by intend?

People ask about this quite often, but Couchbase Lite does not actually store JSON strings in the database. They are stored in a different format so this would not give the benefit that you think (the JSON would need to be reparsed and then broken down into the other format). I'd been pushing for a way to serialize classes directly instead of going through dictionary objects (which seems like the ultimate goal here) but our priority is on things that enterprise clients want and this doesn't seem to be one of them. Note that for it to make it in, it needs to be implemented in C# Java and Objective-C / Swift.

I don't know about JsonConvert but there seems to be a constructor that takes IDictionary<string, object> as argument. So I would try something like this (brain-compiled):
MutableDocument CreateDocument(object data)
{
if (data == null) return null;
var propertyValues = new Dictionary<string, object>();
foreach (var property in data.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance))
{
propertyValues[property.Name] = property.GetValue(data);
}
return new MutableDocument(propertyValues);
}
See if this works.