We Keep Coding

Issue deserializing a date represented as a string in document from Cosmos DB

I currently have a C# application that queries Cosmos DB. In the database everything is stored as strings. When I query the container for the document and pull out the information it looks like the deserialization process is treating it like a date and its applying the Date Time Offset. Strange part is this process of querying the DB never converts it to a date? I would like it to just pass through the app as its stored in Cosmos.
Here is the document shape in Cosmos:
{
"id": "uniqueId",
"MyDate" : "2021-07-24T23:59:00+0000"
}
Here is the code that connects to CosmosDB
using(var client = new CosmosClient("EndPoint","Key")
{
string sqlQueryText = "Select * From c Where .... ";
QueryDefinition querySQL = new QueryDefination(sqlQueryText);
var container = client.GetContainer("databaseId", "ContainerId");
var iii = container.GetItemQueryIterator<Document>(querySQL);
var myList = new List<Document>();
while(iii.HasMoreResults)
{
FeedResponse<Document> currentRS = iii.ReadNextAsync().Result;
foreach(Document doc in currentRS)
{
myList.Add(doc);
}
}
}
Here is the Output of the this process in C#, By output I mean if you look at the myList Object and inspect the Mydate Property you will see this string below.
"07/24/2021 19:59:00"
which is the date & time with the date time offset.
How do I prevent this?
This application should just pass the string straight through with no regard for "Real Type".
I would just like to get "2021-07-24T23:59:00+0000"

MongoDB C# Driver how to update a collection of updated documents

I have a Mongo database with lots of documents. All of the documents have a field called "source" which contains the origin name for the document. But a lot of old documents have this field containing "null" (because I hadn't have source by that time) string value. I want to select all those documents and fix this problem by replacing their "source = "null"" values by new values parsed from another fields of the aforementioned documents.
Here's what I'm doing to fix the this:
public void FixAllSources() {
Task.Run(async ()=> {
var filter = Builders<BsonDocument>.Filter.And(new List<FilterDefinition<BsonDocument>>() {
Builders<BsonDocument>.Filter.Exists("url"),
Builders<BsonDocument>.Filter.Ne("url", BsonNull.Value),
Builders<BsonDocument>.Filter.Eq("source", "null")
});
var result = await m_NewsCollection.FindAsync(filter);
var list = result.ToList();
list.ForEach(bson => {
bson["source"] = Utils.ConvertUrlToSource(bson["url"].AsString);
});
});
}
as you can see, I'm fetching all those documents and replacing their source field by a new value. So now I've got a List of correct BsonDocuments which I need to put back into the database.
Unfortunately, I'm stuck on this. The problem is that all of the "Update" methods require filters and I have no idea what filters should I pass there. I just need to put back all those updated documents, and that's it.
I'd appreciate any help :)
p.s.
I've came up with an ugly solution like this:
list.ForEach(bson => {
bson["source"] = Utils.ConvertUrlToSource(bson["url"].AsString);
try {
m_NewsCollection.UpdateOne( new BsonDocument("unixtime", bson["unixtime"]), new BsonDocument {{"$set", bson}},
new UpdateOptions {IsUpsert = true});
}
catch (Exception e) {
WriteLine(e.StackTrace);
}
});
It works for now. But I'd still like to know the better one in case I need to do something like this again. So I'm not putting my own solution as an answer

Parse LUIS builtin.datetime.date

I have built a Luis model, and I have encountered issues parsing the builtin.datetime.date for an entry like "this weeked" returns 2016-W20-WE (or something similar) which doesnt parse successfuly using DateTime.Parse.
I came across this post which seems to use a class called LUISParse, but I can't find any reference to this on Google.

I have the same problem and see that in their sdk doc they use Chronic on the actual entity string rather than trying to interpret the date format. See Here:
So I did something like this:
Chronic.Parser parser = new Chronic.Parser();
EntityRecommendation date = new EntityRecommendation();
luisResponse.TryFindEntity("builtin.datetime.date", out date);
var dateResult = parser.Parse(date.Entity);
It works since the date.Entity only contains the date related string. Ex: I pass in "Yes I am planning to go next week" to Luis and the entity contains "next week".

Rather than use the raw value, I've found I had to dig into the internal elements and use the "value", "start" and "end" properties. Those properties at least follow date formats. This works for durations such as Weekend or Next 3 Days as well as Today and specific date requests.
if (entity.Type == "builtin.datetimeV2.daterange")
{
var resolutionValues = (IList<object>)entity.Resolution["values"];
foreach (var value in resolutionValues)
{
_start = Convert.ToDateTime(((IDictionary<string, object>)value)["start"]);
_end = Convert.ToDateTime(((IDictionary<string, object>)value)["end"]);
}
}
else if (entity.Type == "builtin.datetimeV2.date")
{
var resolutionValues = (IList<object>)entity.Resolution["values"];
foreach (var value in resolutionValues)
{
_when = Convert.ToDateTime(((IDictionary<string, object>)value)["value"]);
}
}

IPP .NET c# SDK allows query filters, but this does not reduce payload?

Referencing the following IPP Documentation:
https://developer.intuit.com/docs/0025_quickbooksapi/0055_devkits/0150_ipp_.net_devkit_3.0/query_filters
I made the assumption that the following code using the Linq Extentions Projection would alter the request and reduce the payload of the response by only querying for the requested fields and only including those fields (narrow result set) in the response:
public List<ShortAccount> GetFullShortAccountList(bool logRequestResponse)
{
var accounts = new List<ShortAccount>();
var accountQueryService = new QueryService<Account>
(GetIppServiceContext(logRequestResponse));
var selected = accountQueryService.Select(a => new { a.Id, a.Name });
foreach (var account in selected)
{
accounts.Add(new ShortAccount { Id = account.Id, Name = account.Name });
}
return accounts;
}
Now the behavior of this method is as expected, but if I look at the request/response logs (or the actual request and response using Fiddler) the request doesn't change -- it is still "Select * from Account", and the response still includes all the other properties in the Account entity.
In other words, the payload is not reduced one iota.
Am I doing something wrong here? Or do I just understand this incorrectly?
How can I use the SDK to generate a query that would look more like "Select Id, Name from Account", and only return that result set?
Related question -- if this mode of query filtering does not reduce the payload, what is its purpose? You might as well get the whole shebang and just take the fields you need?
Thanks in advance.

That's right #Barrick. The implementation of our query providers is not exactly the same as the standard LINQ. So, Stephan, that's the issue.
If you just want to get specific fields I would suggest you to use IDSQuery like:
QueryService<Account> AccQueryService22 = new QueryService<Account>(context);
var t13 = AccQueryService22.ExecuteIdsQuery("Select Id, Name From Account Where Active in (true, false)");
I will forward the feedback to our team.
Thanks!

MongoDB: update only specific fields

I am trying to update a row in a (typed) MongoDB collection with the C# driver. When handling data of that particular collection of type MongoCollection<User>, I tend to avoid retrieving sensitive data from the collection (salt, password hash, etc.)
Now I am trying to update a User instance. However, I never actually retrieved sensitive data in the first place, so I guess this data would be default(byte[]) in the retrieved model instance (as far as I can tell) before I apply modifications and submit the new data to the collection.
Maybe I am overseeing something trivial in the MongoDB C# driver how I can use MongoCollection<T>.Save(T item) without updating specific properties such as User.PasswordHash or User.PasswordSalt? Should I retrieve the full record first, update "safe" properties there, and write it back? Or is there a fancy option to exclude certain fields from the update?
Thanks in advance

Save(someValue) is for the case where you want the resulting record to be or become the full object (someValue) you passed in.
You can use
var query = Query.EQ("_id","123");
var sortBy = SortBy.Null;
var update = Update.Inc("LoginCount",1).Set("LastLogin",DateTime.UtcNow); // some update, you can chain a series of update commands here
MongoCollection<User>.FindAndModify(query,sortby,update);
method.
Using FindAndModify you can specify exactly which fields in an existing record to change and leave the rest alone.
You can see an example here.
The only thing you need from the existing record would be its _id, the 2 secret fields need not be loaded or ever mapped back into your POCO object.

It´s possible to add more criterias in the Where-statement. Like this:
var db = ReferenceTreeDb.Database;
var packageCol = db.GetCollection<Package>("dotnetpackage");
var filter = Builders<Package>.Filter.Where(_ => _.packageName == packageItem.PackageName.ToLower() && _.isLatestVersion);
var update = Builders<Package>.Update.Set(_ => _.isLatestVersion, false);
var options = new FindOneAndUpdateOptions<Package>();
packageCol.FindOneAndUpdate(filter, update, options);

Had the same problem and since I wanted to have 1 generic method for all types and didn't want to create my own implementation using Reflection, I end up with the following generic solution (simplified to show all in one method):
Task<bool> Update(string Id, T item)
{
var serializerSettings = new JsonSerializerSettings()
{
NullValueHandling = NullValueHandling.Ignore,
DefaultValueHandling = DefaultValueHandling.Ignore
};
var bson = new BsonDocument() { { "$set", BsonDocument.Parse(JsonConvert.SerializeObject(item, serializerSettings)) } };
await database.GetCollection<T>(collectionName).UpdateOneAsync(Builders<T>.Filter.Eq("Id", Id), bson);
}
Notes:
Make sure all fields that must not update are set to default value.
If you need to set field to default value, you need to either use DefaultValueHandling.Include, or write custom method for that update
When performance matters, write custom update methods using Builders<T>.Update
P.S.: It's obviously should have been implemented by MongoDB .Net Driver, however I couldn't find it anywhere in the docs, maybe I just looked the wrong way.

Well there are many ways to updated value in mongodb.
Below is one of the simplest way I choose to update a field value in mongodb collection.
public string UpdateData()
{
string data = string.Empty;
string param= "{$set: { name:'Developerrr New' } }";
string filter= "{ 'name' : 'Developerrr '}";
try
{
//******get connections values from web.config file*****
var connectionString = ConfigurationManager.AppSettings["connectionString"];
var databseName = ConfigurationManager.AppSettings["database"];
var tableName = ConfigurationManager.AppSettings["table"];
//******Connect to mongodb**********
var client = new MongoClient(connectionString);
var dataBases = client.GetDatabase(databseName);
var dataCollection = dataBases.GetCollection<BsonDocument>(tableName);
//****** convert filter and updating value to BsonDocument*******
BsonDocument filterDoc = BsonDocument.Parse(filter);
BsonDocument document = BsonDocument.Parse(param);
//********Update value using UpdateOne method*****
dataCollection.UpdateOne(filterDoc, document);
data = "Success";
}
catch (Exception err)
{
data = "Failed - " + err;
}
return data;
}
Hoping this will help you :)