We Keep Coding

MongoDB/ - Update Multiple Fields inside an array, keep their original value, but update datatype

So the problem I have is that I want to be able to filter on a particular criteria.
I originally tried to filter based on the datatype of the field, I am struggling to do this with arrays though. This did work at one point with single fields, but now not having much joy.
Then as part of the update I want to set specific fields inside the array from string datatype to say integer.
So to summarise I want to do the following:
Filter where the fields in the array are string datatype as I want to update them.
Update only the datatype and retain the value(essentially converting from string to integer)
Get results with modified count to ensure it was successful.
Thanks in advance.
var mongoDb = mongoClient.GetDatabase(config.DatabaseName); // , new MongoDatabaseSettings() { WriteConcern = WriteConcern.Unacknowledged }
var objectId = new ObjectId("63d38c71ef08fde3fbf1b9da");
var filter = Builders<BsonDocument>.Filter.Eq("_id", objectId);
var updateQuery = new BsonDocument { { "$set", new BsonDocument { { "CheckFinalization.Check.ItemDetail.$[].ItemNumber", new BsonDocument { { "$toInt", "$CheckFinalization.Check.ItemDetail.$[].ItemNumber" } } } } } };
var updatePipeline = Builders<BsonDocument>.Update.Pipeline(PipelineDefinition<BsonDocument, BsonDocument>.Create(updateQuery));
var dataCollection = mongoDb.GetCollection<BsonDocument>(config.CollectionName);
var recordResult = dataCollection.Find(filter).Limit(100).ToList();
// check we have records to update.
if (recordResult.Any())
{
var result = dataCollection.UpdateMany(filter, updatePipeline);
if (result.ModifiedCount > 0)
{
// Success
}
else
{
// Failed
}
}`

Couchbase Lite 2 + JsonConvert

The following code sample writes a simple object to a couchbase lite (version 2) database and reads all objects afterwards. This is what you can find in the official documentation here
This is quite a lot of manual typing since every property of every object must be transferred to the MutableObject.
class Program
{
static void Main(string[] args)
{
Couchbase.Lite.Support.NetDesktop.Activate();
const string DbName = "MyDb";
var db = new Database(DbName);
var item = new Item { Name = "test", Value = 5 };
// Serialization HERE
var doc = new MutableDocument();
doc.SetString("Name", item.Name);
doc.SetInt("Value", item.Value);
db.Save(doc);
using (var qry = QueryBuilder.Select(SelectResult.All())
.From(DataSource.Database(db)))
{
foreach (var result in qry.Execute())
{
var resultItem = new Item
{
// Deserialization HERE
Name = result[DbName].Dictionary.GetString("Name"),
Value = result[DbName].Dictionary.GetInt("Value")
};
Console.WriteLine(resultItem.Name);
}
}
Console.ReadKey();
}
class Item
{
public string Name { get; set; }
public int Value { get; set; }
}
}
From my research Couchbase lite uses JsonConvert internally, so there might be a way to simplify all that with the help of JsonConvert.
Anything like:
var json = JsonConvert.SerializeObject(item);
var doc = new MutableDocument(json); // No overload to provide raw JSON
or maybe
var data = JsonConvert.SerializeToDict(item); // JsonConvert does not provide this
var doc = new MutableDocument(data);
Is there or is this some kind of optimization and the preferred approach is by intend?

People ask about this quite often, but Couchbase Lite does not actually store JSON strings in the database. They are stored in a different format so this would not give the benefit that you think (the JSON would need to be reparsed and then broken down into the other format). I'd been pushing for a way to serialize classes directly instead of going through dictionary objects (which seems like the ultimate goal here) but our priority is on things that enterprise clients want and this doesn't seem to be one of them. Note that for it to make it in, it needs to be implemented in C# Java and Objective-C / Swift.

I don't know about JsonConvert but there seems to be a constructor that takes IDictionary<string, object> as argument. So I would try something like this (brain-compiled):
MutableDocument CreateDocument(object data)
{
if (data == null) return null;
var propertyValues = new Dictionary<string, object>();
foreach (var property in data.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance))
{
propertyValues[property.Name] = property.GetValue(data);
}
return new MutableDocument(propertyValues);
}
See if this works.

How do you filter updates to specific fields from ChangeStream in MongoDB

I am setting up a ChangeStream to notify me when a document has changed in a collection so that I can upsert the "LastModified" element for that document to the time of the event. Since this update will cause a new event to occur on the ChangeStream, I need to filter out these updates to prevent an infinite loop (updating the LastModified element because the LastModified element was just updated...).
I have the following code that is working when I specify the exact field:
ChangeStreamOptions options = new ChangeStreamOptions();
options.ResumeAfter = resumeToken;
string filter = "{ $and: [ { operationType: { $in: ['replace','insert','update'] } }, { 'updateDescription.updatedFields.LastModified': { $exists: false } } ] }";
var pipeline = new EmptyPipelineDefinition<ChangeStreamDocument<BsonDocument>>().Match(filter);
var cursor = collection.Watch(pipeline, options, cancelToken);
However, instead of hard-coding the "updateDescription.updatedFields.LastModified", I would like to provide a list of element names that I don't want to exist in the updatedFields document.
I attempted:
string filter = "{ $and: [ { operationType: { $in: ['replace','insert','update'] } }, { 'updateDescription.updatedFields': { $nin: [ 'LastModified' ] } } ] }";
but this didn't work as expected (I still got the update events for the LastModified change.
I originally was using the Filter Builder:
FilterDefinitionBuilder<ChangeStreamDocument<BsonDocument>> filterBuilder = Builders<ChangeStreamDocument<BsonDocument>>.Filter;
FilterDefinition<ChangeStreamDocument<BsonDocument>> filter = filterBuilder.In("operationType", new string[] { "replace", "insert", "update" }); //Only include the change if it was one of these types. Available types are: insert, update, replace, delete, invalidate
filter &= filterBuilder.Nin("updateDescription.updatedFields", ChangedFieldsToIgnore); //If this is an update, only include it if the field(s) updated contains 1+ fields not in the ChangedFieldsToIgnore list
where ChangedFieldsToIgnore is a List containing the field names that I do not want to get events for.
Can anyone help with the syntax that I need to use? or do I need to create a loop around my ChangedFieldsToIgnore list and create a new entry in the filter for each item to "$exists: false"? (this doesn't seem very efficient).
EDIT:
I attempted the following code based on the answer by #wan-bachtiar, but I'm getting an exception on my enumerator.MoveNext() call:
var match1 = new BsonDocument { { "$match", new BsonDocument { { "operationType", new BsonDocument { { "$in", new BsonArray(new string[] { "replace", "insert", "update" }) } } } } } };
var match2 = new BsonDocument { { "$addFields", new BsonDocument { { "tmpfields", new BsonDocument { { "$objectToArray", "$updateDescription.updatedFields" } } } } } };
var match3 = new BsonDocument { { "$match", new BsonDocument { { "tmpfields.k", new BsonDocument { { "$nin", new BsonArray(updatedFieldsToIgnore) } } } } } };
var pipeline = new[] { match1, match2, match3 };
var cursor = collection.Watch<ChangeStreamDocument<BsonDocument>>(pipeline, options, Profile.CancellationToken);
enumerator = cursor.ToEnumerable().GetEnumerator();
enumerator.MoveNext();
ChangeStreamDocument<BsonDocument> doc = enumerator.Current;
The exception is: "{"Invalid field name: \"tmpfields\"."}"
I suspect the problem might be that I'm getting "replace" and "insert" events which do not contain the updateDescription field, so the $addFields/$objectToArray are failing. I'm too new to figure out the syntax, but I think I need to use a filter that does:
{ $match: { "operationType": { $in: ["replace", "insert"] } } }
OR
{ $eq: { "operationTYpe": "update" }} AND { $addFields....}
Also, it appears that the C# driver does not include a Builder that helps with the $addFields and $objectToArray operations. I was only able to use the new BsonDocument {...} method to build the pipeline variable.

ChangedFieldsToIgnore is a List containing the field names that I do not want to get events for.
If you would like to filter based on multiple keys (whether updatedFields contains certain fields), it's easier if you convert the keys to values first.
You can convert the document contained within updatedFields into values by utilising aggregation operator $objectToArray. For example:
pipeline = [{"$addFields": {
"tmpfields":{
"$objectToArray":"$updateDescription.updatedFields"}
}},
{"$match":{"tmpfields.k":{
"$nin":["LastModified", "AnotherUnwantedField"]}}}
];
The above aggregation pipeline adds a temporary field called tmpfields. This new field will pivot the content of updateDescription.updatedFields turning {name:value} into [{k:name, v:value}]. Once we have those keys as values, we can utilise $nin as an array of filter.
UPDATED
The reason you're getting an exception of tmpfields being invalid, is because the result is casted into ChangeStreamDocument model which does not have a recognizable field called tmpfields.
In the case, when it's different operations that does not have field updateDescription.updatedFields, the value of tmpfields would just be null.
Below is an example of MongoDB ChangeStream .Net/C# using MongoDB .Net driver v2.5, along with an aggregation pipeline that modifies the output change stream.
This example is not type safe, and would return BsonDocument :
var database = client.GetDatabase("database");
var collection = database.GetCollection<BsonDocument>("collection");
var options = new ChangeStreamOptions { FullDocument = ChangeStreamFullDocumentOption.UpdateLookup };
// Aggregation Pipeline
var addFields = new BsonDocument {
{ "$addFields", new BsonDocument {
{ "tmpfields", new BsonDocument {
{ "$objectToArray",
"$updateDescription.updatedFields" }
} }
} } };
var match = new BsonDocument {
{ "$match", new BsonDocument {
{ "tmpfields.k", new BsonDocument {
{ "$nin", new BsonArray{"LastModified", "Unwanted"} }
} } } } };
var pipeline = new[] { addFields, match };
// ChangeStreams
var cursor = collection.Watch<BsonDocument>(pipeline, options);
foreach (var change in cursor.ToEnumerable())
{
Console.WriteLine(change.ToJson());
}

I wrote the piece of code below as I was having the same issues you were having. No need to mess around with BsonObjects ...
//The operationType can be one of the following: insert, update, replace, delete, invalidate
//ignore the field lastrun as we would end in an endles loop
var pipeline = new EmptyPipelineDefinition<ChangeStreamDocument<ATask>>()
.Match("{ operationType: { $in: [ 'replace', 'update' ] } }")
.Match(#"{ ""updateDescription.updatedFields.LastRun"" : { $exists: false } }")
.Match(#"{ ""updateDescription.updatedFields.IsRunning"" : { $exists: false } }");
var options = new ChangeStreamOptions { FullDocument = ChangeStreamFullDocumentOption.UpdateLookup };
var changeStream = Collection.Watch(pipeline, options);
while (changeStream.MoveNext())
{
var next = changeStream.Current;
foreach (var obj in next)
yield return obj.FullDocument;
}

Update a nested Array in Mongo with c#

I have a document like this
{
"_id": "63dafa72f21d48312d8ca405",
"tasks": [{
"_ref": "63d8d8d01beb0b606314e322",
"data": {
"values": [{
"key": "Deadline",
"value": "2014-10-13"
}]
}
}, {
"_ref": "84dd046c6695e32322d842f5",
"data": {
"values": []
}
}]
}
Now I want to update the value inside values which is inside data if the _ref field do match my input.
My code so far:
public bool updateProject(Project dbPro, Project pro)
{
var collection = db.GetCollection<BsonDocument>("projects");
var filter = Builders<BsonDocument>.Filter.Eq("_id", ObjectId.Parse( dbPro.Id));
var update = Builders<BsonDocument>.Update.AddToSetEach("tasks", pro.Tasks);
var result = collection.UpdateOne(filter, update);
if (result.IsModifiedCountAvailable)
{
if (result.ModifiedCount == 1)
{
return true;
}
}
return false;
}
At the moment this code does only append the documents as new tasks instead to append the values to the matching tasks. Maybe someone has an idea how to achieve this behavior?
UPDATE
I tried it like #Shane Oborn said. But its still not working for me.
var collection = db.GetCollection<BsonDocument>("projects");
var filter = Builders<BsonDocument>.Filter.Eq("_id", ObjectId.Parse( dbPro.Id));
var update = Builders<BsonDocument>.Update.Push("tags", buildBsonArrayFromTags(pro.Tags));
var result = collection.UpdateOne(filter, update);
if (result.IsModifiedCountAvailable)
{
if (result.ModifiedCount == 1)
{
return true;
}
}
return false;
}
Instead to override the data it appends an array to my array.
UPDATE
OK instead of push i did need set. And it worked then.

I don't have the exact code accessible, but close. I have a method that performs "upserts" (which "adds" if new, or "updates" if existing). This should get you close:
// The variable "doc" below is a BsonDocument
var updateRequests = new List<WriteModel<BsonDocument>>();
updateRequests.Add(new ReplaceOneModel<BsonDocument>(
CreateBsonDocumentFilterDefinition(filterKeyName, filterKeyValue), doc)
{
IsUpsert = true
});
var writeResult = await collection.BulkWriteAsync(updateRequests);
The key objects here for you are "ReplaceOneModel" and the "IsUpsert" property for the filter definition.
Good luck!
UPDATE:
Another method I have that does updates in subdocuments looks like this:
// Below, "subDocument" is a BsonDocument, and "subDocArrayName" is a string
// that should match the name of the array that contains your sub-document
// that will be updated.
var collection = _database.GetCollection<BsonDocument>(collectionName);
var builder = Builders<BsonDocument>.Update;
var update = builder.Push(subDocArrayName, subDocument);
await collection.UpdateOneAsync(CreateBsonDocumentFilterDefinition(filterKeyName, filterKeyValue), update);

Assigning properties to an object

Okay so I have a small section of code which creates a list of objects based on the data model. I don't want to have to create a class for this. It is used on n ASP.net MVC application for populating a user notifications list.
I know there are plenty of other ways to do this such as actually setting up a class for it(probably the easiest method), but I would like to know if there is a way to do what is displayed below.
List<object> notificationList = new List<object>();
object userNotification = new { Text = "Here is some text!", Url = "http://www.google.com/#q=notifications" };
notificationList.Add(userNotification);
foreach(object notification in notificationList)
{
string value = notification.Text;
}
So I haven't populated the list much but for the purposes here you get the idea. After debug I notice that the Text and Url properties exist, however cannot code to get the values???

You need to use dynamic as variable type in the foreach:
foreach(dynamic notification in notificationList)
{
string value = notification.Text;
}

Edit Oops ... you do need "dynamic", either as the List's generic type, or in the foreach.
var notificationList = new List<dynamic>();
var userNotification = new { Text = "Here is some text!", Url = "http://www.google.com/#q=notifications" };
notificationList.Add(userNotification);
foreach (var notification in notificationList)
{
string value = notification.Text;
}
End edit
Anonymous types should be declared using the var keyword:
var userNotification = new { Text = "Here is some text!", Url = "http://www.google.com/#q=notifications" };
You could also use "dynamic" instead of "var", but that deprives you of compile-time checks, and it appears unnecessary in this case (because the type is fully defined at compile time, within the same method scope). A case where you would need to use "dynamic" is where you want to pass the anonymous-typed object as a parameter to another function, eg:
void Func1()
{
var userNotification = new { Text = "Here is some text!", Url = "http://www.google.com/#q=notifications" };
Func2(userNotification);
}
void Func2(dynamic userNotification)
{
string value = notification.Text;
}

Well you could declare the list as an list of dynimac objects:
List<dynamic> notificationList = new List<object>();
var userNotification = new { Text = "Here is some text!", Url = "http://www.google.com/#q=notifications" };
notificationList.Add(userNotification);
foreach(dynamic notification in notificationList)
{
string value = notification.Text;
}
or use var to let the compiler choose the type:
var notificationList = new []
{
new { Text = "Here is some text!", Url = "http://www.google.com/#q=notifications" }
}.ToList();
foreach(var notification in notificationList)
{
string value = notification.Text;
}