We Keep Coding

C# MongoDB Driver: Can't find the way to run complex query for AnyIn filter in MongoDB

I have a document like this:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
And I have an user that has a list of authorized groups, like for example the following:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
What I'm trying to achieve is to retrieve all documents in the database that the user is authorized to retrieve, essentially I want to be able to check in the list "groupsAuthorized" if one of the group contains a subset of an element of the other list "groups" contained in my user authorizations
using the following values:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
my user permissions:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
the user should be able to retrieve this document as the string "France/IDF" is correctly contained in the string "France/IDF/Paris", however, if the values would've been like this:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF"
]
}
my user permissions:
"groups": [
"France/IDF/Paris",
"USA/NY/NYC"
]
it should not work, because my user is only authorized to view documents from France/IDF/Paris and USA/NY/NYC and none of the string inside of the authorizedGroups of my document contains those sequences
I've tried to use a standard LINQ query to achieve this which is fairly simple:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var results = collection.AsQueryable()
.Where(entity => userAuthorizedGroups
.Any(userGroup => entity.authorizedGroups
.Any(entityAuthorizedGroup => entityAuthorizedGroup.Contains(userGroup))));
But i'm getting the famous unsupported filter exception that it seems lot of people is having, i've tried different options found on the internet like the following:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var filter = Builders<PartitionedEntity<Passport>>.Filter.AnyIn(i => i.authorizedGroups, userAuthorizedGroups);
var results = (await collection.FindAsync(filter)).ToList();
return results;
But the problem is this will only check if one of the element of the array is contained inside the other array, It will not correctly work for case like "France/IDF" that should correctly match "France/IDF/Paris" because "France/IDF" string is contained inside the "France/IDF/Paris" string inside of my document
I'm getting a bit clueless on how to achieve this using the mongodb C# driver, i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
Has anyone an Idea on this subject ?

i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
don't do it :)
One place you can start with is here. It describes all the LINQ operators that are supported by the MongoDB .NET driver. As you can see .Contains() isn't mentioned there which means you can't use it and you'll get an arror in the runtime but it does not mean that there's no way to do what you're trying to achieve.
The operator closest to contains you can use is $indexOfBytes which returns -1 if there's no match and the position of a substring otherwise. Also since you need to match an array against another array you need two pairs of $map and $anyElementTrue to do exactly what .NET's .Any does.
Your query (MongoDB client) can look like this:
db.collection.find({
$expr: {
$anyElementTrue: {
$map: {
input: "$groupsAuthorized",
as: "group",
in: {
$anyElementTrue: {
$map: {
input: ["France/IDF/Paris", "USA/NY/NYC"],
as: "userGroup",
in: { $ne: [ -1, { $indexOfBytes: [ "$$userGroup", "$$group" ] } ] }
}
}
}
}
}
}
})
Mongo Playground,
You can run the same query from .NET using BsonDocument class which takes a string (JSON) and converts into a query:
var query = BsonDocument.Parse(#"{
$expr: {
$anyElementTrue:
{
$map:
{
input: '$groupsAuthorized',
as: 'group',
in: {
$anyElementTrue:
{
$map:
{
input: ['France/IDF/Paris', 'USA/NY/NYC'],
as: 'userGroup',
in: { $ne: [-1, { $indexOfBytes: ['$$userGroup', '$$group'] } ] }
}
}
}
}
}
}
}");
var result = col.Find(query).ToList();

How to Replace All in MongoDB Driver? w/ C# [duplicate]

I was surprised to find that the following example code only updates a single document:
> db.test.save({"_id":1, "foo":"bar"});
> db.test.save({"_id":2, "foo":"bar"});
> db.test.update({"foo":"bar"}, {"$set":{"test":"success!"}});
> db.test.find({"test":"success!"}).count();
1
I know I can loop through and keep updating until they're all changed, but that seems terribly inefficient. Is there a better way?

Multi update was added recently, so is only available in the development releases (1.1.3). From the shell you do a multi update by passing true as the fourth argument to update(), where the the third argument is the upsert argument:
db.test.update({foo: "bar"}, {$set: {test: "success!"}}, false, true);
For versions of mongodb 2.2+ you need to set option multi true to update multiple documents at once.
db.test.update({foo: "bar"}, {$set: {test: "success!"}}, {multi: true})
For versions of mongodb 3.2+ you can also use new method updateMany() to update multiple documents at once, without the need of separate multi option.
db.test.updateMany({foo: "bar"}, {$set: {test: "success!"}})

Starting in v3.3 You can use updateMany
db.collection.updateMany(
<filter>,
<update>,
{
upsert: <boolean>,
writeConcern: <document>,
collation: <document>,
arrayFilters: [ <filterdocument1>, ... ]
}
)
In v2.2, the update function takes the following form:
db.collection.update(
<query>,
<update>,
{ upsert: <boolean>, multi: <boolean> }
)
https://docs.mongodb.com/manual/reference/method/db.collection.update/

For Mongo version > 2.2, add a field multi and set it to true
db.Collection.update({query},
{$set: {field1: "f1", field2: "f2"}},
{multi: true })

I've created a way to do this with a better interface.
db.collection.find({ ... }).update({ ... }) -- multi update
db.collection.find({ ... }).replace({ ... }) -- single replacement
db.collection.find({ ... }).upsert({ ... }) -- single upsert
db.collection.find({ ... }).remove() -- multi remove
You can also apply limit, skip, sort to the updates and removes by chaining them in beforehand.
If you are interested, check out Mongo-Hacker

To Update Entire Collection,
db.getCollection('collection_name').update({},
{$set: {"field1" : "value1", "field2" : "value2", "field3" : "value3"}},
{multi: true })

In the MongoDB Client, type:
db.Collection.updateMany({}, $set: {field1: 'field1', field2: 'field2'})
New in version 3.2
Params::
{}: select all records updated
Keyword argument multi not taken

MongoDB will find only one matching document which matches the query criteria when you are issuing an update command, whichever document matches first happens to be get updated, even if there are more documents which matches the criteria will get ignored.
so to overcome this we can specify "MULTI" option in your update statement, meaning update all those documnets which matches the query criteria. scan for all the documnets in collection finding those which matches the criteria and update :
db.test.update({"foo":"bar"},{"$set":{"test":"success!"}}, {multi:true} )

The following command can update multiple records of a collection
db.collection.update({},
{$set:{"field" : "value"}},
{ multi: true, upsert: false}
)

The updateMany() method has the following form:
db.collection.updateMany(
<filter>,
<update>,
{
upsert: <boolean>,
writeConcern: <document>,
collation: <document>,
arrayFilters: [ <filterdocument1>, ... ],
hint: <document|string> // Available starting in MongoDB 4.2.1
}
)
The restaurant collection contains the following documents:
{ "_id" : 1, "name" : "Central Perk Cafe", "violations" : 3 }
{ "_id" : 2, "name" : "Rock A Feller Bar and Grill", "violations" : 2 }
{ "_id" : 3, "name" : "Empire State Sub", "violations" : 5 }
{ "_id" : 4, "name" : "Pizza Rat's Pizzaria", "violations" : 8 }
The following operation updates all documents where violations are greater than 4 and $set a flag for review:
try {
db.restaurant.updateMany(
{ violations: { $gt: 4 } },
{ $set: { "Review" : true } }
);
} catch (e) {
print(e);
}

All latest versions of mongodb updateMany() is working fine
db.getCollection('workers').updateMany({},{$set: {"assignedVehicleId" : "45680"}});

I had the same problem , and i found the solution , and it works like a charm
just set the flag multi to true like this :
db.Collection.update(
{_id_receiver: id_receiver},
{$set: {is_showed: true}},
{multi: true} /* --> multiple update */
, function (err, updated) {...});
i hope that helps :)

You can use.`
Model.update({
'type': "newuser"
}, {
$set: {
email: "abc#gmail.com",
phoneNumber:"0123456789"
}
}, {
multi: true
},
function(err, result) {
console.log(result);
console.log(err);
}) `

Thanks for sharing this, I used with 2.6.7 and following query just worked,
for all docs:
db.screen.update({stat:"PRO"} , {$set : {stat:"pro"}}, {multi:true})
for single doc:
db.screen.update({stat:"PRO"} , {$set : {stat:"pro"}}, {multi:false})

Multiple Filters and Updates in One Mongo Query [duplicate]

I have a collection which elements can be simplified to this:
{tags : [1, 5, 8]}
where there would be at least one element in array and all of them should be different. I want to substitute one tag for another and I thought that there would not be a problem. So I came up with the following query:
db.colll.update({
tags : 1
},{
$pull: { tags: 1 },
$addToSet: { tags: 2 }
}, {
multi: true
})
Cool, so it will find all elements which has a tag that I do not need (1), remove it and add another (2) if it is not there already. The problem is that I get an error:
"Cannot update 'tags' and 'tags' at the same time"
Which basically means that I can not do pull and addtoset at the same time. Is there any other way I can do this?
Of course I can memorize all the IDs of the elements and then remove tag and add in separate queries, but this does not sound nice.

The error is pretty much what it means as you cannot act on two things of the same "path" in the same update operation. The two operators you are using do not process sequentially as you might think they do.
You can do this with as "sequential" as you can possibly get with the "bulk" operations API or other form of "bulk" update though. Within reason of course, and also in reverse:
var bulk = db.coll.initializeOrderedBulkOp();
bulk.find({ "tags": 1 }).updateOne({ "$addToSet": { "tags": 2 } });
bulk.find({ "tags": 1 }).updateOne({ "$pull": { "tags": 1 } });
bulk.execute();
Not a guarantee that nothing else will try to modify,but it is as close as you will currently get.
Also see the raw "update" command with multiple documents.

If you're removing and adding at the same time, you may be modeling a 'map', instead of a 'set'. If so, an object may be less work than an array.
Instead of data as an array:
{ _id: 'myobjectwithdata',
data: [{ id: 'data1', important: 'stuff'},
{ id: 'data2', important: 'more'}]
}
Use data as an object:
{ _id: 'myobjectwithdata',
data: { data1: { important: 'stuff'},
data2: { important: 'more'} }
}
The one-command update is then:
db.coll.update(
'myobjectwithdata',
{ $set: { 'data.data1': { important: 'treasure' } }
);
Hard brain working for this answer done here and here.

Starting in Mongo 4.4, the $function aggregation operator allows applying a custom javascript function to implement behaviour not supported by the MongoDB Query Language.
And coupled with improvements made to db.collection.update() in Mongo 4.2 that can accept an aggregation pipeline, allowing the update of a field based on its own value,
We can manipulate and update an array in ways the language doesn't easily permit:
// { "tags" : [ 1, 5, 8 ] }
db.collection.updateMany(
{ tags: 1 },
[{ $set:
{ "tags":
{ $function: {
body: function(tags) { tags.push(2); return tags.filter(x => x != 1); },
args: ["$tags"],
lang: "js"
}}
}
}]
)
// { "tags" : [ 5, 8, 2 ] }
$function takes 3 parameters:
body, which is the function to apply, whose parameter is the array to modify. The function here simply consists in pushing 2 to the array and filtering out 1.
args, which contains the fields from the record that the body function takes as parameter. In our case, "$tag".
lang, which is the language in which the body function is written. Only js is currently available.

In case you need replace one value in an array to another check this answer:
Replace array value using arrayFilters

How to implement MongoDB nested $elemMatch Query in C#

I have a MongoDB collection in the following format.
{
"_id" : ObjectId("56c6f03ffd07dc1de805e84f"),
"Details" : {
"a" : [
[ {
"DeviceID" : "log0",
"DeviceName" : "Dev0"
},
{
"DeviceID" : "log1",
"DeviceName" : "Dev1"
}
],
[ {
"DeviceID" : "Model0",
"DeviceName" : "ModelName0"
},
{
"DeviceID" : "Model1",
"DeviceName" : "ModelName1"
}
]
]
}
}
And I am trying to fetch all the documents where the DeviceName in array "a" contains a particular value, say "Name0". However I could get the desired result while using below Mongo query:
db.test_collection.find({"Details.a":{$elemMatch:{$elemMatch:{DeviceName : /.*Name0.*/}}}});
Now I am struggling to implement the above query in C#. Can anyone guide me with that?
so far I have tried the below code and it was not working as expected
query = Query.And(Query.ElemMatch("Details.a", Query.And(Query.ElemMatch("DeviceName", Query.Matches("DeviceName", new BsonRegularExpression("Name0"))))));
Thanks in advance

Well, honestly writing queries in C# are bit tricky but you can always play a trick.
var bsonQuery = "{'Details.a':{$elemMatch:{$elemMatch:{DeviceName : /.*Name0.*/}}}}";
var filter = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(bsonQuery);
var result = col.FindSync (filter).ToList();
I'm deserializing a plain MongoDB queries into a BsonDocument which in return I'm passing to FindAsync as filter.
In the end, you'll have desired outcome in variable result.
Note: I'm assuming MongoDB connection has been established and variable col holds reference to MongoDB collection.
EDIT: Please see following link https://groups.google.com/forum/#!topic/mongodb-csharp/0dcoVlbFR2A. Now it's confirmed that C# driver doesn't support nameless filters so writing above query using Buidlers<BsonDocument>.Filter at moment is not supported.
Long story short, you are left with only one choice and that is to query as I mentioned above in my solution.

Get entity by its parent entity's value in Elasticsearch

Imagine we have this simple entity in ES
user {
username: "ultrauser"
name: "Greg"
address: [
{ city: "London" }, {city: "Prague" }
]
}
I need a query, which will return me all addreses for user "ultrauser".
I'm using NEST and for now, I'm doing it the way I select user where username="ultrauser" and then read only the addresses field by myself. So the ES will return me whole user entity, which contain addresses too.
But is it possible for ES to make query like "Give me all addresses, which belongs to user 'ultrauser'"?
I need ES to return list of addresses entities, not user entity, containing addresses. It is simple when you go from root to leafs (get user.address.city), but how can I select from leafs to root easily?
Important is, we can not use parent-child or nested document features because of other reasons.
Thanks for all you ideas.

You should probably read this article: http://www.elasticsearch.org/blog/managing-relations-inside-elasticsearch/ You're trying to apply RDBMS concepts to elasticsearch and that is usually a bad idea. Really, even if you are storing objects, they are still stored flat in elasticsearch behind the scenes.
I think this query will get you to where you want to be though, if I'm understanding you correctly:
{
"query": {
"bool": {
"must": [
{
"term": {
"username": "some matched item"
}
},
{
"filtered": {
"filter": {
"exists": { "field" : "address" }
}
}
}
]
}
},
"fields": [
"address"
]
}
Does it matter if you extract the addresses or if you ask elasticsearch to do it for you? Sometimes you don't want to send all that data over the wire if not needed and that might be your reason.
This will still return something like this:
hits: [
{
_index: indexname
_type: typename
_id: id
_score: 1.4142135
fields: {
address: [
{someaddress_object}
]
}
}, ...
So you will still need to loop through the results anyway when you get them back, just the result size is smaller.