We Keep Coding

C# MongoDB Driver: Can't find the way to run complex query for AnyIn filter in MongoDB

I have a document like this:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
And I have an user that has a list of authorized groups, like for example the following:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
What I'm trying to achieve is to retrieve all documents in the database that the user is authorized to retrieve, essentially I want to be able to check in the list "groupsAuthorized" if one of the group contains a subset of an element of the other list "groups" contained in my user authorizations
using the following values:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
my user permissions:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
the user should be able to retrieve this document as the string "France/IDF" is correctly contained in the string "France/IDF/Paris", however, if the values would've been like this:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF"
]
}
my user permissions:
"groups": [
"France/IDF/Paris",
"USA/NY/NYC"
]
it should not work, because my user is only authorized to view documents from France/IDF/Paris and USA/NY/NYC and none of the string inside of the authorizedGroups of my document contains those sequences
I've tried to use a standard LINQ query to achieve this which is fairly simple:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var results = collection.AsQueryable()
.Where(entity => userAuthorizedGroups
.Any(userGroup => entity.authorizedGroups
.Any(entityAuthorizedGroup => entityAuthorizedGroup.Contains(userGroup))));
But i'm getting the famous unsupported filter exception that it seems lot of people is having, i've tried different options found on the internet like the following:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var filter = Builders<PartitionedEntity<Passport>>.Filter.AnyIn(i => i.authorizedGroups, userAuthorizedGroups);
var results = (await collection.FindAsync(filter)).ToList();
return results;
But the problem is this will only check if one of the element of the array is contained inside the other array, It will not correctly work for case like "France/IDF" that should correctly match "France/IDF/Paris" because "France/IDF" string is contained inside the "France/IDF/Paris" string inside of my document
I'm getting a bit clueless on how to achieve this using the mongodb C# driver, i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
Has anyone an Idea on this subject ?

i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
don't do it :)
One place you can start with is here. It describes all the LINQ operators that are supported by the MongoDB .NET driver. As you can see .Contains() isn't mentioned there which means you can't use it and you'll get an arror in the runtime but it does not mean that there's no way to do what you're trying to achieve.
The operator closest to contains you can use is $indexOfBytes which returns -1 if there's no match and the position of a substring otherwise. Also since you need to match an array against another array you need two pairs of $map and $anyElementTrue to do exactly what .NET's .Any does.
Your query (MongoDB client) can look like this:
db.collection.find({
$expr: {
$anyElementTrue: {
$map: {
input: "$groupsAuthorized",
as: "group",
in: {
$anyElementTrue: {
$map: {
input: ["France/IDF/Paris", "USA/NY/NYC"],
as: "userGroup",
in: { $ne: [ -1, { $indexOfBytes: [ "$$userGroup", "$$group" ] } ] }
}
}
}
}
}
}
})
Mongo Playground,
You can run the same query from .NET using BsonDocument class which takes a string (JSON) and converts into a query:
var query = BsonDocument.Parse(#"{
$expr: {
$anyElementTrue:
{
$map:
{
input: '$groupsAuthorized',
as: 'group',
in: {
$anyElementTrue:
{
$map:
{
input: ['France/IDF/Paris', 'USA/NY/NYC'],
as: 'userGroup',
in: { $ne: [-1, { $indexOfBytes: ['$$userGroup', '$$group'] } ] }
}
}
}
}
}
}
}");
var result = col.Find(query).ToList();

Convert Mongodb Aggregation Pipeline to C# .Net Core

Perhaps someone can help me out with a problem to convert a mongodb aggregation query to C# using the mongodb .Net driver.
According to my problem here I tried to convert the following to C#
db.getCollection('test').aggregate([
{ "$facet": {
"allInRoot1": [{
"$match": { "rootReferenceId": LUUID("9f3a73df-bca7-48b7-b111-285359e50a02") }
}],
"allInRoot2": [{
"$match": { "rootReferenceId": LUUID("27f2b4a6-5471-406a-a39b-1e0b0f8c4eb9") }
}]
}},
{ "$project": {
"difference": {
"$filter": {
"input": "$allInRoot1",
"as": "this",
"cond": { "$in": [ "$$this.reference.id", { "$setDifference": [ "$allInRoot1.reference.id", "$allInRoot2.reference.id" ] } ] }
}
}
}}
])
So far I have this
var matchFilterOne = new ExpressionFilterDefinition<NodeModel>(node => node.RootReferenceId == baseId);
var matchStageOne = PipelineStageDefinitionBuilder.Match(matchFilterOne);
var pipelineOne = PipelineDefinition<NodeModel, NodeModel>.Create(new IPipelineStageDefinition[] { matchStageOne });
var matchFilterTwo = new ExpressionFilterDefinition<NodeModel>(node => node.RootReferenceId == idToExclude);
var matchStageTwo = PipelineStageDefinitionBuilder.Match(matchFilterTwo);
var pipelineTwo = PipelineDefinition<NodeModel, NodeModel>.Create(new IPipelineStageDefinition[] { matchStageTwo });
var facetPipelineOne = AggregateFacet.Create("allInRoot1", pipelineOne);
var facetPipelineTwo = AggregateFacet.Create("allInRoot2", pipelineTwo);
var test = testCollection.Aggregate()
.Facet(facetPipelineOne, facetPipelineTwo)
/* This seems to fail because the facet structure is wrong and it can't access the $allInRoot1 field ...
.Project(#"{
'difference': {
'$filter': {
'input': '$allInRoot1',
'as': 'this',
'cond': {
'$in': [ '$$this.reference.id', { '$setDifference': [ '$allInRoot1.reference.id', '$allInRoot2.reference.id' ] }]
}
}}}")
*/
.FirstOrDefault();
Perhaps someone has a clue pointing me in the right direction? Is it also possible to use the projection with types?
Any help is appreciated!

The best solution so far that i've seen is copying the whole aggregation array into MongoDbCompass's Aggregations tab and then export it in C#.

Map projection result with mongodb C# driver

I need to map to a simplified document some documents in a collection.
I can obtain what I need with this in the mongo shell:
db.getCollection('items').aggregate([
{ "$project": {
"Team": "$TeamId",
"Marker": "$Properties.marker.Value"
}}
])
I need to obtain the same result with C# driver (version 2.3.0); I tried this
var aggregation = m_database.GetCollection<BsonDocument>("items").Aggregate();
var projectionDefinition = new BsonDocument("$project", new BsonDocument
{
{ "Team", "$TeamId"},
{ "Marker", "$Properties.marker.Value" }
});
var query = aggregation.Project(projectionDefinition);
var result = await query.ToListAsync();
but I get the following error
Command aggregate failed: $expressions are not allowed at the top-level of $project
Someone kwons what's going on?

if you call Project you do have already $project in your bson,
so you just simplify your projectionDefinition:
var projectionDefinition = new BsonDocument
{
{ "Team", "$TeamId"},
{ "Marker", "$Properties.marker.Value" }
};
My personal opinion: I would avoid using pure bson, MongoDB driver gives you possibilities to use your c# dto classes for it.

MongoDB C# driver, query by an array element using regex

I am using MongoDB C# driver version 2.2. My collection contains "Parent" objects. Each parent object has an array of children objects. Each child has name value:
"parent": {
"children":[
{ "name": "Bob", "age": 10},
{ "name": "Alice", "age": 7},
{ "name": "Tobias", "age": 11}
]
}
I need to translate the following code into C# statements / LINQ syntax:
db.getCollection('Parents').find({'parent.children': { $elemMatch: { 'name': { $regex: '.*ob.*', $options: 'im' } }}})
I have found there are methods like
var builder = Builders<BsonDocument>.Filter;
builder.Regex("parent.children.name", new BsonRegularExpression(".*ob.*")); //does not work with array
and
builder.AnyEq("parent.children.name", "ob"); //without regex
But I cannot understand how to combine them. Please advise.
UPDATE:
I am using the following for now, please correct me if you know a reason why it should not work correctly:
builder.AnyEq("parent.children.name", new BsonRegularExpression(".*ob.*"))

I am using the following for now:
builder.AnyEq("parent.children.name", new BsonRegularExpression(".*ob.*"))

Can't test C# on this machine. Let me know if this doesn't work:
var filter = Builders<People>.Filter.ElemMatch(x => x.Parent.Children, x => Regex.IsMatch(x.Name, "regex"));
var res = await collection.Find(filter).ToListAsync();
Here's a trick you might like btw:
// Take your inputted `find` query string:
string bsonQuery = "{'parent.children': { $elemMatch: { 'name': { $regex: '.*ob.*', $options: 'im' } }}}";
// Use it as the filter!
var filter = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(bsonQuery);
// Results:
var result = col.FindSync (filter).ToList();

I've tested your current expression (builder.AnyEq("parent.children.name", new BsonRegularExpression(".*ob.*")) on a database at home and I don't believe it behaves in the way you intend.
Although the c# documentation doesn't explicitly state this, mongoDB supports the Regex filter on an array field. I have tested the below expression in C# and have correct results for the Regex despite the field being an array.
builder.Regex(MONGO_FIELD_NAME, new BsonRegularExpression("SOME REGEX"));
In addition, I've tested Regex on the toy example in the [online mongo webshell for query arrays] (https://docs.mongodb.com/manual/tutorial/query-arrays/). Querying for db.inventory.find({tags : {$regex : "^bl"}} will return results with "blank" or "blue" despite the "tag" field being an array.

ElasticSearch - .NET NEST API building multiple aggregrations with object initialization API seems to create an incorrect request

I am building the components of an elastic search query dynamically, the query contains potentially multiple aggregations.
.Net 4.5.2 Nest 2.3.1 Elasticsearch.Net 2.3.1
I can successfully add multiple aggregations by repeating the following structure:
var aggregations = new AggregationDictionary();
aggregations["yyy"] = new AggregationContainer {
Terms = new TermsAggregation("xxx")
{
Field = "afield"
}
};
And then setting the Aggregrations property on the search to the aggregations variable. And all is good.
I can successfully created a single nested aggregration as follows:
var aggregations=new NestedAggregation("Countries") {
Path = "MetaData.GeographicCoverage.Countries",
Aggregations =
new TermsAggregation("Country") {
Field = "MetaData.GeographicCoverage.Countries.Country"
}
};
And again setting the Aggregrations property on the search to the aggregations variable and all is good.
The problem comes when I combine the two approaches to create a query with many aggregrations where one (or more) of which is nested. So the Json generated by the above nested example looks like:
{
"size": 0,
"aggs": {
"Countries": {
"nested": {
"path": "MetaData.GeographicCoverage.Countries"
},
"aggs": {
"Country": {
"terms": {
"field": "MetaData.GeographicCoverage.Countries.Country"
}
}
}
}
}
}
Now when I combine the approaches so that the nested aggregration is added just like those in the very first code snippet:
var aggregations = new AggregationDictionary();
var nested = new NestedAggregation("Countries") {
Path = "MetaData.GeographicCoverage.Countries",
Aggregations =
new TermsAggregation("Country") {
Field = "MetaData.GeographicCoverage.Countries.Country"
}
};
aggregations["Countries"] = new AggregationContainer {
Nested = nested
};
Then the Json of the query that is generated misses the actual "Country" aggregration:
{
"size": 0,
"aggs": {
"Countries": {
"nested": {
"path": "MetaData.GeographicCoverage.Countries"
}
}
}
}
So then - is this a bug or am I using the Nest classes incorrectly? If I am using the classes incorrectly how do I fix the code?
Thanks for any help.

Turns out the way around the problem is to add an explicit cast of the NestedAggregration to an AggregationContainer and then to add it directly to the AggregationsDictionary.
var aggregations = new AggregationDictionary();
var nested =
new NestedAggregation("Countries") {
Path = "MetaData.GeographicCoverage.Countries",
Aggregations =
new TermsAggregation("Country") {
Field = "MetaData.GeographicCoverage.Countries.Country"
}
};
aggregations["Countries"] = (AggregationContainer)nested;