We Keep Coding

C# MongoDB Driver: Can't find the way to run complex query for AnyIn filter in MongoDB

I have a document like this:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
And I have an user that has a list of authorized groups, like for example the following:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
What I'm trying to achieve is to retrieve all documents in the database that the user is authorized to retrieve, essentially I want to be able to check in the list "groupsAuthorized" if one of the group contains a subset of an element of the other list "groups" contained in my user authorizations
using the following values:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
my user permissions:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
the user should be able to retrieve this document as the string "France/IDF" is correctly contained in the string "France/IDF/Paris", however, if the values would've been like this:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF"
]
}
my user permissions:
"groups": [
"France/IDF/Paris",
"USA/NY/NYC"
]
it should not work, because my user is only authorized to view documents from France/IDF/Paris and USA/NY/NYC and none of the string inside of the authorizedGroups of my document contains those sequences
I've tried to use a standard LINQ query to achieve this which is fairly simple:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var results = collection.AsQueryable()
.Where(entity => userAuthorizedGroups
.Any(userGroup => entity.authorizedGroups
.Any(entityAuthorizedGroup => entityAuthorizedGroup.Contains(userGroup))));
But i'm getting the famous unsupported filter exception that it seems lot of people is having, i've tried different options found on the internet like the following:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var filter = Builders<PartitionedEntity<Passport>>.Filter.AnyIn(i => i.authorizedGroups, userAuthorizedGroups);
var results = (await collection.FindAsync(filter)).ToList();
return results;
But the problem is this will only check if one of the element of the array is contained inside the other array, It will not correctly work for case like "France/IDF" that should correctly match "France/IDF/Paris" because "France/IDF" string is contained inside the "France/IDF/Paris" string inside of my document
I'm getting a bit clueless on how to achieve this using the mongodb C# driver, i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
Has anyone an Idea on this subject ?

i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
don't do it :)
One place you can start with is here. It describes all the LINQ operators that are supported by the MongoDB .NET driver. As you can see .Contains() isn't mentioned there which means you can't use it and you'll get an arror in the runtime but it does not mean that there's no way to do what you're trying to achieve.
The operator closest to contains you can use is $indexOfBytes which returns -1 if there's no match and the position of a substring otherwise. Also since you need to match an array against another array you need two pairs of $map and $anyElementTrue to do exactly what .NET's .Any does.
Your query (MongoDB client) can look like this:
db.collection.find({
$expr: {
$anyElementTrue: {
$map: {
input: "$groupsAuthorized",
as: "group",
in: {
$anyElementTrue: {
$map: {
input: ["France/IDF/Paris", "USA/NY/NYC"],
as: "userGroup",
in: { $ne: [ -1, { $indexOfBytes: [ "$$userGroup", "$$group" ] } ] }
}
}
}
}
}
}
})
Mongo Playground,
You can run the same query from .NET using BsonDocument class which takes a string (JSON) and converts into a query:
var query = BsonDocument.Parse(#"{
$expr: {
$anyElementTrue:
{
$map:
{
input: '$groupsAuthorized',
as: 'group',
in: {
$anyElementTrue:
{
$map:
{
input: ['France/IDF/Paris', 'USA/NY/NYC'],
as: 'userGroup',
in: { $ne: [-1, { $indexOfBytes: ['$$userGroup', '$$group'] } ] }
}
}
}
}
}
}
}");
var result = col.Find(query).ToList();

How to query sub document (list type) in Cosmos Db

I have a collection in Azure CosmosDB, and each document looks like this:
{
"id": "random",
"TeacherName": "Ben",
"Students": [
{
"Name": "John",
"Telephone": ""
},
{
"Name": "Mary",
"Telephone": ""
}
],
}
TeacherName is string, Students is a list of student
I need to do: Given a user name (user1), query and return all the documents, either teacher name is "user1" or there is a student with name "user1".
I tried a few options, but cannot do it.
The closest solution I found so far is to use .SelectMany(), but I found .SelectMany will do a join and will duplicate the return results.
This is my query:
client.CreateDocumentQuery().SelectMany((x) => x.Students.Where(s=>s.Name == "user1" || x.TeacherName == "user1")
If only the above sample document in the collection, and when I searched user name "Ben", 2 records will be returned ((number of result) * (number of students)). I have to remove the duplicate at client side and pagination is kind of broken.
Is it possible to issue a single query to achieve what I need?

client.CreateDocumentQuery().SelectMany((x) => x.Students.Where(s=>s.Name == "user1" || x.TeacherName == "user1")
Actually the SelectMany method in your code is similar to the sql as below:
SELECT c.id from c
join x in c.Students
where c.TeacherName ='Ben' or x.Name = 'Ben'
Output
[
{
"id": "1"
},
{
"id": "1"
}
]
If there's a join, there's going to be duplicate data. As far as I know, automatic removal of duplicate data is not supported (like Distinct Keywords in traditional database) by Azure Cosmos DB yet.
It seems that there is no way to remove the repeat data in the query SQL level.
If you don't want to handle with your query result set in the loop locally, I strongly suggest you using a stored procedure to handle with result data in Azure Cosmos DB to release the pressure on your local server.
Or , you could complete the query directly in the Stored Procedure if your data is not too large.Please refer to the js code as below:
// SAMPLE STORED PROCEDURE
function sample(idsArrayString,courses) {
var collection = getContext().getCollection();
var isAccepted = collection.queryDocuments(
collection.getSelfLink(),
'SELECT * FROM root r',
function (err, feed, options) {
if (err) throw err;
if (!feed || !feed.length) getContext().getResponse().setBody('no docs found');
else {
var returnResult = [];
for(var i = 0;i<feed.length;i++){
var doc = feed[i];
if(doc.TeacherName == 'Ben'){
returnResult.push(feed[i]);
break;
}
var std = doc.Students;
for (var s in std) {
if(s.Name == 'Ben'){
returnResult.push(feed[i]);
break;
}
}
}
getContext().getResponse().setBody(returnResult);
}
});
if (!isAccepted) throw new Error('The query was not accepted by the server.');
}
Update Answer:
I checked the Azure Cosmos DB pricing details. Then it doesn't show that stored procedure is more expensive than a single query. Actually, the cost depends on Rus and RuS depends on the complexity of the query and the amount of concurrency, etc.
You could refer to the RUs document. Also , you could know the RUs charge by the http request header :x-ms-request-charge. Please see this useful trace:
How to caculate the Azure Cosmos DB RU used in server side scripting.
Hope it helps you.

DocumentDB unsupported query

I try to use dictionary in IQueryable but I received run time error ,I know the problem occur because in real time IQueryable is not familiar with that object , I try to convert IQueryable to IEnumerable , but I have problem with the execution of the query.
May someone can give me a hint how to execute that function ?
I have the following code:
Dictionary<String, int> coursesType= new Dictionary<string, int>();
var userQuery = _client.CreateDocumentQuery<ObjectModel.Student>(uriStudentCollection, options).
Where(x =>coursesType.ContainsKey(x.MainCourse)
&& !x.Courses.ContainsKey(requestCourse)).OrderBy(x => x.Grade).AsDocumentQuery();
var feedResponse = await userQuery.ExecuteNextAsync<ObjectModel.Student>();
foreach (var ad in feedResponse.AsEnumerable())
{
results.Add(ad);
}
UPDATE STATUS: I STILL NOT RECEIVED ANSWER TO MY QUESTION
***UPDATE : I add example of my doc.
{
"id": "a5d7f123-80d5-5094-84fb-08c3bc4ccp972",
"StudentName": "Philip",
"Courses": {
"Math": {
"id": "Math",
"Grade": "98",
"Place": "NYC"
}
},
"Rank":"AA"
}
UPDATE NUMBER 3
I write the following query :
SqlQuerySpec q = new SqlQuerySpec()
{
QueryText = "SELECT * FROM c WHERE (CONTAINS(LOWER(c[\"courseName\"]),#text) OR CONTAINS(LOWER(c[\"courseDescription\"]),#text) ) AND (udf.CourseContainsKey(c[\"Courses\"],#courseId)=false)",
Parameters = new SqlParameterCollection()
{
new SqlParameter("#text", text),
new SqlParameter("#courseId", courseId)
}
};
When I write the query like that, the query work fine, but IF I add the ORDER BY command to the query I received empty set....
"SELECT * FROM c WHERE (CONTAINS(LOWER(c[\"courseName\"]),#text) OR CONTAINS(LOWER(c[\"courseDescription\"]),#text) ) AND (udf.CourseContainsKey(c[\"Courses\"],#courseId)=false) ORDER BY c[\"courseName\"] ASC"
Thanks
Thanks,
MAK

{"Method 'ContainsKey' is not supported."}
Based on your query, you could use the following code:
var userQuery = _client.CreateDocumentQuery<ObjectModel.Student>(uriStudentCollection, options).
Where(x =>coursesType.Keys.Contains(x.MainCourse)
&& !x.Courses.Keys.Contains(requestCourse)).OrderBy(x => x.Grade).AsDocumentQuery();
Additionally, if you enable Cross partition query, you would get the following error:
Cross partition query with TOP/ORDER BY or aggregate functions is not supported.
If the filter could not executed on CosmosDB side, I assumed that you need to pull the records from azure side, then filter on your client side. Additionally, here is a similar issue, you could refer to here.
UPDATE:
Sample document:
{
"id": "1ba6178b-7c22-440a-a4a2-25b4bc636b30",
"MainCourse": "b",
"Grade": "B",
"Courses": {
"a": "a",
"b": "b"
}
}
Query:
SELECT * FROM root WHERE ((root["MainCourse"] IN ("a", "b")) AND (root["Courses"]["a"] != null)) ORDER BY root["Grade"] ASC
Modify your C# code as follows:
!x.Courses.Keys.Contains(requestCourse)
//To
x.Courses[requestCourse]==null
UPDATE2:
For filtering a specific course name not contains within the Courses property, I assumed that you could use User-defined functions, here is the code snippet, you could refer to it:
udf.CourseContainsKey:
function CourseContainsKey (courses,courseName) {
if(courses==undefined||courseName==undefined)
return false;
return courses.hasOwnProperty(courseName);
}
Test:
UPDATE3:
I have not found any better way to create the query with the UDF, you could follow the code below to create your document query:
var query = $"SELECT* FROM root WHERE (root[\"MainCourse\"] IN({String.Join(",", coursesType.Keys.Select(k=>$"\"{k}\"").ToArray())})) AND udf.CourseContainsKey(root[\"Courses\"],\"{requestCourse}\")=false ORDER BY root[\"Grade\"] ASC";
var items=client.CreateDocumentQuery<CourseSample>(UriFactory.CreateDocumentCollectionUri(DatabaseId, DocumentCollectionId), sqlExpression:query).ToList();

MongoDB C# driver, query by an array element using regex

I am using MongoDB C# driver version 2.2. My collection contains "Parent" objects. Each parent object has an array of children objects. Each child has name value:
"parent": {
"children":[
{ "name": "Bob", "age": 10},
{ "name": "Alice", "age": 7},
{ "name": "Tobias", "age": 11}
]
}
I need to translate the following code into C# statements / LINQ syntax:
db.getCollection('Parents').find({'parent.children': { $elemMatch: { 'name': { $regex: '.*ob.*', $options: 'im' } }}})
I have found there are methods like
var builder = Builders<BsonDocument>.Filter;
builder.Regex("parent.children.name", new BsonRegularExpression(".*ob.*")); //does not work with array
and
builder.AnyEq("parent.children.name", "ob"); //without regex
But I cannot understand how to combine them. Please advise.
UPDATE:
I am using the following for now, please correct me if you know a reason why it should not work correctly:
builder.AnyEq("parent.children.name", new BsonRegularExpression(".*ob.*"))

I am using the following for now:
builder.AnyEq("parent.children.name", new BsonRegularExpression(".*ob.*"))

Can't test C# on this machine. Let me know if this doesn't work:
var filter = Builders<People>.Filter.ElemMatch(x => x.Parent.Children, x => Regex.IsMatch(x.Name, "regex"));
var res = await collection.Find(filter).ToListAsync();
Here's a trick you might like btw:
// Take your inputted `find` query string:
string bsonQuery = "{'parent.children': { $elemMatch: { 'name': { $regex: '.*ob.*', $options: 'im' } }}}";
// Use it as the filter!
var filter = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(bsonQuery);
// Results:
var result = col.FindSync (filter).ToList();

I've tested your current expression (builder.AnyEq("parent.children.name", new BsonRegularExpression(".*ob.*")) on a database at home and I don't believe it behaves in the way you intend.
Although the c# documentation doesn't explicitly state this, mongoDB supports the Regex filter on an array field. I have tested the below expression in C# and have correct results for the Regex despite the field being an array.
builder.Regex(MONGO_FIELD_NAME, new BsonRegularExpression("SOME REGEX"));
In addition, I've tested Regex on the toy example in the [online mongo webshell for query arrays] (https://docs.mongodb.com/manual/tutorial/query-arrays/). Querying for db.inventory.find({tags : {$regex : "^bl"}} will return results with "blank" or "blue" despite the "tag" field being an array.

Mongodb Array Update

I have following mongodb json.I would like to update price of product_id:2 from 4 to 5 in the document with document _id:id1
Can someone guide me how to do that in mongodb or using c#
{
"_id": "id1",
"products": [
{
"product_id": "pr1",
"price":1,
"qty":5
},
{
"product_id": "pr2",
"price":4,
"qty":10
},
{
"product_id": "pr3",
"price":8,
"qty":9
}
]
}

Download the C# mongodb driver from mondodb site.
var collection = DataBase.GetCollection<MyProducts>("products");
var query = Query<MyProduct>.Where(p => (m.productID == "pr2"));
var product = collection.FindOne(query);
product.Price = 5;
collection.Save(product);

Either use the .update function in mongo db or pull the object out using linq something like:
var query = (from c in Collection.AsQueryable<T>() where c.Id == Id select c).First<T>();
adjusting the above variables accordingly.
Then changing whatever that object is then saving it using Collection.Save(obj) this will re write the whole object. Look here: http://docs.mongodb.org/manual/reference/method/db.collection.save/ and http://docs.mongodb.org/manual/tutorial/modify-documents/