We Keep Coding

Whole MongoDB documented deleted when filtering on a subdocument - c# delete

I've created my MongoDB documents below with subdocuments/arrays, however the arrays aren't named and I would like to delete the whole subdocument if a match is found on an elements within a subdocument.
For example, if a match is found on userID and userLogs.Name. My query is deleting the whole documents instead of only the userLog array. I've also tried other methods such as Pull and PullFilter whilst researching this topic but it doesn't seem to work with this structure, please can you advise on whether there is a way or if I will have to change my document structure?
Document
{
"_id" : ObjectId("43535"),
"userID" : "1",
"userLogs" : [
{
"logID" : 1,
"Name" : "Book 1",
"Genre" : "Fiction",
....
},
{
"logID" : 2,
"Name" : "Book 2",
"Genre" : "Non-Fiction",
....
}
]
}
C# Code behind
var collection = db.GetCollection<BsonDocument>("Users");
var arrayFilter = Builders<BsonDocument>.Filter.Eq("userID", uID) &
Builders<BsonDocument>.Filter.Eq("userLogs.Name", name);
collection.DeleteOne(arrayFilter);

Thank you Christoph! I also solved this using the following method:
var query = Builders<BsonDocument>.Filter.Eq("UserID", uID) &
Builders<BsonDocument>.Filter.Eq("userLogs.Name", name);
var update = Builders<BsonDocument>.Update.Pull("userLogs", new BsonDocument(){
{ "Name", name }
});
collection.UpdateOne(query, update);

C# MongoDB Driver: Can't find the way to run complex query for AnyIn filter in MongoDB

I have a document like this:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
And I have an user that has a list of authorized groups, like for example the following:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
What I'm trying to achieve is to retrieve all documents in the database that the user is authorized to retrieve, essentially I want to be able to check in the list "groupsAuthorized" if one of the group contains a subset of an element of the other list "groups" contained in my user authorizations
using the following values:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF/Paris"
]
}
my user permissions:
"groups": [
"France/IDF",
"USA/NY/NYC"
]
the user should be able to retrieve this document as the string "France/IDF" is correctly contained in the string "France/IDF/Paris", however, if the values would've been like this:
my document:
{
"id": "xxxxxxxxxxxx",
"groupsAuthorized": [
"USA/California/SF",
"France/IDF"
]
}
my user permissions:
"groups": [
"France/IDF/Paris",
"USA/NY/NYC"
]
it should not work, because my user is only authorized to view documents from France/IDF/Paris and USA/NY/NYC and none of the string inside of the authorizedGroups of my document contains those sequences
I've tried to use a standard LINQ query to achieve this which is fairly simple:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var results = collection.AsQueryable()
.Where(entity => userAuthorizedGroups
.Any(userGroup => entity.authorizedGroups
.Any(entityAuthorizedGroup => entityAuthorizedGroup.Contains(userGroup))));
But i'm getting the famous unsupported filter exception that it seems lot of people is having, i've tried different options found on the internet like the following:
var userAuthorizedGroups = new List<string> { "France/IDF/Paris", "USA/NY/NYC" };
var filter = Builders<PartitionedEntity<Passport>>.Filter.AnyIn(i => i.authorizedGroups, userAuthorizedGroups);
var results = (await collection.FindAsync(filter)).ToList();
return results;
But the problem is this will only check if one of the element of the array is contained inside the other array, It will not correctly work for case like "France/IDF" that should correctly match "France/IDF/Paris" because "France/IDF" string is contained inside the "France/IDF/Paris" string inside of my document
I'm getting a bit clueless on how to achieve this using the mongodb C# driver, i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
Has anyone an Idea on this subject ?

i'm starting to think that I should just pull all documents to client and do the filtering manually but that would be quite messy
don't do it :)
One place you can start with is here. It describes all the LINQ operators that are supported by the MongoDB .NET driver. As you can see .Contains() isn't mentioned there which means you can't use it and you'll get an arror in the runtime but it does not mean that there's no way to do what you're trying to achieve.
The operator closest to contains you can use is $indexOfBytes which returns -1 if there's no match and the position of a substring otherwise. Also since you need to match an array against another array you need two pairs of $map and $anyElementTrue to do exactly what .NET's .Any does.
Your query (MongoDB client) can look like this:
db.collection.find({
$expr: {
$anyElementTrue: {
$map: {
input: "$groupsAuthorized",
as: "group",
in: {
$anyElementTrue: {
$map: {
input: ["France/IDF/Paris", "USA/NY/NYC"],
as: "userGroup",
in: { $ne: [ -1, { $indexOfBytes: [ "$$userGroup", "$$group" ] } ] }
}
}
}
}
}
}
})
Mongo Playground,
You can run the same query from .NET using BsonDocument class which takes a string (JSON) and converts into a query:
var query = BsonDocument.Parse(#"{
$expr: {
$anyElementTrue:
{
$map:
{
input: '$groupsAuthorized',
as: 'group',
in: {
$anyElementTrue:
{
$map:
{
input: ['France/IDF/Paris', 'USA/NY/NYC'],
as: 'userGroup',
in: { $ne: [-1, { $indexOfBytes: ['$$userGroup', '$$group'] } ] }
}
}
}
}
}
}
}");
var result = col.Find(query).ToList();

in mongodb, what is the most efficient way to get the first and last document

I have documents like this:
class A
{
DateTime T;
...
}
and I would like to find the earliest and the newest document.
Is it better to do this:
var First = db.Collection.AsQueryable().OrderBy(_ => _.t).FirstOrDefault();
var Last = db.Collection.AsQueryable().OrderByDescending(_ => _.t).FirstOrDefault();
or,
var First = db.Collection.AsQueryable().OrderBy(_ => _.t).FirstOrDefault();
var Last = db.Collection.AsQueryable().OrderBy(_ => _.t).LastOrDefault();
or
var C = db.Collection.AsQueryable().OrderBy(_ => _.t);
var First = C.FirstOrDefault();
var Last = C.LastOrDefault();
I am wondering if there is anything with the underlying implementation that would change the speed between these options?
I am wondering if the sort has been done once already, is it possible that the result is cached and getting the first and the last elements would be faster?

Profiler becomes your friend when you're not sure about driver syntax. To enable logging for all queries you need to run on your database below statement:
db.setProfilingLevel(2)
Then to check last query executed on the database you need to run:
db.system.profile.find().limit(1).sort( { ts : -1 } ).pretty()
So for the first code snippet you will get:
"pipeline" : [ { "$sort" : { "t" : 1 } },
{ "$limit" : 1 }
]
"pipeline" : [ { "$sort" : { "t" : -1 } },
{ "$limit" : 1 }
]
For the second pair it prints
"pipeline" : [ { "$sort" : { "t" : 1 } },
{ "$limit" : 1 }
]
and throws NotSupportedException for LastOrDefault on my machine, if it works on your MongoDB driver's version you can check generated MongoDB statement using profiler
For the last one when you hover on c in your Visual Studio it prints
{aggregate([{ "$sort" : { "t" : 1 } }])}
but since it's of type IOrderedQueryable<T> it is only not materialized query so it will get executed on the database when you run FirstOrDefault generating the same aggregation body as previous statements. I'm getting NotSupportedException here as well. Here you can find a list of supported LINQ operators and both Last and LastOrDefault are not implemented so you need to sort descending.

ElasticSearch.net NEST use a Wildcard index in query

Hi I'm new to Elastic Search and I'm making some experiments to understand the basics of query through ElasticSearch.net and NEST.
I'm trying to translate a query with this syntax:
curl -XGET 'http://myserver.com:9200/myindexes-*/XY/_search/?size=1000&pretty=1' -d '
{
"query": {
"bool": {
"filter": [
{ "match": { "LOGTYPE" : "XY" }},
{ "match": { "USER" : "mrossi" }},
{ "wildcard": { "DATA": "m.rossi*" }},
{ "match": { "CODE" : "WZ" }},
{ "range": { "timestamp": { "gte": "2015-05-02" }}}
]
}
}
}
'
I've found how to build the multiple field query, (thanks to those who posted it) but I still need to understand how to create the Wildcard index in the query.
The team feeding the Elastic Search engine has created an index in the format MyIndexes-YYYYMMDD with an index per each date because each date has millions of rows of data (they are log files). I haven't yet found a way to make a query on more than one index, if it is so simple as writing the wildcard in the index name or if I need to do something different.
If you have any clue, Thank you in advance.
Edited after some tests on the answer received: I've tried some tests and even if the call to elastic search seems to be successful reading the results I'm not able to understand if I have a problem in my query or if there are no data.
Valid NEST response built from a successful low level call on POST: /myindex-%2A/ml/_search?pretty=true
# Audit trail of this API call:
- [1] HealthyResponse: Node: http://username:pwd#mydomain.com:9200/ Took: 00:00:00.0580006
# Request:
{"size":1000,"query":{"bool":{"filter":[{"match":{"LOGTYPE":{"query":"XY"}}}]}}}
# Response:
{
"took" : 31,
"timed_out" : false,
"_shards" : {
"total" : 270,
"successful" : 270,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}
The above data is what I can see from the DebugInformation of the response. My uncertainty is connected with how the POST command is built, in fact even if my entity is named XY uppercase in the post appears in lowercase, and also the pretty variable has a value of true instead of 1. the size parameter is set inside the request and not as a variable of the POST as in the sample I posted in the first message.
I've simplified the filters to set just one filter and also in the request data I see that in my sample the "XY" filter has a query: instead of nothing as it is in the sample.
so at the moment the query seems to be sintactically correct but I don't know if it is correct on my index data.
can you suggest how to try and use something like select top 100 * from my table just to see if I can find the data, and then try and implement filters???
thank you again

You can use a wildcard in the Index name portion of your query.
var client = new ElasticClient();
client.Search<MyObject>(s=>s
.Index("myindexes-*")
...
)

You should be able to use the wildcard in the index... is your query not working?

Execute complex mongo JSON query using C# mongo driver

I have following mongo shell query:
{'field':'FieldOne','value':'FieldOne','category':'categoryOne'} , { 'Color' : { '$elemMatch' : { 'Value' : 'Green' } }}
Is there any way to execute same query using C# mongo driver?. I have tried using below C# code, but only the first one is getting executed:
BsonDocument query = BsonDocument.Parse("{'field':'Overall','value':'Overall','category':'LoggedIncidents'} , { 'Priority' : { '$elemMatch' : { 'Value' : 'P1' } }}");
QueryDocument queryDoc = new QueryDocument(query);
var result = collection.Find(queryDoc).ToListAsync().Result;
The first item ({'field':'FieldOne','value':'FieldOne','category':'categoryOne'}) is being executed but not the second one ({ 'Color' : { '$elemMatch' : { 'Value' : 'Green' } }}).

You are using it in find command if i'm not wrong.
the first "{}" is what the filter condition for find is. next set of parameters are essentially to show/hide the fields you need.
so if you want to filter by value of array "green" you can do this.
db.objects.find({"color":{$elemMatch:{value:"green"}}})
should give you the results you are looking for. but if you want to specifically show/hide fields, you can do
db.objects.find({"color":{$elemMatch:{value:"green"}}},{"field":1})
is this what you want to attain?