We Keep Coding

MongoDB - How to update a single object in the array of objects inside a document

I have the following document structure:
{
"Agencies": [
{
"name": "tcs",
"id": "1",
"AgencyUser": [
{
"UserName": "ABC",
"Code": "ABC40",
"Link": "http.ios.com",
"TotalDownloads": 0
},
{
"UserName": "xyz",
"Code": "xyz20",
"Link": "http.ios.com",
"TotalDownloads": 0
}
]
}
]
}
Like I have multiple agencies and each agency contains a list of agents.
What I am trying is to pass the Code and update the TotalDownloads field of the agent that matches the code.
For example, if someone uses the code ABC40 so I need to update the field TotalDownloads of the agent called "ABC".
What I have tried is as below:
public virtual async Task UpdateAgentUsersDownloadByCode(string Code)
{
var col = _db.GetCollection<Agencies>(Agencies.DocumentName);
FilterDefinition<Agencies> filter = Builders<Agencies>.Filter.Eq("AgencyUsers.Code", Code);
UpdateDefinition<Agencies> update = Builders<Agencies>.Update.Inc(x => x.AgencyUsers.FirstOrDefault().TotalDownloads, 1);
await col.UpdateOneAsync(filter, update);
}
It is giving me the following error:
Unable to determine the serialization information for x => x.AgencyUsers.FirstOrDefault().TotalDownloads.
Where I'm wrong?

Note: From the attached sample document, the array property name: AgencyUser is not matched with the property name that you specified in the update operation, AgencyUsers.
Use arrayFilters with $[<identifier>] positional filtered operator to update the element(s) in the array.
MongoDB syntax
db.Agencies.update({
"AgencyUsers.Code": "ABC40"
},
{
$inc: {
"AgencyUsers.$[agencyUser].TotalDownloads": 1
}
},
{
arrayFilters: [
{
"agencyUser.Code": "ABC40"
}
]
})
Demo # Mongo Playground
MongoDB .NET Driver syntax
UpdateDefinition<Agencies> update = Builders<Agencies>.Update
.Inc("AgencyUsers.$[agencyUser].TotalDownloads", 1);
UpdateOptions updateOptions = new UpdateOptions
{
ArrayFilters = new[]
{
new BsonDocumentArrayFilterDefinition<Agencies>(
new BsonDocument("agencyUser.Code", Code)
)
}
};
UpdateResult result = await col.UpdateOneAsync(filter, update, updateOptions);
Demo

Mongo Aggregation With Projection

Given the following document
{
"Id": "MyNormalObjectId",
"CompanyId" : 1234,
"User" :
{
"UserId":4567,
"FirstName":"Tester",
"Lastname":"McCtesterson"
}
}
How do I write my aggregate query to return a list of all unique users in a given company (including first and last name)?
FilterDefinitionBuilder<MyDoc> builder = Builders<MyDoc>.Filter;
var filter = builder .Eq(t => t.CompanyId, companyId);
var aggregate = _col.Aggregate();
aggregate.Match(filter).GroupBy(t=>t.User.UserId, ?????? )
Desired result would be a HashSet of UserMetadata. I've seen a lot of people go straight to BSon for what they need and I can if I HAVE to. Strongly typed is always better.

You can make use of the $addToSet operator inside the $group pipeline stage
to get the desired output.
db.collection.aggregate([
{
"$match": {
"CompanyId": 1234 // Find Conditions go here
}
},
{
"$group": {
"_id": "$CompanyId",
"Users": {
"$addToSet": "$User",
},
},
},
])

Datehistogram aggregate returns empty buckets

I'm trying to aggregate some data by day to make some charts but the datehistogram aggregate return only empty bucket.
My data look like:
Date: July 2nd 2019, 12:08:50.647
_id: 4959287196855971761665003616
And my Nest request :
DateTime now = DateTime.Now;
var descriptor = new SearchDescriptor<ModelWrapper>()
.Index("command-*")
.AllTypes()
.From(0)
.Size(100)
.Sort(s => s.Descending("Date"))
.Aggregations(a => a
.DateHistogram("daily", v => v
.Field(p => p.Date)
.Interval(DateInterval.Day)
.ExtendedBounds(now.AddMonths(-2), now)
))
When I make the request on the kibana console:
{
"aggs" : {
"daily" : {
"date_histogram" : {
"field" : "Date",
"interval" : "1D",
"extended_bounds": {
"min": "2019-06-02T12:01:02.123",
"max": "2019-07-02T12:01:02.123"
},
}
}
}
}
I get the expected result.
I'm on Nest 6.2 and Elastic 6.2

So after some fiddling around I used this piece of code to serialize my SearchDescriptor to the sent string:
var json = m_client.RequestResponseSerializer.SerializeToString(descriptor);
which got me:
{
"aggs": {
"daily": {
"date_histogram": {
"extended_bounds": {
"max": "2019-07-02T16:25:57.522217",
"min": "2019-05-02T16:25:57.522217"
},
"field": "date",
"interval": "day"
}
}
},
"from": 0,
"size": 100,
"sort": [{
"Date": {
"order": "desc"
}
}]
}
The issue here is that my field is put in lowercase, wich is expected behaviour (as pointed out here ).
The solution is to override the DefaultFieldNameInferrer which does this transformation, to do this I added to my connection settings:
var settings = new ConnectionSettings(pool);
settings.DefaultFieldNameInferrer(p => p);
And now everything work as expected.
Edit:
As noted by Russ Cam I can also pass a string to Field() so another solution is just to do:
.Field(nameof(ModelWrapper.Date))

Elasticsearch proper way to escape spaces, ? doesn't work in all scenarios

I'm trying to get searching with spaces to work properly in elasticsearch but having a ton of trouble getting it to behave the same way as it does on another field.
I have two fields, Name and Addresses.First().Line1 that I want to be able to search and preserve spaces in the search. For instance, searching for Bob Smi* would return Bob Smith but not just Bob.
This is working for my Name field by doing a query string search with the space replaced with ?. I'm also doing a wildcard so my final query is *bob?smi*.
However, when I try to also search by line1, I get no results. E.g. *4800* returns a record with line1 like 4800 Street, but when I do the same transformation with 4800 street to get *4800?street*, I get no results.
Below is my query
{
"from": 0,
"size": 50,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "*4800?Street*",
"fields": [
"name",
"addresses.line1"
]
}
}
]
}
}
}
returns no result.
Why would *bob?smi* return result with name Bob Smith but *4800?street* not return result with line item 4800 street?
Below is how both fields are set up in the index:
.Text(smd => smd.Name(c => c.Name).Analyzer(ElasticIndexCreator.SortAnalyzer).Fielddata())
.Nested<Address>(nomd => nomd.Name(p => p.PrimaryAddress).Properties(MapAddressProperties))
//from MapAddressProperties()
.Text(smd2 => smd2.Name(x => x.Line1).Analyzer(ElasticIndexCreator.SortAnalyzer).Fielddata())
Mappings in elastic:
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
"addresses": {
"line1": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
}
Is there some other, better way to escape a space in an elasticsearch querystring? I've also tried \\ and \\\\ (in C# evaluates to \\) instead of the ? to no avail.

Finally found the correct setup after tons of time experimenting. The configuration that worked for me was as follows:
Use Text with Field Data in the columns
Search using QueryString with wildcard placeholders, replacing spaces with ? e.g. bob smith is entered, query elastic with *bob?smith*
Use Nested queries for child objects. Oddly, addresses.line1 will return data when searching for say 4800 but not when trying to do *4800?street*. Using a nested query allows this to function properly .
From what I hear, having to use field data is very memory intensive, and having to use wildcards is very time intensive, so this is probably not an optimal solution but it's the only one I've found. If there is another better way to do this, please enlighten me.
Example queries in C# using Nest:
var query = Query<Student>.QueryString(qs => qs
.Fields(f => f
.Field(c => c.Name)
//.Field(c => c.PrimaryAddress.Line1) //this doesn't work
)
.Query(testCase.Term)
);
query |= Query<Student>.Nested(n => n
.Path(p => p.Addresses)
.Query(q => q
.QueryString(qs => qs
.Fields(f => f.Field(c => c.Addresses.First().Line1))
.Query(testCase.Term)
)
)
);
Example Mapping:
.Map<Student>(s => s.Properties(p => p
.Text(t => t.Name(z => z.Name).Fielddata())
.Nested<StudentAddress>(n => n
.Name(ap => ap.Addresses)
.Properties(ap => ap.Text(t => t.Name(z => z.Line1).Fielddata())
)
))

Try using addresses.line1.keyword (that is, try the keyword multi-field that you defined for addresses.line1) in the fields parameter a term-level wildcard query:
{
"query": {
"wildcard": {
"addresses.line1.keyword": {
"wildcard": "*4800 street*"
}
}
}
}
Per Elasticsearch documentation on full-text wildcard searches, if you search against addresses.line1 (whose type is text so full-text search rules apply), the search will be performed against each term analyzed out of the field, that is, once against 4800 and again against street, none of which would match your *4800?street* wildcard. The addresses.line1.keyword multi-field contains the original 4800 street value, and should match your query pattern using a term-level wildcard query.
By the way, a minor nit: the mapping type definition itself seems incomplete for the addresses field. You said it is:
"addresses": {
"line1": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
}
But IMHO it should instead be:
"addresses": {
"properties": {
"line1": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
}
}

ElasticSearch C# client (NEST): access nested aggregation with Spaces

Assuming my 2 values are "Red Square" and "Green circle",
when i run the aggregation using Elastic search i get 4 values instead of 2, space separated?
They are Red, Square, Green, circle.
Is there a way to get the 2 original values.
The code is below:
var result = this.client.Search<MyClass>(s => s
.Size(int.MaxValue)
.Aggregations(a => a
.Terms("field1", t => t.Field(k => k.MyField))
)
);
var agBucket = (Bucket)result.Aggregations["field1"];
var myAgg = result.Aggs.Terms("field1");
IList<KeyItem> list = myAgg.Items;
foreach (KeyItem i in list)
{
string data = i.Key;
}

In your mapping, you need to set the field1 string as not_analyzed, like this:
{
"your_type": {
"properties": {
"field1": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
You can also make field1 a multi-field and make it both analyzed and not_analyzed to get the best of both worlds (i.e. text matching on the analyzed field + aggregation on the exact value of the not_analyzed raw sub-field).
{
"your_type": {
"properties": {
"field1": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
If you choose this second option, you'll need to run your aggregation on field1.raw instead of field1.