I want to run a MultiGet (mget) search query on several IDs across two indexes. This is because I have two indexes, but I don't know which index contains my ID. This is the query:
GET _mget
{
"docs" : [
{
"_id": "id1",
"_index": "index1"
},
{
"_id": "id1",
"_index": "index2"
}
/* .... */
]
}
The query works great manually - I get the results and I just ignore the result that returns found: false.
Nest does not support this functionality, only on one index. So I tried to use the low-level client to achieve this, like so:
var data = PostData.Serializable(new
{
docs = new[]
{
new {
_id = "1",
_index = "index1"
},
new
{
_id = "1",
_index = "index2"
}
}
});
var response = await lowLevelClient.MultiGetAsync<MultiGetResponse>(data);
However, I'm getting the following exception: Elasticsearch.Net.UnexpectedElasticsearchClientException: 'Constructor on type 'Nest.MultiGetResponseFormatter' not found.'.
Is this the right way to achieve what I want?
Following will help you achieve what you are looking for with NEST
var request = new MultiGetRequest();
request.Documents = new IMultiGetOperation[]
{
new MultiGetOperation<object>("id1") { Index = "index1" },
new MultiGetOperation<object>("id1") { Index = "index2" },
};
var multiGetResponse = await client.MultiGetAsync(request);
Related
I have the following document structure:
{
"Agencies": [
{
"name": "tcs",
"id": "1",
"AgencyUser": [
{
"UserName": "ABC",
"Code": "ABC40",
"Link": "http.ios.com",
"TotalDownloads": 0
},
{
"UserName": "xyz",
"Code": "xyz20",
"Link": "http.ios.com",
"TotalDownloads": 0
}
]
}
]
}
Like I have multiple agencies and each agency contains a list of agents.
What I am trying is to pass the Code and update the TotalDownloads field of the agent that matches the code.
For example, if someone uses the code ABC40 so I need to update the field TotalDownloads of the agent called "ABC".
What I have tried is as below:
public virtual async Task UpdateAgentUsersDownloadByCode(string Code)
{
var col = _db.GetCollection<Agencies>(Agencies.DocumentName);
FilterDefinition<Agencies> filter = Builders<Agencies>.Filter.Eq("AgencyUsers.Code", Code);
UpdateDefinition<Agencies> update = Builders<Agencies>.Update.Inc(x => x.AgencyUsers.FirstOrDefault().TotalDownloads, 1);
await col.UpdateOneAsync(filter, update);
}
It is giving me the following error:
Unable to determine the serialization information for x => x.AgencyUsers.FirstOrDefault().TotalDownloads.
Where I'm wrong?
Note: From the attached sample document, the array property name: AgencyUser is not matched with the property name that you specified in the update operation, AgencyUsers.
Use arrayFilters with $[<identifier>] positional filtered operator to update the element(s) in the array.
MongoDB syntax
db.Agencies.update({
"AgencyUsers.Code": "ABC40"
},
{
$inc: {
"AgencyUsers.$[agencyUser].TotalDownloads": 1
}
},
{
arrayFilters: [
{
"agencyUser.Code": "ABC40"
}
]
})
Demo # Mongo Playground
MongoDB .NET Driver syntax
UpdateDefinition<Agencies> update = Builders<Agencies>.Update
.Inc("AgencyUsers.$[agencyUser].TotalDownloads", 1);
UpdateOptions updateOptions = new UpdateOptions
{
ArrayFilters = new[]
{
new BsonDocumentArrayFilterDefinition<Agencies>(
new BsonDocument("agencyUser.Code", Code)
)
}
};
UpdateResult result = await col.UpdateOneAsync(filter, update, updateOptions);
Demo
I have a collection like below
{
"id":"5fd13c33ac0277c435117519",
"content":"test",
"votes":[
{
"user":"22",
"isLike":true
},
{
"user":"25",
"isLike":false
},
{
"user":"43",
"isLike":false
}
]
},
{
"id":"5fd13c33ac0277c435237443",
"content":"test 2",
"votes":[
{
"user":"25",
"isLike":true
},
{
"user":"43",
"isLike":false
}
]
}
How can I get the result below with c# driver by querying over votes.user -ie. 25- and then merge those 2 fields from parent and nested array?
{
"id":"5fd13c33ac0277c435117519",
"isLike":false
},
{
"id":"5fd13c33ac0277c435237443",
"isLike":true
}
EDIT: I've got the result after some try and fails on mongoplayground.net, stil not sure how to convert it over c# driver.
db.collection.aggregate([
{
$unwind: "$votes"
},
{
$match: {
"votes.user": "25"
}
},
{
$replaceWith: {
id: "$id",
isLike: "$votes.isLike"
}
}
])
I've managed it like below and wanted to share if someone needs it in the future.
I personally decided to go one-to-many path with separated vote collection after some readings.
var match = new BsonDocument("votes.user", 25);
var replace = new BsonDocument(new List<BsonElement> {
new BsonElement("id", "$id"),
new BsonElement("isLike", "$votes.isLike"),
});
return await _collection.Aggregate()
.Unwind(c => c.votes)
.Match(match)
.ReplaceWith<UserVote>(replace)
.ToListAsync();
I have a document like:
{
"Id":"1",
"Name":"product1",
"Categories":["Cat1",
"Cat2",
"Cat3"]
},
{
"Id":"2",
"Name":"product2",
"Categories":["Cat3",
"Cat2",
"Cat6"]
}
Now I want return a distinct List of all categories.
I tried CollapseParameter, but it doesn't work.
var foo = _solr.Query(new SolrQuery("*"), new QueryOptions
{
Collapse = new CollapseParameters("Categories")
});
var bar= results.CollapseExpand; //NULL
How can I get a list of all categories without iterating through all the documents? Do I have to create new Documents for the categories? Or should I work with Faceting?
It looks like the JSON is not valid. So I created a solution changing the Json:
{
"items": [{
"Id": "1",
"Name": "product1",
"Categories": ["Cat1", "Cat2", "Cat3"]
}, {
"Id": "2",
"Name": "product2",
"Categories": ["Cat3", "Cat2", "Cat6"]
}]
}
And then, I used foreach and LINQ structure:
var x = categories.SelectMany(item => item.Categories).Distinct();
If you are using solr and you want all distinct value for specific field then must start reading solr faceting doc, its amazing.
Only add below query in your query and you we get full list of all distinct categories.
&facet=true&facet.field=Categories
That's it, It will gives result set as you expected.
QueryOptions options = new QueryOptions
{
StartOrCursor = new StartOrCursor.Start(0),
Rows = 1,
Facet = new FacetParameters { Queries = new[] { new SolrFacetFieldQuery("Categories") } }
};
var result = _solrConnection.Query(new SolrQuery("*"), options);
if (result.FacetFields.ContainsKey("Categories"))
{
return result.FacetFields["Categories"]
.Where(li => li.Value > 0)
.Select(y=> y.Key)
.ToList();
}
That's my current solution. It works, but I hoped there would be a better way. I'm searching for a result just to get all the categories.
I am setting up a ChangeStream to notify me when a document has changed in a collection so that I can upsert the "LastModified" element for that document to the time of the event. Since this update will cause a new event to occur on the ChangeStream, I need to filter out these updates to prevent an infinite loop (updating the LastModified element because the LastModified element was just updated...).
I have the following code that is working when I specify the exact field:
ChangeStreamOptions options = new ChangeStreamOptions();
options.ResumeAfter = resumeToken;
string filter = "{ $and: [ { operationType: { $in: ['replace','insert','update'] } }, { 'updateDescription.updatedFields.LastModified': { $exists: false } } ] }";
var pipeline = new EmptyPipelineDefinition<ChangeStreamDocument<BsonDocument>>().Match(filter);
var cursor = collection.Watch(pipeline, options, cancelToken);
However, instead of hard-coding the "updateDescription.updatedFields.LastModified", I would like to provide a list of element names that I don't want to exist in the updatedFields document.
I attempted:
string filter = "{ $and: [ { operationType: { $in: ['replace','insert','update'] } }, { 'updateDescription.updatedFields': { $nin: [ 'LastModified' ] } } ] }";
but this didn't work as expected (I still got the update events for the LastModified change.
I originally was using the Filter Builder:
FilterDefinitionBuilder<ChangeStreamDocument<BsonDocument>> filterBuilder = Builders<ChangeStreamDocument<BsonDocument>>.Filter;
FilterDefinition<ChangeStreamDocument<BsonDocument>> filter = filterBuilder.In("operationType", new string[] { "replace", "insert", "update" }); //Only include the change if it was one of these types. Available types are: insert, update, replace, delete, invalidate
filter &= filterBuilder.Nin("updateDescription.updatedFields", ChangedFieldsToIgnore); //If this is an update, only include it if the field(s) updated contains 1+ fields not in the ChangedFieldsToIgnore list
where ChangedFieldsToIgnore is a List containing the field names that I do not want to get events for.
Can anyone help with the syntax that I need to use? or do I need to create a loop around my ChangedFieldsToIgnore list and create a new entry in the filter for each item to "$exists: false"? (this doesn't seem very efficient).
EDIT:
I attempted the following code based on the answer by #wan-bachtiar, but I'm getting an exception on my enumerator.MoveNext() call:
var match1 = new BsonDocument { { "$match", new BsonDocument { { "operationType", new BsonDocument { { "$in", new BsonArray(new string[] { "replace", "insert", "update" }) } } } } } };
var match2 = new BsonDocument { { "$addFields", new BsonDocument { { "tmpfields", new BsonDocument { { "$objectToArray", "$updateDescription.updatedFields" } } } } } };
var match3 = new BsonDocument { { "$match", new BsonDocument { { "tmpfields.k", new BsonDocument { { "$nin", new BsonArray(updatedFieldsToIgnore) } } } } } };
var pipeline = new[] { match1, match2, match3 };
var cursor = collection.Watch<ChangeStreamDocument<BsonDocument>>(pipeline, options, Profile.CancellationToken);
enumerator = cursor.ToEnumerable().GetEnumerator();
enumerator.MoveNext();
ChangeStreamDocument<BsonDocument> doc = enumerator.Current;
The exception is: "{"Invalid field name: \"tmpfields\"."}"
I suspect the problem might be that I'm getting "replace" and "insert" events which do not contain the updateDescription field, so the $addFields/$objectToArray are failing. I'm too new to figure out the syntax, but I think I need to use a filter that does:
{ $match: { "operationType": { $in: ["replace", "insert"] } } }
OR
{ $eq: { "operationTYpe": "update" }} AND { $addFields....}
Also, it appears that the C# driver does not include a Builder that helps with the $addFields and $objectToArray operations. I was only able to use the new BsonDocument {...} method to build the pipeline variable.
ChangedFieldsToIgnore is a List containing the field names that I do not want to get events for.
If you would like to filter based on multiple keys (whether updatedFields contains certain fields), it's easier if you convert the keys to values first.
You can convert the document contained within updatedFields into values by utilising aggregation operator $objectToArray. For example:
pipeline = [{"$addFields": {
"tmpfields":{
"$objectToArray":"$updateDescription.updatedFields"}
}},
{"$match":{"tmpfields.k":{
"$nin":["LastModified", "AnotherUnwantedField"]}}}
];
The above aggregation pipeline adds a temporary field called tmpfields. This new field will pivot the content of updateDescription.updatedFields turning {name:value} into [{k:name, v:value}]. Once we have those keys as values, we can utilise $nin as an array of filter.
UPDATED
The reason you're getting an exception of tmpfields being invalid, is because the result is casted into ChangeStreamDocument model which does not have a recognizable field called tmpfields.
In the case, when it's different operations that does not have field updateDescription.updatedFields, the value of tmpfields would just be null.
Below is an example of MongoDB ChangeStream .Net/C# using MongoDB .Net driver v2.5, along with an aggregation pipeline that modifies the output change stream.
This example is not type safe, and would return BsonDocument :
var database = client.GetDatabase("database");
var collection = database.GetCollection<BsonDocument>("collection");
var options = new ChangeStreamOptions { FullDocument = ChangeStreamFullDocumentOption.UpdateLookup };
// Aggregation Pipeline
var addFields = new BsonDocument {
{ "$addFields", new BsonDocument {
{ "tmpfields", new BsonDocument {
{ "$objectToArray",
"$updateDescription.updatedFields" }
} }
} } };
var match = new BsonDocument {
{ "$match", new BsonDocument {
{ "tmpfields.k", new BsonDocument {
{ "$nin", new BsonArray{"LastModified", "Unwanted"} }
} } } } };
var pipeline = new[] { addFields, match };
// ChangeStreams
var cursor = collection.Watch<BsonDocument>(pipeline, options);
foreach (var change in cursor.ToEnumerable())
{
Console.WriteLine(change.ToJson());
}
I wrote the piece of code below as I was having the same issues you were having. No need to mess around with BsonObjects ...
//The operationType can be one of the following: insert, update, replace, delete, invalidate
//ignore the field lastrun as we would end in an endles loop
var pipeline = new EmptyPipelineDefinition<ChangeStreamDocument<ATask>>()
.Match("{ operationType: { $in: [ 'replace', 'update' ] } }")
.Match(#"{ ""updateDescription.updatedFields.LastRun"" : { $exists: false } }")
.Match(#"{ ""updateDescription.updatedFields.IsRunning"" : { $exists: false } }");
var options = new ChangeStreamOptions { FullDocument = ChangeStreamFullDocumentOption.UpdateLookup };
var changeStream = Collection.Watch(pipeline, options);
while (changeStream.MoveNext())
{
var next = changeStream.Current;
foreach (var obj in next)
yield return obj.FullDocument;
}
I am building the components of an elastic search query dynamically, the query contains potentially multiple aggregations.
.Net 4.5.2 Nest 2.3.1 Elasticsearch.Net 2.3.1
I can successfully add multiple aggregations by repeating the following structure:
var aggregations = new AggregationDictionary();
aggregations["yyy"] = new AggregationContainer {
Terms = new TermsAggregation("xxx")
{
Field = "afield"
}
};
And then setting the Aggregrations property on the search to the aggregations variable. And all is good.
I can successfully created a single nested aggregration as follows:
var aggregations=new NestedAggregation("Countries") {
Path = "MetaData.GeographicCoverage.Countries",
Aggregations =
new TermsAggregation("Country") {
Field = "MetaData.GeographicCoverage.Countries.Country"
}
};
And again setting the Aggregrations property on the search to the aggregations variable and all is good.
The problem comes when I combine the two approaches to create a query with many aggregrations where one (or more) of which is nested. So the Json generated by the above nested example looks like:
{
"size": 0,
"aggs": {
"Countries": {
"nested": {
"path": "MetaData.GeographicCoverage.Countries"
},
"aggs": {
"Country": {
"terms": {
"field": "MetaData.GeographicCoverage.Countries.Country"
}
}
}
}
}
}
Now when I combine the approaches so that the nested aggregration is added just like those in the very first code snippet:
var aggregations = new AggregationDictionary();
var nested = new NestedAggregation("Countries") {
Path = "MetaData.GeographicCoverage.Countries",
Aggregations =
new TermsAggregation("Country") {
Field = "MetaData.GeographicCoverage.Countries.Country"
}
};
aggregations["Countries"] = new AggregationContainer {
Nested = nested
};
Then the Json of the query that is generated misses the actual "Country" aggregration:
{
"size": 0,
"aggs": {
"Countries": {
"nested": {
"path": "MetaData.GeographicCoverage.Countries"
}
}
}
}
So then - is this a bug or am I using the Nest classes incorrectly? If I am using the classes incorrectly how do I fix the code?
Thanks for any help.
Turns out the way around the problem is to add an explicit cast of the NestedAggregration to an AggregationContainer and then to add it directly to the AggregationsDictionary.
var aggregations = new AggregationDictionary();
var nested =
new NestedAggregation("Countries") {
Path = "MetaData.GeographicCoverage.Countries",
Aggregations =
new TermsAggregation("Country") {
Field = "MetaData.GeographicCoverage.Countries.Country"
}
};
aggregations["Countries"] = (AggregationContainer)nested;