We Keep Coding

Searching in multiple elasticsearch indexes using NEST

I am trying to search for a text in elasticsearch using nest 7.10.1. I want to search in two different indexes, I get a response in the form of documents, but I cannot access its properties because the result has the combination of two indexes. Below is the code I tried. Both the indexes has same properties. What do I use in the foreach loop to access the key and values of the result documents.
public void searchIndices(string query) {
var response = client.Search<object>(
s => s.Index("knowledgearticles_index, index2")
.Query(q => q.Match(m => m.Field("locationName")
.Query(query))));
Console.WriteLine(response.Documents);
foreach(object r in response.Documents) {
}
}
I am using elasticsearch 7.10.2

Each raw hit coming back in the search response has the _index meta field associated with it:
"hits" : {
"total" : {
"value" : 91,
"relation" : "eq"
},
"hits" : [
{
"_index" : "knowledgearticles_index", <---
"_type" : "_doc",
"_id" : "r_oLl3cBZOT6A8Qby8Qd",
"_score" : 1.0,
"_source" : {
...
}
}
Now, in NEST,
.Documents is a convenient shorthand for retrieving the _source for each hit
-- meaning that you'll have lost access to the meta properties.
So the trick is to use a loop like this instead:
foreach (var hit in response.HitsMetadata.Hits) {
Console.WriteLine(hit);
}

If the two indices that you're searching over contain different JSON structures, then the T in Search<T> will need to be a type that the differing JSON structures can be deserialize to. object will work as per the example in your question, but then there is no typed access for any of the properties.
A simple approach to overcoming this is to hook up the JsonNetSerializer and then use JObject for T
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var connectionSettings =
new ConnectionSettings(pool, sourceSerializer: JsonNetSerializer.Default);
var client = new ElasticClient(connectionSettings);
var response = client.Search<JObject>(s => s
.Index("knowledgearticles_index, index2")
.Query(q => q
.Match(m => m
.Field("locationName")
.Query(query)
)
)
);
We now have a way of accessing properties on JObject, but will still need to handle the type of each property.

Execute complex mongo JSON query using C# mongo driver

I have following mongo shell query:
{'field':'FieldOne','value':'FieldOne','category':'categoryOne'} , { 'Color' : { '$elemMatch' : { 'Value' : 'Green' } }}
Is there any way to execute same query using C# mongo driver?. I have tried using below C# code, but only the first one is getting executed:
BsonDocument query = BsonDocument.Parse("{'field':'Overall','value':'Overall','category':'LoggedIncidents'} , { 'Priority' : { '$elemMatch' : { 'Value' : 'P1' } }}");
QueryDocument queryDoc = new QueryDocument(query);
var result = collection.Find(queryDoc).ToListAsync().Result;
The first item ({'field':'FieldOne','value':'FieldOne','category':'categoryOne'}) is being executed but not the second one ({ 'Color' : { '$elemMatch' : { 'Value' : 'Green' } }}).

You are using it in find command if i'm not wrong.
the first "{}" is what the filter condition for find is. next set of parameters are essentially to show/hide the fields you need.
so if you want to filter by value of array "green" you can do this.
db.objects.find({"color":{$elemMatch:{value:"green"}}})
should give you the results you are looking for. but if you want to specifically show/hide fields, you can do
db.objects.find({"color":{$elemMatch:{value:"green"}}},{"field":1})
is this what you want to attain?

How to make an accent insensitive search in elasticsearch with nest c# client?

I'm an elasticsearch newbie.
Lets say we have a class like this:
public class A
{
public string name;
}
And we have 2 documents which have names like "Ayşe" and "Ayse".
Now, I want to be able to store names with their accents but when I search want to be able take results of accent insensitive query as accent sensitive results.
For ex: When I search for "Ayse" or "Ayşe", it should return both "Ayşe" and "Ayse" as they stored (with accent).
Right now when I search for "Ayse" it only returns "Ayse" but I want to have "Ayşe" as a result too.
When I checked elasticsearch documentation, I see that folded properties is needed to be used to achive that. But I couldn't understand how to do it with Nest attributes / functions.
BTW I'm using AutoMap to create mappings right now and if it is possible I want to be able to continue to use it.
I'm searching for an answer for 2 days right now and couldn't figure it out yet.
What/where changes are required? Can you provide me code sample(s)?
Thank you.
EDIT 1:
I figured out how to use analyzers to create sub fields of a property and achive results with term based query against sub fields.
Now, I know I can do a multi field search but is there a way to include sub fields with full text search?
Thank you.

You can configure an analyzer to perform analysis on the text at index time, index this into a multi_field to use at query time, as well as keep the original source to return in the result. Based on what you have in your question, it sounds like you want a custom analyzer that uses the asciifolding token filter to convert to ASCII characters at index and search time.
Given the following document
public class Document
{
public int Id { get; set;}
public string Name { get; set; }
}
Setting up a custom analyzer can be done when an index is created; we can also specify the mapping at the same time
client.CreateIndex(documentsIndex, ci => ci
.Settings(s => s
.NumberOfShards(1)
.NumberOfReplicas(0)
.Analysis(analysis => analysis
.TokenFilters(tokenfilters => tokenfilters
.AsciiFolding("folding-preserve", ft => ft
.PreserveOriginal()
)
)
.Analyzers(analyzers => analyzers
.Custom("folding-analyzer", c => c
.Tokenizer("standard")
.Filters("standard", "folding-preserve")
)
)
)
)
.Mappings(m => m
.Map<Document>(mm => mm
.AutoMap()
.Properties(p => p
.String(s => s
.Name(n => n.Name)
.Fields(f => f
.String(ss => ss
.Name("folding")
.Analyzer("folding-analyzer")
)
)
.NotAnalyzed()
)
)
)
)
);
Here I've created an index with one shard and no replicas (you may want to change this for your environment), and have created a custom analyzer, folding-analyzer that uses the standard tokenizer in conjunction with the standard token filter and a folding-preserve token filter that perform ascii folding, storing the original tokens in addition to the folded tokens (more on why this may be useful in a minute).
I've also mapped the Document type, mapping the Name property as a multi_field, with default field not_analyzed (useful for aggregations) and a .folding sub-field that will be analyzed with the folding-analyzer. The original source document will also be stored by Elasticsearch by default.
Now let's index some documents
client.Index<Document>(new Document { Id = 1, Name = "Ayse" });
client.Index<Document>(new Document { Id = 2, Name = "Ayşe" });
// refresh the index after indexing to ensure the documents just indexed are
// available to be searched
client.Refresh(documentsIndex);
Finally, searching for Ayşe
var response = client.Search<Document>(s => s
.Query(q => q
.QueryString(qs => qs
.Fields(f => f
.Field(c => c.Name.Suffix("folding"))
)
.Query("Ayşe")
)
)
);
yields
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.163388,
"hits" : [ {
"_index" : "documents",
"_type" : "document",
"_id" : "2",
"_score" : 1.163388,
"_source" : {
"id" : 2,
"name" : "Ayşe"
}
}, {
"_index" : "documents",
"_type" : "document",
"_id" : "1",
"_score" : 0.3038296,
"_source" : {
"id" : 1,
"name" : "Ayse"
}
} ]
}
}
Two things to highlight here:
Firstly, the _source contains the original text that was sent to Elasticsearch so by using response.Documents, you will get the original names, for example
string.Join(",", response.Documents.Select(d => d.Name));
would give you "Ayşe,Ayse"
Secondly, remember that we preserved the original tokens in the asciifolding token filter? Doing so means that we can perform queries that undergo analysis to match accent insensitively but also take into account accent sensitivity when it comes to scoring; in the example above, the score for Ayşe matching Ayşe is higher than for Ayse matching Ayşe because the tokens Ayşe and Ayse are indexed for the former whilst only Ayse is indexed for the latter. When a query that undergoes analysis is performed against the Name property, the query is analyzed with the folding-analyzer and a search for matches is performed
Index time
----------
document 1 name: Ayse --analysis--> Ayse
document 2 name: Ayşe --analysis--> Ayşe, Ayse
Query time
-----------
query_string query input: Ayşe --analysis--> Ayşe, Ayse
search for documents with tokens for name field matching Ayşe or Ayse

How to implement MongoDB nested $elemMatch Query in C#

I have a MongoDB collection in the following format.
{
"_id" : ObjectId("56c6f03ffd07dc1de805e84f"),
"Details" : {
"a" : [
[ {
"DeviceID" : "log0",
"DeviceName" : "Dev0"
},
{
"DeviceID" : "log1",
"DeviceName" : "Dev1"
}
],
[ {
"DeviceID" : "Model0",
"DeviceName" : "ModelName0"
},
{
"DeviceID" : "Model1",
"DeviceName" : "ModelName1"
}
]
]
}
}
And I am trying to fetch all the documents where the DeviceName in array "a" contains a particular value, say "Name0". However I could get the desired result while using below Mongo query:
db.test_collection.find({"Details.a":{$elemMatch:{$elemMatch:{DeviceName : /.*Name0.*/}}}});
Now I am struggling to implement the above query in C#. Can anyone guide me with that?
so far I have tried the below code and it was not working as expected
query = Query.And(Query.ElemMatch("Details.a", Query.And(Query.ElemMatch("DeviceName", Query.Matches("DeviceName", new BsonRegularExpression("Name0"))))));
Thanks in advance

Well, honestly writing queries in C# are bit tricky but you can always play a trick.
var bsonQuery = "{'Details.a':{$elemMatch:{$elemMatch:{DeviceName : /.*Name0.*/}}}}";
var filter = MongoDB.Bson.Serialization.BsonSerializer.Deserialize<BsonDocument>(bsonQuery);
var result = col.FindSync (filter).ToList();
I'm deserializing a plain MongoDB queries into a BsonDocument which in return I'm passing to FindAsync as filter.
In the end, you'll have desired outcome in variable result.
Note: I'm assuming MongoDB connection has been established and variable col holds reference to MongoDB collection.
EDIT: Please see following link https://groups.google.com/forum/#!topic/mongodb-csharp/0dcoVlbFR2A. Now it's confirmed that C# driver doesn't support nameless filters so writing above query using Buidlers<BsonDocument>.Filter at moment is not supported.
Long story short, you are left with only one choice and that is to query as I mentioned above in my solution.

MongoDB/C# Update Collection entries

Hello I have a mongoDB Collection called Nodes whose structure is as follows:
{
"_id" : new BinData(3, "Ljot2wCBskWG9VobsLA0zQ=="),
"logicalname" : "Test Node",
"host" : "eh5tmb054pc",
"port" : 104,
"appendtimestamp" : false,
"studies" : ["1.3.12.2.1107"],
"tests" : [],
"mainentries" : [{
"_id" : "1.3.12.2.1107",
"Index" : 0,
"Type" : "Study"
}]
}
I created a new key called "mainentries" which is now storing the "studies" and "tests". So in order to support my new versions without hassle, I now want to write a method in my Setup Helper, which would enable me to read this collection - Check if studies,tests exists , If yes add the key "mainentries" and remove the studies/tests key.
My question is: What kind of query must I use to reach each collection of Nodes to check for the fields and update. I am using the MongoDB-CSharp community driver.
Would appreciate any help and pointers.

You can simply check whether the field(s) still exist(s):
var collection = db.GetCollection<Node>("nodes");
var nodes = collection.Find(Query.And( // might want Query.Or instead?
Query<Node>.Exists(p => p.Tests),
Query<Node>.Exists(p => p.Studies)).SetSnapshot();
foreach(var node in nodes) {
// maybe you want to move the Tests and Studies to MainEntries here?
node.MainEntries = new List<MainEntries>();
node.Test = null;
node.Studies = null;
collection.Update(node);
}
If you don't want to migrate the data, but just remove the fields and create new ones, you can also do in a simple batch update using $exists, $set and $remove