We Keep Coding

Searching in multiple elasticsearch indexes using NEST

I am trying to search for a text in elasticsearch using nest 7.10.1. I want to search in two different indexes, I get a response in the form of documents, but I cannot access its properties because the result has the combination of two indexes. Below is the code I tried. Both the indexes has same properties. What do I use in the foreach loop to access the key and values of the result documents.
public void searchIndices(string query) {
var response = client.Search<object>(
s => s.Index("knowledgearticles_index, index2")
.Query(q => q.Match(m => m.Field("locationName")
.Query(query))));
Console.WriteLine(response.Documents);
foreach(object r in response.Documents) {
}
}
I am using elasticsearch 7.10.2

Each raw hit coming back in the search response has the _index meta field associated with it:
"hits" : {
"total" : {
"value" : 91,
"relation" : "eq"
},
"hits" : [
{
"_index" : "knowledgearticles_index", <---
"_type" : "_doc",
"_id" : "r_oLl3cBZOT6A8Qby8Qd",
"_score" : 1.0,
"_source" : {
...
}
}
Now, in NEST,
.Documents is a convenient shorthand for retrieving the _source for each hit
-- meaning that you'll have lost access to the meta properties.
So the trick is to use a loop like this instead:
foreach (var hit in response.HitsMetadata.Hits) {
Console.WriteLine(hit);
}

If the two indices that you're searching over contain different JSON structures, then the T in Search<T> will need to be a type that the differing JSON structures can be deserialize to. object will work as per the example in your question, but then there is no typed access for any of the properties.
A simple approach to overcoming this is to hook up the JsonNetSerializer and then use JObject for T
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var connectionSettings =
new ConnectionSettings(pool, sourceSerializer: JsonNetSerializer.Default);
var client = new ElasticClient(connectionSettings);
var response = client.Search<JObject>(s => s
.Index("knowledgearticles_index, index2")
.Query(q => q
.Match(m => m
.Field("locationName")
.Query(query)
)
)
);
We now have a way of accessing properties on JObject, but will still need to handle the type of each property.

How get List of JSON Objects from within a JSON object (non-array) (NOT Deserializing)

I'm using C# with Json.NET NuGet.
I have JSON that looks like this:
{
"pre" : "",
"options": {
"0001" : {
"id" : "0001",
"desc" : "first"
},
"0002" : {
"id" : "0002",
"desc" : "second"
},
"0003" : {
"id" : "0003",
"desc" : "third"
}
},
"post" : ""
}
How can I query the above Json to get a List<JObject> with the 3 option items in it?
Or/Also How can I get just the second item where item 2 should be:
{
"id" : "0002",
"desc" : "second"
}
I've tried stuff like
var items = json.SelectTokens("options[*]").ToList();
and
var item = json.SelectTokens("options[1]");
but those clearly don't work.
EDIT:
In case I wasn't clear I DO NOT want to deserialize. I want a List<JObject>.

Your json is valid but your thinking of this json is not quite accurate.
You are thinking that options have a list of objects that you want to iterate over, but, thats not the case. options is not a list but an object that has more objects within it.. Not an array.
You can access each of the element within the JObject by first looking up its properties. Properties are 0001, 0002 etc. Once you have those, you can iterate over the properties of options and get the values you need.
JObject options = (JObject)JObject.Parse(json)["options"];
// Get a list of all tokens within this object.
List<JObject> allObjects = new List<JObject>();
foreach (var node in options.Properties())
allObjects.Add((JObject)options[node.Name]);
// Access the IDs
allObjects.ForEach(x => Console.WriteLine(x["id"].ToString()));
// Access the 2nd ID only
Console.WriteLine(); // Just to space it out.
Console.WriteLine(allObjects[1]["id"].ToString());
Output
0001
0002
0003
0002

You could create the required List using Linq. For example,
var list = ((JObject)JObject.Parse(str)["options"])
.Properties()
.Select(x=>x.Value)
.Cast<JObject>()
.ToList();
For accessing the second element, you could use
var secondId = (string)list[1]["id"];
var secondDesc = (string)list[1]["desc"];

I was actually close but trying too hard:
var items = json.SelectToken("options").ToList();
I was unable to figure out how to get a single option from a query but since I got the whole list I did it like this:
var item = json.SelectToken("options").ToList()[1];

Add JObject to JArray on nth position

I'm currently working on an ASP.NET Web API .NET Framework 4.7.2. I try to alter some JSON data in my service class. I try to add a new object after every 2nd object in my JArray.
I thought about manipulating the JSON data, rather then concrete objects because the received data will most likely be dynamic data. I'm using the library JObject, but I'm getting some error without any real exception messages.
My received JSON structure looks like that:
{ "data" : [
{"IsPlaceholder": 0, "Name" : "Test1", "Size" : 2 },
{"IsPlaceholder": 0, "Name" : "Test2", "Size" : 3 },
{"IsPlaceholder": 0, "Name" : "Test3", "Size" : 1 }
]}
My service class looks like that:
public class MyService : IMyService
{
public async Task<JObject> UpdateInformationAsync(JObject coolData)
{
// Random placeholder, new placeholder object after 2nd
var placeholder = JObject.FromObject(new PlaceholderVm());
var cnt = 0;
foreach (JObject c in coolData["data"] as JArray)
{
if (cnt % 2 == 0)
{
coolData["data"][cnt].AddAfterSelf(placeholder);
}
cnt++;
}
return coolData;
}
}
My placeholder view model looks like that:
public class PlaceholderVm
{
public int IsPlaceholder => 1;
public string Name => "Placeholder";
public float Size { get; set; } = 0;
}
When I try to add a placeholderVm to my JArray, it works fine the first time, but on the 2nd iteration it throws an error without exception message.
Do you know how I can add a new JObject on nth position to my JArray?

This is because you are mutating the underlying collection while looping through it in the foreach. This is the reason you'll often times see folks initialize a new List<T> when doing operations like this, to avoid this error.
It actually yields this exception :
Run-time exception (line 21): Collection was modified; enumeration operation may not execute.
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
The easiest way around it is to simply create a new collection and place things where you want them to go. In your case, this may look like :
var jObject = new JObject();
JArray jArray = new JArray();
foreach (JObject c in coolData["data"] as JArray)
{
jArray.Add(c);
if (cnt % 2 == 0)
{
jArray[jArray.Count - 1].AddAfterSelf(placeholder);
}
cnt++;
}
jObject.Add("data", jArray);
Here's a .NET Fiddle

Picking Out Simple Properties from Hierarchical JSON

* Despite the edit to my title by another user, I am seeking a solution that uses JSON.NET's library from C# *
A reply containing psuedocode is fine! :)
I'm trying to work with hierarchical data provided by a JSON dataset. I'm using C# and JSON.NET. I'm open to using Linq in general and Linq for JSON.NET in particular if it would help; otherwise, using non-Linq C#/JSON.NET is fine.
Ideally, I am trying to accomplish two things elegantly:
I want to extract JSON that represents each branch and that branch's own properties--not its child (nested) branch objects (I will explain more in a moment).
I want to track the parent node as I create my branch objects.
For further consideration, please refer to the following JSON excerpt:
{
"Branch1": {
"Prop1A" : "1A",
"Prop1B" : "1B",
"Prop1C" : "1C",
"Branch2" : {
"Prop2A" : "2A",
"Prop2B" : "2B",
"Prop2C" : "2C",
"Branch3" : {
"Prop3A" : "3A",
"Prop3B" : "3B",
"Prop3C" : "3C"
}
}
}
}
Related to Goal 1 (from above):
Given JSON that is composed of nested JSON objects, I want to pick out only the simple (string) properties for each branch. For instance, I would like to extract the JSON for Branch1 that would contain only Prop1A, Prop1B, and Prop1C properties. I would then like to extract the JSON for Branch2 that would contain only Prop2A, Prop2B, and Prop2C properties, etc. I realize that I can represent the entire JSON as a JSON.NET JToken object then iterate through its Children() and look only for JTokenType.Property types, but perhaps there is a more elegant way to quickly pick out just the property types using Linq...? In the end, I would have three separate JSON objects that would look like this:
JSON Object 1:
{
"Prop1A" : "1A",
"Prop1B" : "1B",
"Prop1C" : "1C"
}
JSON Object 2:
{
"Prop2A" : "2A",
"Prop2B" : "2B",
"Prop2C" : "2C"
}
JSON Object 3:
{
"Prop3A" : "3A",
"Prop3B" : "3B",
"Prop3C" : "3C"
}
Related to Goal 2 (from above):
Ideally, each extracted JSON above would also have a property indicating its parent. Thus, the final JSON objects would look something like this:
{
"Prop1A" : "1A",
"Prop1B" : "1B",
"Prop1C" : "1C",
"Parent" : ""
}
And:
{
"Prop2A" : "2A",
"Prop2B" : "2B",
"Prop2C" : "2C",
"Parent" : "Branch1"
}
And:
{
"Prop3A" : "3A",
"Prop3B" : "3B",
"Prop3C" : "3C",
"Parent" : "Branch2"
}
Any thoughts?

You can use JContainer.DescendantsAndSelf() to find all objects in the JSON hierarchy, then for each object, loop through its properties and filter out those whose value is a JValue primitive. Thus the following query creates a List<JObject> containing the property names and values you require:
var root = (JContainer)JToken.Parse(jsonString);
var query1 = from o in root.DescendantsAndSelf().OfType<JObject>() // Find objects
let l = o.Properties().Where(p => p.Value is JValue) // Select their primitive properties
where l.Any() // Skip objects with no properties
select new JObject(l); // And return a JObject
var list1 = query1.ToList();
To always skip the root object even if it has primitive properties, use JContainer.Descendants(). And if you really only want string-valued properties (rather than primitive properties), you can check the JToken.Type property:
let l = o.Properties().Where(p => p.Value.Type == JTokenType.String) // Select their string-valued properties
The query can be enhanced to include a synthetic "Parent" property giving the name of the immediate parent property containing the object, using JToken.Ancestors:
var query2 = from o in root.DescendantsAndSelf().OfType<JObject>() // Find objects
let l = o.Properties().Where(p => p.Value is JValue) // Select their primitive properties
where l.Any() // Skip objects with no properties
// Add synthetic "Parent" property
let l2 = l.Concat(new[] { new JProperty("Parent", o.Ancestors().OfType<JProperty>().Select(a => a.Name).FirstOrDefault() ?? "") })
select new JObject(l2); // And return a JObject.
var list2 = query2.ToList();
However, in your desired output you seem to want the property name of the parent of the object, rather than the property name of the object. If so, you can do:
var query3 = from o in root.DescendantsAndSelf().OfType<JObject>() // Find objects
let l = o.Properties().Where(p => p.Value is JValue) // Select their primitive properties
where l.Any() // Skip objects with no properties
// Add synthetic "Parent" property
let l2 = l.Concat(new[] { new JProperty("Parent", o.Ancestors().OfType<JProperty>().Skip(1).Select(a => a.Name).FirstOrDefault() ?? "") })
select new JObject(l2); // And return a JObject.
var list3 = query3.ToList();
For the final query, if I do:
Console.WriteLine(JsonConvert.SerializeObject(list3, Formatting.Indented));
The following output is generated, showing the JObject list has the contents you require:
[
{
"Prop1A": "1A",
"Prop1B": "1B",
"Prop1C": "1C",
"Parent": ""
},
{
"Prop2A": "2A",
"Prop2B": "2B",
"Prop2C": "2C",
"Parent": "Branch1"
},
{
"Prop3A": "3A",
"Prop3B": "3B",
"Prop3C": "3C",
"Parent": "Branch2"
}
]
Note that if the JSON objects themselves have a property named "Parent", the JObject constructor may throw a duplicated key exception.

MongoDB Get names of all keys in collection using c#

How can I get names of all the keys in a MongoDB collection using c#.
I am using mongocsharpdriver.
I am able to get all the records using
var collection1 = database1.GetCollection("Games").FindAll();
Now I need the key names to display/use it. I need key names of collection that I have fetched.
e.g. If I have collection which
{ "_id" : ObjectId("c3"), "GameID" : 20, "GameName" : "COD5", "Cost" : 100}
{ "_id" : ObjectId("c4"), "GameID" : 21, "GameName" : "NFS", "Publisher" : "EA"}
{ "_id" : ObjectId("c5"), "GameID" : 22, "GameName" : "CS", "Cost" : 200}
So I should get list of keys like GameID, GameName, Cost, Publisher.
I also went through MongoDB Get names of all keys in collection but was not able to implement it, didnot understood it & got problem with mapreduce.

Inspired from the link in your question:
string map = #"function() {
for (var key in this) { emit(key, null); }
}";
string reduce = #"function(key, stuff) { return null; }";
string finalize = #"function(key, value){
return key;
}";
MapReduceArgs args = new MapReduceArgs();
args.FinalizeFunction = new BsonJavaScript(finalize);
args.MapFunction = new BsonJavaScript(map);
args.ReduceFunction = new BsonJavaScript(reduce);
var results = collection1.MapReduce(args);
foreach (BsonValue result in results.GetResults().Select(item => item["_id"]))
{
Console.WriteLine(result.AsString);
}

A very inefficient, but simple way to do this is
HashSet<string> keys = new HashSet<string>();
foreach (var rover in collection1.FindAll())
{
rover.Names.ToList().ForEach(p => keys.Add(p));
}
Keep in mind that finding the set of keys, no matter how it's implemented, will always have to iterate the entire collection, so it will be terribly slow on larger collections.
It makes sense to use Map/Reduce for this problem on larger collections, because that avoids all the data transfer and deserialization overhead that is incurred by the solution I posted above, but you should generally try to avoid doing something like this at all. At least, don't do it where it's needed synchronously.
If you somehow need to know the set of all fields quickly, you're better off keeping track of the fields during writes and store the list of fields in a separate collection somewhere.