I'm trying to pull back records based on a query and sort them using the _uid field. In my case the _uid is the Type followed by # followed by the id that I set. My index is full of files of code and an example of the _uid would be myType#MyDocuments/File.txt
So I'm doing a sort on _uid ascending. It Mostly works, it sorts the types into order but within the types it only sorts correctly with the upper most directory.
So I'll see something like
Accounting/AP_ABC.asp
Accounting/AR_ABC.asp
Accounting/Account.asp
Which isn't right because Account should come before AP and AR.
Is there a way to make sure this would sort correctly?
EDIT
Adding a mapping from my index
"dotnet":{"properties":{"fileContents":{"type":"string"},"filePath":{"type":"string"},"lastUpdate":{"type":"date","format":"dateOptionalTime"},"type":{"type":"string"}}}
Create a new not_analyzed field like sortid which will hold not analyzed values of your ids(Accounting/Account.asp). This article will explain in details why would you like to do this.
UPDATE:
Try to apply case-insensitive sorting.
Later on I'll update my answer with an working example.
UPDATE2
The easiest way to achievie what you are trying do is to create
index with following mapping:
client.CreateIndex(descriptor => descriptor
.Index(indexName)
.AddMapping<Document>(m => m
.Properties(p => p
.String(s => s.Name(n => n.Id).Index(FieldIndexOption.NotAnalyzed)))));
class Document
{
public string Id { get; set; }
}
Index some documents with lowercase id values:
client.Index(new Document {Id = "Accounting/AP_ABC.asp".ToLower()});
client.Index(new Document {Id = "Accounting/AR_ABC.asp".ToLower()});
client.Index(new Document {Id = "Accounting/Account.asp".ToLower()});
Then for this sorting
var searchResponse = client.Search<Document>(s => s
.Sort(sort => sort
.OnField(f => f.Id).Ascending()));
we will get
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 3,
"max_score": null,
"hits": [
{
"_index": "indexname",
"_type": "document",
"_id": "accounting/account.asp",
"_score": null,
"_source": {
"id": "accounting/account.asp"
},
"sort": [
"accounting/account.asp"
]
},
{
"_index": "indexname",
"_type": "document",
"_id": "accounting/ap_abc.asp",
"_score": null,
"_source": {
"id": "accounting/ap_abc.asp"
},
"sort": [
"accounting/ap_abc.asp"
]
},
{
"_index": "indexname",
"_type": "document",
"_id": "accounting/ar_abc.asp",
"_score": null,
"_source": {
"id": "accounting/ar_abc.asp"
},
"sort": [
"accounting/ar_abc.asp"
]
}
]
}
}
But if you really care about Ids as you provided(e.g
Accounting/AP_ABC.asp) you can use mentioned earlier
Case-Insensitive Sorting.
To apply this solution with NEST:
Create mapping as below
client.CreateIndex(descriptor => descriptor
.Index(indexName)
.Analysis(analysisDescriptor => analysisDescriptor
.Analyzers(a => a
.Add("case_insensitive_sort", new CustomAnalyzer
{
Tokenizer = "keyword",
Filter = new List<string> {"lowercase"}
})))
.AddMapping<Document>(m => m
.Properties(p => p
.String(s => s
.Name(n => n.Id)
.Analyzer("case_insensitive_sort")))));
Index documents:
client.Index(new Document {Id = "Accounting/AP_ABC.asp"});
client.Index(new Document {Id = "Accounting/AR_ABC.asp"});
client.Index(new Document {Id = "Accounting/Account.asp"});
And for sorting we will sort we will get following result
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 3,
"max_score": null,
"hits": [
{
"_index": "indexname",
"_type": "document",
"_id": "Accounting/Account.asp",
"_score": null,
"_source": {
"id": "Accounting/Account.asp"
},
"sort": [
"accounting/account.asp"
]
},
{
"_index": "indexname",
"_type": "document",
"_id": "Accounting/AP_ABC.asp",
"_score": null,
"_source": {
"id": "Accounting/AP_ABC.asp"
},
"sort": [
"accounting/ap_abc.asp"
]
},
{
"_index": "indexname",
"_type": "document",
"_id": "Accounting/AR_ABC.asp",
"_score": null,
"_source": {
"id": "Accounting/AR_ABC.asp"
},
"sort": [
"accounting/ar_abc.asp"
]
}
]
}
}
Hope it will help.
Related
I have documents like this:
[
// 1
{
"_id": ObjectId("573f3944a75c951d4d6aa65e"),
"Source": "IGN",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Japan",
}
]
}
]
},
// 2
{
"_id": ObjectId("573f3d41a75c951d4d6aa65f"),
"Source": "VG",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "USA"
}
]
}
]
},
// 3
{
"_id": ObjectId("573f4367a75c951d4d6aa660"),
"Source": "NRK",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Germany"
}
]
}
]
},
// 4
{
"_id": ObjectId("573f4571a75c951d4d6aa661"),
"Source": "VG",
"Family": [
{
"Countries": [
{
"uid": 10,
"name": "France"
}
]
}
]
},
// 5
{
"_id": ObjectId("573f468da75c951d4d6aa662"),
"Source": "IGN",
"Family": [
{
"Countries": [
{
"uid": 14,
"name": "England"
}
]
}
]
}
]
I want to return only the documents with source equals 'Countries.uid' equal 17
so I have in the end :
[
{
"_id": ObjectId("573f3944a75c951d4d6aa65e"),
"Source": "IGN",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Japan",
}
]
}
]
},
{
"_id": ObjectId("573f3d41a75c951d4d6aa65f"),
"Source": "VG",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "USA"
}
]
}
]
},
{
"_id": ObjectId("573f4367a75c951d4d6aa660"),
"Source": "NRK",
"Family": [
{
"Countries": [
{
"uid": 17,
"name": "Germany"
}
]
}
]
}
]
How can I do this with the official C# MongoDB driver?
Tried this :
public List<Example> getLinkedCountry(string porduitId)
{
var filter = Builders<Example>.Filter.AnyIn("Family.Countries.uid", porduitId);
var cursor = await _certificats.FindAsync(filter);
var docs = cursor.ToList();
return docs;
}
Unfortunately, I think my filter is wrong.
Is there a way to find all the documents by accessing the nested list by id and retrieving it?
Solution 1
Use ElemMatch instead of AnyIn.
var filter = Builders<Example>.Filter.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId));
Output
Solution 2
If you are unconfident with MongoDB .Net Driver syntax, you can convert the query as BsonDocument via MongoDB Compass (Export to language feature).
var filter = new BsonDocument("Family.Countries.uid", porduitId);
Just to expand on #Yong Shun 's answer,
if you just want to return the list of nested documents and not all of it, you have a few options.
Using project
var filter = Builders<Example>.Filter.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId));
var project = Builders<Example>.Project.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId)
);
var examples = await collection.filter(filter).Project<Example>(project).toListAsync();
Using the aggregate pipeline
var filter = Builders<Example>.Filter.ElemMatch(
x => x.Family,
y => y.Countries.Any(z => z.uid == porduitId));
var project = Builders<ServiceProvider>.Projection.Expression(
x => x.Faimily.Where(y => y.uid == porduitId)
);
var result = await collection
.Aggregate()
.Match(filter)
.Project(project)
.ToListAsync(); //Here result is a list of Iterable<Countries>
Elastic search noob question.
After reading some questions on SO I figured out to POST some data to my Index but I'm having some trouble getting back results.
I'm getting one document (as expected) but all properties are empty. The number of bytes 740 match the number of bytes received in Postman. I suspect the data isn't mapped right somehow..
Code:
var client = CreateCloudClient();
var indexName = "index-b";
var mustClauses = new List<QueryContainer>();
mustClauses.Add(new WildcardQuery
{
Field = new Field("FirstName"),
Value = "*ralf*"
});
var searchRequest = new SearchRequest<ProfileEntity>(indexName)
{
Query = new BoolQuery { Must = mustClauses }
};
var searchResponse = await client.SearchAsync<ProfileEntity>(searchRequest);
if (searchResponse.Hits.Any())
{
var person = searchResponse.Hits.First();
}
Debugger result
When using Postman:
GET ../westeurope.azure.elastic-cloud.com:9243/index-b/_search
{
"query": {
"bool": {
"must": [
{
"wildcard": {
"FirstName": "*ralf*"
}
}
]
}
}
}
I'm getting:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "index-b",
"_type": "_doc",
"_id": "0ec623e7-3837-4e83-808d-fed01398d1ab",
"_score": 1.0,
"_source": {
"Id": 0,
"ProfileId": "0ec623e7-3837-4e83-808d-fed01398d1ab",
"FirstName": "Ralf",
"LastName": "de K",
"BirthDate": "1999-09-19T00:00:00Z",
"BirthPlace": "Zwolle",
"BirthPlaceLocation": {
"Longitude": 41.9074,
"Latitude": -78.7911
},
"City": "Zwolle",
"Email": "email#example.org",
"ObjectIdentifier": "0ec623e7-3837-4e83-808d-fed01398d1ab",
"MobileNumber": "06123123123",
"Height": 2,
"BodyBuild": 0,
"Ethnicity": 8,
"Education": 3,
"Gender": 2,
"Created": "0001-01-01T00:00:00",
"Updated": "0001-01-01T00:00:00"
}
}
]
}
}
It was my own mistake. I used ElasticLowLevelClient to add a new index and I (probably) should have used ElasticClient because the first adds the index with pascal casing and the latter camel casing. Could be a config/setting but usinf ElasticClient fixes my problem.
class MyObject{
public string Name{get;set;}
public List<string> Tags{get;set;}
}
/*Create mapping */
client.Map<MyObject>(m =>
m.Properties(props =>
props.String(s =>
s.Name(p => p.Name)
.Path(MultiFieldMappingPath.Full)
.Index(FieldIndexOption.NotAnalyzed)
.Fields(f =>
f.String(ps =>
ps.Name(p => p.Name.Suffix("searchable"))
.Index(FieldIndexOption.Analyzed)
)
)
)
)
);
How to index NotAnalyzed for field Tags same field Name
I want to search exactly one phrase in the Tags field
Example: I want search "elastic search" to find out which object contains exactly that word in the Tags field
Obj1:
{
"Name":"Object 1",
"Tags":["elastic search","how to code"]
}
Obj2:
{
"Name":"Object 2",
"Tags":["elastic","c#"]
}
Obj3:
{
"Name":"Object 2",
"Tags":["learn elastic search","learn C#"]
}
===> Result: Obj 1
Based on your request I will create a test1 index.
PUT test1/doc/1
{
"Name": "Object 1",
"Tags": [
"elastic search",
"how to code"
]
}
PUT test1/doc/2
{
"Name":"Object 2",
"Tags":["elastic","c#"]
}
So I will write query to fetch the exact term elastic search as you mentioned in your example.
GET test1/doc/_search
{
"query": {
"term": {
"Tags.keyword":
"elastic search"
}
}
}
So the result is for the below query is
curl -XGET "http://localhost:9200/test1/doc/_search"
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "test1",
"_type": "doc",
"_id": "2",
"_score": 1,
"_source": {
"Name": "Object 2",
"Tags": [
"elastic",
"c#"
]
}
},
{
"_index": "test1",
"_type": "doc",
"_id": "1",
"_score": 1,
"_source": {
"Name": "Object 1",
"Tags": [
"elastic search",
"how to code"
]
}
}
]
}
}
So now the query to fetch documents based on your field.
curl -XGET "http://localhost:9200/test1/doc/_search" -H 'Content-Type: application/json' -d'
{
"query": {
"term": {
"Tags.keyword":
"elastic search"
}
}
}'
And the result is
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.2876821,
"hits": [
{
"_index": "test1",
"_type": "doc",
"_id": "1",
"_score": 0.2876821,
"_source": {
"Name": "Object 1",
"Tags": [
"elastic search",
"how to code"
]
}
}
]
}
}
Hope it works. Let me know if you are still facing any issues.
This is a repost of a previous question that was wrongly closed as a duplicate:
Say I have two collections of the same type similar to the following:
OldCollection: [{ID: 1, other props}, {ID: 2, other props}, {ID: 3, other props}]
NewCollection: [{ID: 1, other props}, {ID: 3, other props}, {ID: 4, other props}]
Is there a way to zip the collections matching ID's to get a Tuple result like the following:
[{OLD-ID1, NEW-ID1},
{OLD-ID2, null},
{OLD-ID3, NEW-ID3},
{null, NEW-ID4}]
So basically I want to zip the collections together matching on ID and if only one collection has an entry with a particular ID the Tuple should fill in null for that spot.
IMPORTANT: This is not a duplicate of the full outer join solution. I do not want to combine my results so that Col1-ID1 gets merged with Col2-ID1. I Just want to Tuple them so I can see them side by side. Think of it as Collection 1 is the old values and Collection 2 is the new values. I want them to be paired up into Tuples so I can see ID1 and ID3 were updated, ID2 was removed, and ID4 was added.
Here is a DotNetFiddle example that almost does what I want.
The results of that fiddle is:
[
{
"Item1": {
"id": 1,
"name": "firstOld"
},
"Item2": {
"id": 1,
"name": "firstNew"
}
},
{
"Item1": {
"id": 2,
"name": "secondOld"
},
"Item2": {
"id": 3,
"name": "thirdNew"
}
},
{
"Item1": {
"id": 3,
"name": "thirdOld"
},
"Item2": {
"id": 4,
"name": "fourthNew"
}
},
{
"Item1": null,
"Item2": {
"id": 5,
"name": "fifthNew"
}
}
]
What I want is this:
[
{
"Item1": {
"id": 1,
"name": "firstOld"
},
"Item2": {
"id": 1,
"name": "firstNew"
}
},
{
"Item1": {
"id": 2,
"name": "secondOld"
},
"Item2": null
},
{
"Item1": {
"id": 3,
"name": "thirdOld"
},
"Item2": {
"id": 3,
"name": "thirdNew"
}
},
{
"Item1": null,
"Item2": {
"id": 4,
"name": "fourthNew"
}
},
{
"Item1": null,
"Item2": {
"id": 5,
"name": "fifthNew"
}
}
]
You want a full-outer-join anyway, but you could use this easy approach:
var oldByID = oldCollection.ToDictionary(x => x.Id);
var newByID = newCollection.ToDictionary(x => x.Id);
IEnumerable<int> allIds = oldByID.Keys.Union(newByID.Keys);
var oldAndNew = allIds.Select(id =>
(Old: oldByID.TryGetValue(id, out var oldObj) ? oldObj : null,
New: newByID.TryGetValue(id, out var newObj) ? newObj : null));
So first create two dictionaries to lookup the old and new objects via Id efficiently. Then collect all Id's and use Select to get the old/new object with each Id. If it's not available in the dictionary it will assign null to the tuple item.
I have following JToken output. How can I retrieve the 'value' here from TenantID which should be 1 in this case?
{[
{
"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}
]}
This is my current code :
JToken value;
if (usr.Profile.TryGetValue("tenantid", out value))
{
JObject inner = value["value"].Value<JObject>(); //not working with null error
User.TenantID = (string)value;
}
User.obj = usr.Profile;
EDIT - please find below the complete JToken output :
{[
{
"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}
]}
Count: 1
First (Newtonsoft.Json.Linq.JContainer): {{
"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}}
First (Newtonsoft.Json.Linq.JToken): {{
"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}}
HasValues: true
IsReadOnly: false
Last (Newtonsoft.Json.Linq.JContainer): {{
`"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}}
Last (Newtonsoft.Json.Linq.JToken): {{
"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}}
Next: (null)
Parent: {"tenantid": [
{
"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}
]}
Path: "tenantid"
Previous: (null)
Static members:
Non-public members:
IEnumerator:
Root: {{
"name": "Rx Sidhu",
"given_name": "Rx",
"family_name": "Sidhu",
"locale": "en_US",
"emails": [
"ranxdeep#xxx.com",
"ranxdeep#xxx.com",
"ranxdeep#hx.com"
],
"nickname": "ranxdeep#xxx.com",
"email": "ranxdeep#xxx.com",
"picture": "https://apis.live.net/v5.0/f1xxxxxx/picture",
"roles": [
"Account Admin",
"Admin"
],
"tenantid": [
{
"value": 1,
"metadata": {
"userType": 0,
"flags": 8,
"type": {
"type": "INT4",
"name": "Int",
"id": 56
},
"colName": "TenantID"
}
}
],
"email_verified": true,
"clientID": "wxxxvC8",
"updated_at": "2015-07-15T16:26:30.526Z",
"user_id": "windowslive|f1axxxac",
"identities": [
{
"access_token": "EwBwAq1",
"provider": "windowslive",
"user_id": "f1aexxxxxac",
"connection": "windowslive",
"isSocial": true
}
],
"created_at": "2015-07-01T06:08:21.358Z"
}}
Type: 2
I need to check if the tenantID actually exists and then get value, else return null or 0.
You should be able to do:
JObject jObject = JObject.Parse(...);
JToken value = jObject.SelectToken("value");
You parse your object, then the inner contents should be exposed in which you can leverage the SelectToken method to find that specific value.
To build it out a bit, you could potentially do:
public static JToken FindToken<T>(string key, T value)
{
string serialized = NewtonsoftJsonSerializer.Instance.Serialize(value);
var jObject = JObject.Parse(serialized);
var jToken = jObject.SelectToken(key);
if(jToken != null)
return jToken;
return null;
}
I still cannot get the overall picture of your JSON, though have a look at this. It may help you.
var str = #"{
""x"": [{
""value"": 1,
""metadata"": {
""userType"": 0,
""flags"": 8,
""type"": {
""type"":""INT4"",
""name"":""Int"",
""id"": 56
},
""colName"":""TenantID""
}
}
]
}";
var parentJObject = JObject.Parse(str);
var xJArray = (JArray)parentJObject["x"];
// first item in JArray which is the object of interest
// look for the appropriate index of the JObject of your data
var firstJTokenInxJArray = (JObject)xJArray[0];
Console.WriteLine(firstJTokenInxJArray["value"].ToString());
I got what I was looking for :
JToken value;
if (usr.Profile.TryGetValue("tenantid", out value))
{
User.TenantID = (int)value [0] ["value"];
}