We Keep Coding

ElasticSearch Exact match on multiple fields

I'm newbie of Elastic Search, I'm trying to get an exact match on every field of an object in elasticsearch index. For example I have two object:
{
"_index": "sql",
"_type": "_doc",
"_id": "mpovsH",
"_score": 1.0,
"_source": {
"entityId": 1,
"user": "userfirst",
"descr": "testfirst",
}
},
{
"_index": "sql",
"_type": "_doc",
"_id": "mpovsH",
"_score": 1.0,
"_source": {
"entityId": 2,
"user": "usersecond",
"descr": "testsecond",
}
}
I want the search the string "userfirst" on all fields of the object, and get only the first one as response. I tried:
var searchResponse = client.SearchAsync<MyObject>(s => s
.Source(sf => sf)
.Query(q => q
.MultiMatch(a => a
.Query(queryValue)))).Result;
Where queryValue is "userfirst" but I get both object in results. How can I change it? Also, I would not write every single field if possible to search, because my object is way more bigger.
EDIT: I managed to get only one results with this query:
var searchResponse = client.SearchAsync<TendersElasticSearch>(s => s
.Source(sf => sf)
.Query(qn => qn
.MatchPhrasePrefix(ma => ma
.Field(x => x.User)
.Query(queryValue)))).Result;
But with this query, I get results only on field user. I would like to search on all fields of every object. Any tips?

Adding a working example with index data, mapping, search query, and search result
Index Mapping:
{
"mappings": {
"properties": {
"users": {
"type": "nested"
}
}
}
}
Index Data:
{
"users": [
{
"entityId": 1,
"user": "userfirst",
"descr": "testfirst"
},
{
"entityId": 2,
"user": "usersecond",
"descr": "testsecond"
}
]
}
Search Query:
{
"query": {
"nested": {
"path": "users",
"query": {
"bool": {
"must": [
{ "match": { "users.user": "userfirst" }}
]
}
},
"inner_hits":{}
}
}
}
Search Query using Multi match:
{
"query": {
"nested": {
"path": "users",
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "userfirst",
"fields": [
"users.user"
]
}
}
]
}
},
"inner_hits": {}
}
}
}
Search Result:
hits": [
{
"_index": "stof_64061575",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "users",
"offset": 0
},
"_score": 0.6931471,
"_source": {
"entityId": 1,
"user": "userfirst",
"descr": "testfirst"
}
}
]

C# Query :
var searchResponse = client.SearchAsync<MyObject>(s => s
.Source(sf => sf
.Nested(n=>n.Path("users").
Query(qn=>qn.bool(
b=> b.Must(
m => m. .Query(q => q
.MultiMatch(a => a
.Query(queryValue))))))
)
).Result;

How can I get top 10 results from Elastic search using rest queries with the max hit documents?

I want to get the top 10 documents\rows where these rows should be ordered with most recently and multiple times accessed docs on top.
here's what I tried:
{
"size": 10,
"query": {
"range": {
"searchDate": {
"gte": "DateTime.Now.AddDays(-30)"
},
"aggs": {
"top_tags": {
"terms": {
"field": "searchDate"
},
"aggs": {
"top_otf_hits": {
"top_hits": {
"sort": [
{
"searchDate": {
"order": "desc"
}
}
],
"_source": {
"includes": [
"origin",
"destination"
]
},
"size": 1
}
}
}
}
}
}
}
}

{
"from": 0,
"size": 0,
"sort": [{
"searchDate": "desc"
}, "_score"],
"query": {
"range": {
"searchDate": {
"gte": "2018-02-28",
"lte": "2018-03-05",
"format": "yyyy-MM-dd"
}
}
},
"aggs": {
"frequent": {
"terms": {
"field": "tripKey"
},
"aggs": {
"top_otf_hits": {
"top_hits": {
"sort": [{
"searchDate": {
"order": "desc"
}
}],
"_source": {
"include": ["*"]
},
"size": 1
}
}
}
}
}
}

nested aggregations filtering

I have my document indexed with locations nested,
{
"name": "name 1",
"locations": [
{
"region": "region1",
"city": "city1",
"suburb": "suburb1"
},
{
"region": "region2",
"city": "city2",
"suburb": "suburb2"
},
{
region": "region1",
"city": "city5",
"suburb": "suburb4"
}]
}
I have my query as
{
"query": {
"nested": {
"path": "locations",
"query": {
"bool": {
"must": [
{
"term": {
"locations.region.keyword": {
"value": "region1"
}
}
}
]
}
}
}
}
}
I want aggregate only cities for region1. I've tried nested aggregations, nested with filter aggregations, and with reverse nested. Nothing seems to work. The problem is since documents come with other regions in the locations collection, everything get aggregated even cities that don't belong to region1.
any ideas?
EDIT:
Mappings:
"my_index": {
"mappings": {
"my_type": {
"properties": {
"locations": {
"type": "nested",
"properties": {
"city": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"suburb": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
Query:
{
"size": 0,
"query": {
"nested": {
"query": {
"bool": {
"must": [
{
"terms": {
"locations.region.keyword": [
"region1"
]
}
}
]
}
},
"path": "locations"
}
},
"aggs": {
"City": {
"nested": {
"path": "locations"
},
"aggs": {
"City": {
"terms": {
"field": "locations.city.keyword",
"size": 100,
"order": [
{
"_count": "desc"
},
{
"_term": "asc"
}
]
},
"aggs": {
"City": {
"reverse_nested": {}
}
}
}
}
}
}
}

Assuming your mapping is correct as per your usage in the query
You may use the below mentioned query to use filters in your aggregation.
{
"query": {
"match_all": {}
},
"aggs": {
"city_agg": {
"nested": {
"path": "locations"
},
"aggs": {
"filter_locations_regions": {
"filter": {
"term": {
"locations.region.keyword": "region1"
}
},
"aggs": {
"cities_in_region_agg": {
"terms": {
"field": "locations.city.keyword",
"size": 100,
"order": [{
"_count": "desc"
},
{
"_term": "asc"
}]
}
}
}
}
}
}
}
}

Elasticsearch - define query string as a filter in C#

I have the next working elastic search query :
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"IsDeleted": false
}
},
{
"query": {
"query_string": {
"fields": [
"payPlan.PayPlanData.*"
],
"query": "(StartInterval :[1 TO 100] AND (EndInterval :[1 TO 9999999]))"
}
}
}
]
}
}
}
}
}
Is it possible to define it as a c# Nest.FilterDescriptor element?
The main issue is defining a QueryStringQuery as a Filter

Here is an example of what I managed to create to implement an elastic search query string as a filter, I added other filters so you can see a real example:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"NotSpan": true
}
},
{
"and": {
"filters": [
{
"bool": {
"must": [
{
"fquery": {
"query": {
"query_string": {
"query": "(StartTimeAsNumber :[1 TO 12345] AND (EndTimeAsNumber :[12345 TO 9999999]))",
"default_field": "PayPlan.PayPlanData.*"
}
}
}
}
]
}
}
]
}
}
],
"must_not": [
{
"term": {
"userRole": "admin"
}
}
]
}
}
}
}
}

Using C# Nest API to get nested json data is not retrieving data

I have the following json coming through the elasticsearch:
{
"_index": "data-2016-01-14",
"_type": "type-data",
"_id": "AVJBBNG-TE8FYIA1rf1p",
"_score": 1,
"_source": {
"#message": {
"timestamp": 1452789770326461200,
"eventID": 1452789770326461200,
"eventName": "New",
"Price": "38.34",
"Qty": 100,
"statistic_LatencyValue_ns": 1142470,
"statistic_LatencyViolation": false,
"statistic_LossViolation": false
},
"#timestamp": "2016-01-14T16:42:50.326Z",
"#fields": {
"timestamp": "1452789770326"
}
},
"fields": {
"#timestamp": [
1452789770326
]
}
}
I'm using Nest to try to get the eventName data i created the class and marked the property:
public class ElasticTest
{
[ElasticProperty(Type = FieldType.Nested)]
public string eventName { get; set; }
}
But the following query is returning 0 results, what am i doing wrong?
var result = client.Search<CorvilTest>(s => s
.From(0)
.Size(10000)
.Query(x => x
.Term(e => e.eventName,"New"))
);
var r = result.Documents;
Mapping definition:
{
"data-2016-01-14": {
"mappings": {
"type-data": {
"properties": {
"#fields": {
"properties": {
"timestamp": {
"type": "string"
}
}
},
"#message": {
"properties": {
"OrderQty": {
"type": "long"
},
"Price": {
"type": "string"
},
"eventID": {
"type": "long"
},
"eventName": {
"type": "string"
},
"statistic_LatencyValue_ns": {
"type": "long"
},
"statistic_LatencyViolation": {
"type": "boolean"
},
"statistic_LossViolation": {
"type": "boolean"
},
"timestamp": {
"type": "long"
}
}
},
"#timestamp": {
"type": "date",
"format": "dateOptionalTime"
}
}
}
}
}
}

I see that the field #message.eventName is using a standard analyzer which means that its value is lower-cased and split at word boundaries before indexing. Hence the value "new" is indexed and not "New". Read more about it here. You need to be mindful about this fact when using a Term Query. Another thing is that the field eventName is not of nested type. So the code below should work for you.
var result = client.Search<CorvilTest>(s => s
.From(0)
.Size(10000)
.Query(x => x
.Term(e => e.Message.EventName, "new"))); // Notice I've used "new" and not "New"
var r = result.Documents;
For the above code to work the definition of CorvilTest class should be something like below:
public class CorvilTest
{
[ElasticProperty(Name = "#message")]
public Message Message { get; set; }
/* Other properties if any */
}
public class Message
{
[ElasticProperty(Name = "eventName")]
public string EventName { get; set; }
}