I am using Nest elasticSearch as a client library to interact with the elasticSearch indices.
I am trying to send match_phrase query using the following code:
var searchResponse = elasticClient.Search<ProductType>(s => s
.Index(indices)
.Type(Types.Type(typeof(ProductType)))
.From(0)
.Size(5)
.Query(q =>
q.MatchPhrase(m => m
.Field(Infer.Field<ProductType>(ff => ff.Title))
.Slop(5)
.Query("my query")
)
)
);
It's generating the following query :
GET /product/_search
{
"from": 0,
"size": 5,
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"query": {
"match": {
"title": {
"type": "phrase",
"query": "my query",
"slop": 5
}
}
}
}
When I execute the above query it returns parsing_exception:
[match] query does not support [type]
I was expecting the above code to return query like the following:
GET /product/_search
{
"from": 0,
"size": 5,
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"query": {
"match_phrase": {
"title": {
"query": "my query",
"slop": 5
}
}
}
}
So is there anything wrong with my code and how can I get rid of it?
After investigation and depending on match query does not support type I have found that the server I am hosting my cluster on upgraded ElasticSearch into V6.5.0 and it turns out I should upgrade my Nest NuGet package and now it's generating the match_phrase query as expected.
Related
I'm trying to aggregate some data by day to make some charts but the datehistogram aggregate return only empty bucket.
My data look like:
Date: July 2nd 2019, 12:08:50.647
_id: 4959287196855971761665003616
And my Nest request :
DateTime now = DateTime.Now;
var descriptor = new SearchDescriptor<ModelWrapper>()
.Index("command-*")
.AllTypes()
.From(0)
.Size(100)
.Sort(s => s.Descending("Date"))
.Aggregations(a => a
.DateHistogram("daily", v => v
.Field(p => p.Date)
.Interval(DateInterval.Day)
.ExtendedBounds(now.AddMonths(-2), now)
))
When I make the request on the kibana console:
{
"aggs" : {
"daily" : {
"date_histogram" : {
"field" : "Date",
"interval" : "1D",
"extended_bounds": {
"min": "2019-06-02T12:01:02.123",
"max": "2019-07-02T12:01:02.123"
},
}
}
}
}
I get the expected result.
I'm on Nest 6.2 and Elastic 6.2
So after some fiddling around I used this piece of code to serialize my SearchDescriptor to the sent string:
var json = m_client.RequestResponseSerializer.SerializeToString(descriptor);
which got me:
{
"aggs": {
"daily": {
"date_histogram": {
"extended_bounds": {
"max": "2019-07-02T16:25:57.522217",
"min": "2019-05-02T16:25:57.522217"
},
"field": "date",
"interval": "day"
}
}
},
"from": 0,
"size": 100,
"sort": [{
"Date": {
"order": "desc"
}
}]
}
The issue here is that my field is put in lowercase, wich is expected behaviour (as pointed out here ).
The solution is to override the DefaultFieldNameInferrer which does this transformation, to do this I added to my connection settings:
var settings = new ConnectionSettings(pool);
settings.DefaultFieldNameInferrer(p => p);
And now everything work as expected.
Edit:
As noted by Russ Cam I can also pass a string to Field() so another solution is just to do:
.Field(nameof(ModelWrapper.Date))
I have a problem about querying a complex data type with c# nest api in elasticsearch. My model in elasticsearch is like this:
"hits": [
{
"_index": "post",
"_type": "postmodel",
"_source": {
"projectId": "2",
"language": "en",
"postDate": "2017-06-11T08:39:32Z",
"profiles": [
{
"label": "Emotional",
"confidence": 1
}
]
}
},
{
"_index": "post",
"_type": "postmodel",
"_source": {
"projectId": "3",
"language": "en",
"postDate": "2017-06-11T08:05:01Z",
"profiles": [
{
"label": "Fact oriented",
"confidence": 0.69
},
{
"label": "Rational",
"confidence": 1
}
]
}
},
...
By using c# Nest API, i want to fetch the postmodels which is projectId=3 and with "Rational" profile. My current code looks like this:
var postModels = await _elasticClient.SearchAsync<PostModel>(s => s
.Index("post")
.Query(q =>
{
QueryContainer query = new QueryContainer();
query = query && q.Match(m => m.Field(f => f.ProjectId)
.Query("3"));
return query;
}));
But i dont know how to query "Profiles". i want to extend my query to fetch specific profiles as well. I would be happy if someone can help me with this problem. Thank you in advance.
I am fairly new to ElasticSearch and am having issues getting search results that I perceive to be good. My objective is to be able to search an index of medications (6 fields) against a phrase that the user enters. It could be one ore more words. I've tried a few approaches, but I'll outline the best one I've found so far below. Let me know what I'm doing wrong. I'm guessing that I'm missing something fundamental.
Here is a subset of the fields that I'm working with
...
"hits": [
{
"_index": "indexus2",
"_type": "Medication",
"_id": "17471",
"_score": 8.829264,
"_source": {
"SearchContents": " chew chewable oral po tylenol",
"MedShortDesc": "Tylenol PO Chew",
"MedLongDesc": "Tylenol Oral Chewable"
"GenericDesc": "ACETAMINOPHEN ORAL"
...
}
}
...
The fields that I'm searching against used an Edge NGram Analyzer. I'm using the C# Nest library for the indexing
settings.Analysis.Tokenizers.Add("edgeNGram", new EdgeNGramTokenizer()
{
MaxGram = 50,
MinGram = 2,
TokenChars = new List<string>() { "letter", "digit" }
});
settings.Analysis.Analyzers.Add("edgeNGramAnalyzer", new CustomAnalyzer()
{
Filter = new string[] { "lowercase" },
Tokenizer = "edgeNGram"
});
I am using a more_like_this query against the fields in question
GET indexus2/Medication/_search
{
"query": {
"more_like_this" : {
"fields" : ["MedShortDesc",
"MedLongDesc",
"GenericDesc",
"SearchContents"],
"like_text" : "vicodin",
"min_term_freq" : 1,
"max_query_terms" : 25,
"min_word_len": 2
}
}
}
The problem is that for this search for 'vicodin', I'd expect to see matches with the full work first, but I don't. Here is a subset of the results from this query. Vicodin doesn't show up until the 7th result
"hits": [
{
"_index": "indexus2",
"_type": "Medication",
"_id": "31192",
"_score": 4.567309,
"_source": {
"SearchContents": " oral po victrelis",
"MedShortDesc": "Victrelis PO",
"MedLongDesc": "Victrelis Oral",
"RepresentativeRoutedGenericDesc": "BOCEPREVIR ORAL",
...
}
}
<5 more similar results>
{
"_index": "indexus2",
"_type": "Medication",
"_id": "26198",
"_score": 2.2836545,
"_source": {
"SearchContents": " (original 5 500 feeding mg strength) tube via vicodin",
"MedShortDesc": "Vicodin 5 mg-500 mg (Original Strength) via feeding tube",
"MedLongDesc": "Vicodin 5 mg-500 mg (Original Strength) via feeding tube",
"GenericDesc": "HYDROCODONE BITARTRATE/ACETAMINOPHEN ORAL",
...
}
}
Field Mappings
"OrderableMedLongDesc": {
"type": "string",
"analyzer": "edgeNGramAnalyzer"
},
"OrderableMedShortDesc": {
"type": "string",
"analyzer": "edgeNGramAnalyzer"
},
"RepresentativeRoutedGenericDesc": {
"type": "string",
"analyzer": "edgeNGramAnalyzer"
},
"SearchContents": {
"type": "string",
"analyzer": "edgeNGramAnalyzer"
},
Here is what ES shows for my _settings for analyzers
"analyzer": {
"edgeNGramAnalyzer": {
"type": "custom",
"filter": [
"lowercase"
],
"tokenizer": "edgeNGram"
}
},
"tokenizer": {
"edgeNGram": {
"min_gram": "2",
"type": "edgeNGram",
"max_gram": "50"
}
}
As per the above mapping edgeNGramAnalyzer is the search-analyzer for the fields as a result the search query would also get "edge ngrammed". You probably do not want this .
Change the mapping to set only the index_analyzer option as edgeNgramAnalyzer.
The search_analyzer would then default to standard.
Example:
"SearchContents": {
"type": "string",
"index_analyzer": "edgeNGramAnalyzer"
},
I am using the NEST API (v0.12.0.0) to interface with an ElasticSearch (v1.0.1) index and I just started receiving a JsonSerializationException when retrieving my data. I'm not sure if this is a NEST issue or otherwise, but it just randomly started happening and we haven't made any major changes to our implementation or infrastructure.
I am attempting to retrieve the Ids of my data (stored as a Guid) with a typed Search<>() and I am getting an exception when the data is processed by JSON.NET.
client.Search<ESEventItem>(s =>
s.Index("dev-events004")
.Fields(f => f.Id).Size(100000)
.Type("event").MatchAll()).Documents.ToList()
Running this same query manually in Sense produces no noticeable issues:
POST /dev-events004/event/_search
{
"size": 100000,
"query": {
"match_all": {}
},
"fields": [
"id"
]
}
{
"took": 2088,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 19257,
"max_score": 1,
"hits": [
{
"_index": "dev-events004",
"_type": "event",
"_id": "670a1055-cbe3-480e-b807-a2b500f9dfb3",
"_score": 1,
"fields": {
"id": [
"670a1055-cbe3-480e-b807-a2b500f9dfb3"
]
}
},
/* ... additional results ... */
]
}
}
If I perform a raw, untyped query Fields(new[] { "Id" }) it does not throw an exception. Likewise, if I return the whole ESEventItem object, rather than just the Id fields, it also works without an exception.
To the NEST developer: this question is mirrored as an issue on the github project.
This is due the fact that elasticsearch 1.0 changed how fields are returned. The upcomming NEST 1.0 will support this.
I am looking for a good way to feed a d3.js bubble chart with data from my MVC application. For example the standard bubble chart expects nested data in the form:
{
"name": "flare",
"children": [
{
"name": "analytics",
"children": [
{
"name": "cluster",
"children": [
{
"name": "CNN",
"size": 3938
}
]
},
{
"name": "graph",
"children": [
{
"name": "MTV",
"size": 3534
}
]
}
]
}
]
}
What I have on the server side is this linq query to a SQL database:
var results = from a in db.Durations
where a.Category == "watch"
group a by a.Description
into g
select new
{
name = g.Key,
size = g.Select(d => new{d.Begin, d.End}).Sum(d => SqlFunctions.DateDiff("hh", d.Begin, d.End))
};
return Json(results, JsonRequestBehavior.AllowGet);
The query result, parsed as Json, looks like this:
[{"name":"CNN","size":1950},{"name":"MTV","size":1680}]
I've got stuck in the head on what would be a good way to achieve the correct formatting and to create the nested structure from my query results..
server-side, using anonymous types
server-side, adjusting the linq-query
client-side, using d3.js nest
use a simpler bubble model since for my purpose, the nested
structure with children is not really needed
something totally different and much much cooler than 1-4
Thank you for any input.
Replace your return statement with the following one.
return Json(new
{
name = "Sites",
children = results
},
JsonRequestBehavior.AllowGet);
That will give you the following:
{
"name": "Sites",
"children": [
{
"name": "CNN",
"size": 1950
},
{
"name": "MTV",
"size": 1680
}
]
}
To serve as an example, suppose each website had an additional string Type property, with values such as "News" or "Music". Then you could do the following.
return Json(new
{
name = "Sites",
children = results.GroupBy(site => site.Type).Select(group => new
{
name = group.Key,
children = group
}
},
JsonRequestBehavior.AllowGet);
This would give you something like the following.
{
"name": "Sites",
"children": [
{
"name": "News",
"children": [
{
"name": "CNN",
"size": 1950
},
{
"name": "The Verge",
"size": 1600
}
]
},
{
"name": "Music",
"children": [
{
"name": "MTV",
"size": 1680
},
{
"name": "Pandora",
"size": 2000
}
]
}
]
}