I'm using Nest to build a query. I have a class called Event which has a property called Location and a property called Address (a string). So the indices look something like this:
"id" : "something1",
"location": {
"address": "The Club",
}
"id" : "something2",
"location": {
"address": "The Hole",
}
I want to create a query where the user types "The Club" (the location variable) and it only retrieves the first item that has "The Club" as the address
This is my current query
var skip = (page - 1) * rows;
var searchResponse = await _elasticClient.SearchAsync<Event>(s => s
.From(skip)
.Size(rows)
.Query(q => q
.QueryString(qs => qs
.Fields(fields => fields
.Field(f => f.Location.Address)
)
.Analyzer("keyword")
.Query("*" + (location ?? string.Empty) + "*")
)
));
Currently, the query retrieves both objects since it finds "The". Do I need to add something to my index previously or what should I do?
For the record I added this when creating the index but it has no effect
client.Indices.Create(eventsIndex, c => c.Map<Event>(m => m
.Properties(p => p
.Text(t => t
.Name(n => n.Location.Address)
.Analyzer("keyword"))
)
));
I have two suggestions:
1 - Add the stop filter (default language english) that way you won't produce tokens like the term "the", all stopwords will be removed. Now you will only match the term "hotel", the term "the" will be filtered out.
POST _analyze
{
"tokenizer": "standard",
"filter": ["stop"],
"text": ["the hotel"]
}
Token
{
"tokens": [
{
"token": "hotel",
"start_offset": 4,
"end_offset": 9,
"type": "<ALPHANUM>",
"position": 1
}
]
}
2 - Use the default_operator AND in Query String query.
Related
I have an ElasticSearch index called challenges, which contains objects of type Challenge.
When I execute the following filter query in the Kibana console, it returns the 9 results, which are correct.
GET challenges/_search
{
"query": {
"bool": {
"filter": [
{
"term": {
"type": "Orphan"
}
}
]
}
}
}
However, the following query from the Nest client, returns zero hits:
var challenges = await _client.SearchAsync<Challenge>(s => s
.Query(q => +q
.Term(t => t.Type, Models.Enums.ChallengeType.Orphan)
)
);
I've also tried the following variation, to no avail:
var challenges = await _client.SearchAsync<Challenge>(s => s
.Query(q => q
.Bool(b => b
.Filter(f => f
.Term(t => t
.Field(f => f.Type)
.Value(challengeType)
)
)
)
)
);
The type property on which I'm filtering, is an enum with the following values:
public enum ChallengeType
{
SixDimensions,
Intro,
Normal,
UserCreated,
Orphan,
Youmate
}
and is stored as a keyword in the index.
An example object that is actually in the index:
{
"id": "3bce0ce1-9676-4858-b165-1442a443bf5a",
"icon": "water-bottle.png",
"index": 0,
"default-time": "09:00",
"default-days": [
"Saturday",
"Monday",
"Wednesday"
],
"default-repetitions": 3,
"category": "A",
"title": {
"Persian": "آب خوردن"
},
"dimension": "Physical",
"type": "Orphan",
"id-package": "00000000-0000-0000-0000-000000000000",
"intro-pages": [ ],
"date-created": "2020-10-14T12:39:21.8427517+03:30",
"notify": true,
"template": 0,
"belongs-to-user": "00000000-0000-0000-0000-000000000000",
"active": false
}
Do you have any suggestions as to why the results from the console differ from when it is executed from the Nest client?
The enum should be marked with StringEnumAttribute to serialize as a string
[StringEnum]
public enum ChallengeType
{
SixDimensions,
Intro,
Normal,
UserCreated,
Orphan,
Youmate
}
Without this, the enum will be serialized as its underlying integer value.
I am trying to use NEST to create search query dynamically based on user's input.
I want to add multiple filter in Filter with Term but string field searching is not possible and I cannot find any solution.
Code for example is that, this code try to search on string field an it is not working
var response = await _elasticClient.SearchAsync<CustomerAddressInfo>(p => p
.Query(q => q
.Bool(b => b
.Filter(f => f.Term(t => t.Field(p => p.AccountAddressId).Value(type.AccountAddressId)))
)
)
);
And the other search simple is with integer field and it is working with success
var response = await _elasticClient.SearchAsync<CustomerAddressInfo>(p => p
.Query(q => q
.Bool(b => b
.Filter(f => f.Term(t => t.Field(p => p.CreateUnitId).Value(type.CreateUnitId)))
)
)
);
But; if I search data on string field with Match keyword, again it is successfull on search
var response = await _elasticClient.SearchAsync<CustomerAddressInfo>(p => p
.Query(q => q
.Match(m => m
.Field(f => f.AccountAddressId)
.Query(type.AccountAddressId)
)
)
);
And the question is, how can I give multiple search criteria with Match query method or how can I seach on string field by Term query method on elastic
I am not familiar with NEST, but to search on multiple fields using match query or term query, you can refer following example :
Bool query is used to combine one or more clauses, to know more refer this
Avoid using the term query for text fields.
By default, Elasticsearch changes the values of text fields as part of
analysis. This can make finding exact matches for text field values
difficult.
To search text field values, use the match query instead.
Index Mapping
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"cost": {
"type": "long"
}
}
}
}
Index data:
{
"name":"apple",
"cost":"40"
}
{
"name":"apple",
"cost":"55"
}
Search Query: Multiple Search criteria with match
{
"query": {
"bool": {
"must": [
{ "match": { "name": "apple" }},
{ "match": { "cost": 40 }}
]
}
}
}
Search on-field by term query
{
"query": {
"bool" : {
"must" :[
{"term" : { "name" : "apple" }},
{"term": { "cost":40 }}
]
}
}
}
Search Result:
"hits": [
{
"_index": "my-index",
"_type": "_doc",
"_id": "3",
"_score": 1.1823215,
"_source": {
"name": "apple",
"cost": "40"
}
}
]
Hey i do not get the whole requirements of yours. But if you want to add multiple condition on filter then you can do like below.
QueryContainer qSs = null;
foreach(var query in queries) // let say queries is list of yours search item
{
qSs &= new TermQuery { Field = "your_field_name", Value = query };
}
var searchResults = await _elasticClient.SearchAsync<CustomerAddressInfo>(s => s
.Query(q => q
.Bool(b => b.Filter(qSs) )
)
);
I'm new to LINQ and I need to write a LINQ query that returns each project's grade, called notet also the average of all notes.
Here is the query I have:
`var query = _context.Set<EvaluationResult>()
.Include(x => x.RatingLevel)
.Include(x => x.Skill)
.Include(x => x.Evaluation)
.ThenInclude(y => y.Project)
.ThenInclude(z => z.ProjectRatingLevels)
.ThenInclude(a => a.RatingLevel)
.Include(y => y.Evaluation.Project)
.ThenInclude(y => y.Degree)
.Where(x => x.Evaluation.Project.DegreeId == QueriedDegreeId)
.GroupBy(i => new { project = i.Evaluation.Project })
.Select(g => new
{
project = g.Select(y => y.Evaluation.Project.Label)
.Distinct().FirstOrDefault(),
note = Math.Round(((g.Sum(y => (double)y.Skill.Weight * (double)y.RatingLevel.Rate) /
g.Sum(y => (double)y.RatingLevel.Rate)) * 100) /
(double)g.Key.project.ProjectRatingLevels
.Select(z => z.RatingLevel.Rate)
.Max(), 2, MidpointRounding.AwayFromZero)
});
Here is the result:
[
{
"project": "Projet 1",
"note": 42.86
},
{
"project": "Projet 2",
"note": 41.67
},
{
"project": "Projet 3",
"note": 46.67
}
]
What I want is to add another value, average, which is just the Average of all "note" values, like so (the asterisks are just for emphasis):
[
{
"project": "Projet 1",
"note": 42.86,
**"average": 43.73**
},
{
"project": "Projet 2",
"note": 41.67,
**"average": 43.73**
},
{
"project": "Projet 3",
"note": 46.67,
**"average": 43.73**
}
]
MY PROBLEM
I'm stuck trying to calculate an Average of all the returned notes. I have no idea how to proceed. I tried to add an average key in my Select after note and project that reused the note query, like this:
average = g.Average(Math.Round(((g.Sum(... etc
But this gives me type errors: CS1929 'IGrouping<<anonymous type: Project project>, EvaluationResult>' does not contain a definition for average 'Average'. So I'm at an utter loss of what to do.
Can someone point me in the right direction? Am I missing something, like the need to use a Key or something?
Is your goal to do it all in one query ?
The only possible Resultset, with Aggregate and the Aggregated Items, is a GroupBy.
If you want to aggregate all, you want only one group, so you have to have a fictive key,the same one for each item
So Append:
.GroupBy(x => 1) /* makes one group for all, all have the same key */
.Select(g => new { average = g.Average(x => x.notes), items = g.Select(x => x)});
But this is really Forcing the SQL-Server to do the average. You Manifest the items in your memory anyway. So you can also take your existing Resultset, manifested with ToList or ToArray and just compute
var result = <yourquery>.ToList();
double average = result.Average(x => x.notes);
The only difference is, this is done on your CPU, not on the SQL-Server.
I have the query shown below:
var queryResult =
await
elastic.SearchAsync<CounterData>(s => s
.Query(q => q
.Bool(b => b
.Must(
m => m.ConstantScore(c => c
.Filter(f => f
.Term(x => x.CounterId, maxList))
),
m => m.ConstantScore(c => c.
Filter(f => f
.Term(x => x.Type, counterType))),
m => m.ConstantScore(c => c.
Filter(f => f.
DateRange(d => d.
GreaterThanOrEquals(lowerBound).Field(r => r.Date)))))))
.AllTypes()
.Scroll("1m")
.Size(10000));
Where maxList is a list of integers. I want to check if the term is in the list but looks like this does not work.
Any ideas how I can check if the term matches any of the elements in the list?
Something like the following will do it
var maxList = new[] { 1, 2, 3, 4, 5};
var counterType = "counter-type";
var lowerBound = DateTime.UtcNow.Date.AddDays(-7);
var queryResult = client.Search<CounterData>(s => s
.Query(q => +q
.DateRange(d => d
.Field(r => r.Date)
.GreaterThanOrEquals(lowerBound)
) && +q
.Term(x => x.Type, counterType) && +q
.Terms(t => t
.Field(x => x.CounterId)
.Terms(maxList)
)
)
.AllTypes()
.Scroll("1m")
.Size(10000)
);
A few things to highlight
+ unary operator applied to a QueryContainerDescriptor<T> is a shorthand for wrapping a query in a bool filter query. I think this is what you want in your case as you don't need to calculate scores, you just want to find matches to a predicate.
&& is overloaded for QueryContainer such that when applied to two QueryContainers, it is a shorthand for a bool must query with two must query clauses. However in this example, the queries all have the + unary operator applied so are bool filter queries, so will be &&'ed together as filter queries.
The value passed to Size() when using Scrolling (i.e. specifying a Scroll() time) is the number of documents to fetch from each shard per scroll, not total documents per scroll. So total documents will be Size() * number of shards. This might be a lot of documents per scroll.
Use the terms query to find documents that match on a field against any one of a list of terms (not analyzed).
The end query json looks like
POST http://localhost:9200/examples/_search?scroll=1m
{
"size": 10000,
"query": {
"bool": {
"filter": [
{
"range": {
"date": {
"gte": "2016-08-04T00:00:00Z"
}
}
},
{
"term": {
"type": {
"value": "counter-type"
}
}
},
{
"terms": {
"counterId": [
1,
2,
3,
4,
5
]
}
}
]
}
}
}
I'm developing an .NET application using Elastic Search. I used ES River to index the data.
Results (in Sense) look kinda like this:
{
"_index": "musicstore",
"_type": "songs",
"_id": "J2k-NjXjRa-mgWKAq0RMlw",
"_score": 1,
"_source": {
"songID": 42,
"tempo": "andante",
"metrum": "3/4 E8",
"intonation": "F",
"title": "Song",
"archiveSongNumber": "3684",
"Year": 2000,
"Place": "London"
}
},
To access the indexed data I use NEST queries similar to this:
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.title, "Song")));
I'm having a problem that the query doesn't return any results, when I search for a certain field.
For instance when I search for a title, songID, tempo or archiveSongNumber the query works fine and it returns the same results as Sense, but when I search for Year, Place, metrum, etc. the query doesn't return any results, but it should (Sense does and it should).
Queries like these work (and return the right results):
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.title, "Song")));
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.songID, 42)));
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.archiveSongNumber , "3684")));
Queries like these don't return any results (but they should):
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.Place, "London")));
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.Year, 2000)));
What am I doing wrong? Did I mess up when I was indexing data?
Update:
Mapping looks like this:
{
"musicstore": {
"mappings": {
"songs": {
"properties": {
"Year": {
"type": "long"
},
"Place": {
"type": "string"
},
"archiveSongNumber": {
"type": "string"
},
"songID": {
"type": "long"
},
"intonation": {
"type": "string"
},
"metrum": {
"type": "string"
},
"title": {
"type": "string"
},
"tempo": {
"type": "string"
}
}
}
}
}
}
Update 2:
ES river request looks like this:
PUT /_river/songs_river/_meta
{
"type":"jdbc",
"jdbc": {
"driver":"com.microsoft.sqlserver.jdbc.SQLServerDriver",
"url":"jdbc:sqlserver://ip_address:1433;databaseName=database",
"user":"user",
"password":"password",
"strategy":"simple",
"poll":"300s",
"autocommit":true,
"fetchsize":10,
"max_retries":3,
"max_retries_wait":"10s",
"index":"musicstore",
"type":"songs",
"analysis": {
"analyzer" :{
"whitespace" :{
"type" : "whitespace",
"filter":"lowercase"
}
}
},
"sql":"some_sql_query"
}
}
ES client configuration looks like this:
private static ElasticClient ElasticClient
{
get
{
Uri localhost = new Uri("http://localhost:9200");
var setting = new ConnectionSettings(localhost);
setting.SetDefaultIndex("musicstore").MapDefaultTypeNames(d => d.Add(typeof(Song), "songs"));
setting.SetConnectionStatusHandler(c =>
{
if (!c.Success)
throw new Exception(c.ToString());
});
return new ElasticClient(setting);
}
}
From looking at your mapping, the issue here is most likely that all of your fields are being analyzed when indexing, but you are using term queries with NEST, which are not analyzed, meaning they will only find exact matches. If you don't explicitly specify an analyzer in your mappings, Elasticsearch defaults to the standard analyzer.
When you perform a search in Elasticsearch using a query string, like you're doing in Sense: GET _search/?q=Place:London, a query string query is what's being run by Elasticsearch, which is different than a term query.
From your examples though, it doesn't look like you are actually using query string syntax. You probably want a match query instead:
client.Search<Song>(s => s
.Query(q => q
.Match(m => m
.OnField(p => p.Place)
.Query("London")
)
)
);
If you do however want a query string query like the one you're performing with Sense, than you can use QueryString:
client.Search<Song>(s => s
.Query(q => q
.QueryString(qs => qs
.OnFields(p => p.Place)
.Query("London")
)
)
);
Hope that helps. I suggest checking out the getting started guide, specifically the section on exact values vs. full text.
Add "keyword" suffix to your Term Field :
var result = ElasticClient.Search<Song>(s => s
.Query(q => q
.Term(p => p
.Field(x => x.Year.Suffix("keyword")).Value(2000))));
Try it, it will work!