I'm trying to aggregate some data by day to make some charts but the datehistogram aggregate return only empty bucket.
My data look like:
Date: July 2nd 2019, 12:08:50.647
_id: 4959287196855971761665003616
And my Nest request :
DateTime now = DateTime.Now;
var descriptor = new SearchDescriptor<ModelWrapper>()
.Index("command-*")
.AllTypes()
.From(0)
.Size(100)
.Sort(s => s.Descending("Date"))
.Aggregations(a => a
.DateHistogram("daily", v => v
.Field(p => p.Date)
.Interval(DateInterval.Day)
.ExtendedBounds(now.AddMonths(-2), now)
))
When I make the request on the kibana console:
{
"aggs" : {
"daily" : {
"date_histogram" : {
"field" : "Date",
"interval" : "1D",
"extended_bounds": {
"min": "2019-06-02T12:01:02.123",
"max": "2019-07-02T12:01:02.123"
},
}
}
}
}
I get the expected result.
I'm on Nest 6.2 and Elastic 6.2
So after some fiddling around I used this piece of code to serialize my SearchDescriptor to the sent string:
var json = m_client.RequestResponseSerializer.SerializeToString(descriptor);
which got me:
{
"aggs": {
"daily": {
"date_histogram": {
"extended_bounds": {
"max": "2019-07-02T16:25:57.522217",
"min": "2019-05-02T16:25:57.522217"
},
"field": "date",
"interval": "day"
}
}
},
"from": 0,
"size": 100,
"sort": [{
"Date": {
"order": "desc"
}
}]
}
The issue here is that my field is put in lowercase, wich is expected behaviour (as pointed out here ).
The solution is to override the DefaultFieldNameInferrer which does this transformation, to do this I added to my connection settings:
var settings = new ConnectionSettings(pool);
settings.DefaultFieldNameInferrer(p => p);
And now everything work as expected.
Edit:
As noted by Russ Cam I can also pass a string to Field() so another solution is just to do:
.Field(nameof(ModelWrapper.Date))
Related
I have the following document structure:
{
"Agencies": [
{
"name": "tcs",
"id": "1",
"AgencyUser": [
{
"UserName": "ABC",
"Code": "ABC40",
"Link": "http.ios.com",
"TotalDownloads": 0
},
{
"UserName": "xyz",
"Code": "xyz20",
"Link": "http.ios.com",
"TotalDownloads": 0
}
]
}
]
}
Like I have multiple agencies and each agency contains a list of agents.
What I am trying is to pass the Code and update the TotalDownloads field of the agent that matches the code.
For example, if someone uses the code ABC40 so I need to update the field TotalDownloads of the agent called "ABC".
What I have tried is as below:
public virtual async Task UpdateAgentUsersDownloadByCode(string Code)
{
var col = _db.GetCollection<Agencies>(Agencies.DocumentName);
FilterDefinition<Agencies> filter = Builders<Agencies>.Filter.Eq("AgencyUsers.Code", Code);
UpdateDefinition<Agencies> update = Builders<Agencies>.Update.Inc(x => x.AgencyUsers.FirstOrDefault().TotalDownloads, 1);
await col.UpdateOneAsync(filter, update);
}
It is giving me the following error:
Unable to determine the serialization information for x => x.AgencyUsers.FirstOrDefault().TotalDownloads.
Where I'm wrong?
Note: From the attached sample document, the array property name: AgencyUser is not matched with the property name that you specified in the update operation, AgencyUsers.
Use arrayFilters with $[<identifier>] positional filtered operator to update the element(s) in the array.
MongoDB syntax
db.Agencies.update({
"AgencyUsers.Code": "ABC40"
},
{
$inc: {
"AgencyUsers.$[agencyUser].TotalDownloads": 1
}
},
{
arrayFilters: [
{
"agencyUser.Code": "ABC40"
}
]
})
Demo # Mongo Playground
MongoDB .NET Driver syntax
UpdateDefinition<Agencies> update = Builders<Agencies>.Update
.Inc("AgencyUsers.$[agencyUser].TotalDownloads", 1);
UpdateOptions updateOptions = new UpdateOptions
{
ArrayFilters = new[]
{
new BsonDocumentArrayFilterDefinition<Agencies>(
new BsonDocument("agencyUser.Code", Code)
)
}
};
UpdateResult result = await col.UpdateOneAsync(filter, update, updateOptions);
Demo
i want to filter my query using c# and EF Core so that from this list:
{
"result": [
{
"commissionId": "b99a0152-b3a5-4ff6-f19e-08da6a45d751",
"commissionCode": "0001",
"commissionDescription": "Description1",
"activityId": "323e6237-c3f6-4616-3117-08da6a28ad38",
"activityCode": "01.1",
"activityDescription": "DELETE FILE",
"minuteWorked": 10440,
"activityList": null,
"timeWorked": "7.06:00:00"
},
{
"commissionId": "b99a0152-b3a5-4ff6-f19e-08da6a45d751",
"commissionCode": "0001",
"commissionDescription": "Description1",
"activityId": "95d37329-acac-4443-3118-08da6a28ad38",
"activityCode": "01.2",
"activityDescription": "DOWNLOAD FILE",
"minuteWorked": 15,
"activityList": null,
"timeWorked": "00:15:00"
},
{
"commissionId": "b99a0152-b3a5-4ff6-f19e-08da6a45d751",
"commissionCode": "0001",
"commissionDescription": "Description1",
"activityId": "89fd1d93-b5b8-4c08-3119-08da6a28ad38",
"activityCode": "01.3",
"activityDescription": "FILE SAVE",
"minuteWorked": 0,
"activityList": null,
"timeWorked": "00:00:00"
},
I get this:
{
"result": [
{
"commissionId": "b99a0152-b3a5-4ff6-f19e-08da6a45d751",
"commissionCode": "0001",
"commissionDescription": "Description1",
"activityList": "{ 323e6237-c3f6-4616-3117-08da6a28ad38",95d37329-acac-4443-3118-
08da6a28ad38,89fd1d93-b5b8-4c08-3119-08da6a28ad38 }
"timeWorked": "7.06:00:00"
},
Just to clarify the first list is obtained doing a:
GroupBy( c => new {c.ActivityId, c.CommissionId}
And then:
Select(grp => new RegistrationStatisticViewModel() {CommissionId = grp.Key.CommissionId,
CommissionCode = grp.First().Commission.Code,
CommissionDescription = grp.First().Commission.Description,
ActivityId = grp.Key.ActivityId,
ActivityCode = grp.First().Activity.Code,
ActivityDescription = grp.First().Activity.Description,
MinuteWorked = grp.Sum(c => c.MinuteWorked)
})
.ToListAsync(),
The second list essentially groups all Commission with the same id and then i guess ? it has a property of type List<T> that stores all the activityId.
So i want to group all the activity for one commissionId but only have one commissionCode etc.
Thanks to whoever replies!
Try this:
grp
.ToList()
.Where(cm => cm.CommissionId = CommissionId)
.ToList()
.ForEach(c =>
{
grp.activityList.Add(c.activityId);
});
NOTE: untested code.
I am using Nest elasticSearch as a client library to interact with the elasticSearch indices.
I am trying to send match_phrase query using the following code:
var searchResponse = elasticClient.Search<ProductType>(s => s
.Index(indices)
.Type(Types.Type(typeof(ProductType)))
.From(0)
.Size(5)
.Query(q =>
q.MatchPhrase(m => m
.Field(Infer.Field<ProductType>(ff => ff.Title))
.Slop(5)
.Query("my query")
)
)
);
It's generating the following query :
GET /product/_search
{
"from": 0,
"size": 5,
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"query": {
"match": {
"title": {
"type": "phrase",
"query": "my query",
"slop": 5
}
}
}
}
When I execute the above query it returns parsing_exception:
[match] query does not support [type]
I was expecting the above code to return query like the following:
GET /product/_search
{
"from": 0,
"size": 5,
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"query": {
"match_phrase": {
"title": {
"query": "my query",
"slop": 5
}
}
}
}
So is there anything wrong with my code and how can I get rid of it?
After investigation and depending on match query does not support type I have found that the server I am hosting my cluster on upgraded ElasticSearch into V6.5.0 and it turns out I should upgrade my Nest NuGet package and now it's generating the match_phrase query as expected.
This may be too basic of a question for SO, but I thought I would ask anyway.
I getting my feet wet with ElasticSearch and am trying to return a single document that has an exact match to my field of interest.
I have the field "StoryText" which is mapped as type "string" and indexed as "not_analyzed".
When I search using a the basic URI query:
123.456.0.789:9200/stories/storyphrases/_search?q=StoryText:"The boy sat quietly"
I return an exact matched document as I expected with a single hit.
However, when I use the search functionality:
GET 123.456.0.789:9200/stories/storyphrases/_search
{
"query" : {
"filtered" : {
"filter" : {
"term" : {
"StoryText" : "The boy sat quietly"
}
}
}
}
}
I get multiple documents returned with many hits (i.e. "The boy sat loudly", "The boy stood quietly" etc. etc.)
Could somebody help me to understand how I need to restructure my search request to mimic the result I get using the basic query parameter?
At present I am using NEST in C# to generate my search request which looks like this
var searchresults = client.Search<stories>(p => p
.Query(q => q
.Filtered(r => r
.Filter(s => s.Term("StoryText", inputtext))
)
)
);
Thanks very much for any and all reads and or thoughts!
UPDATE: Mappings are listed below
GET /stories/storyphrases/_mappings
{
"stories": {
"mappings": {
"storyphrases": {
"dynamic": "strict",
"properties": {
"#timestamp": {
"type": "date",
"format": "date_optional_time"
},
"#version": {
"type": "string"
},
"SubjectCode": {
"type": "string"
},
"VerbCode": {
"type": "string"
},
"LocationCode": {
"type": "string"
},
"BookCode": {
"type": "string"
},
"Route": {
"type": "string"
},
"StoryText": {
"type": "string",
"index": "not_analyzed"
},
"SourceType": {
"type": "string"
},
"host": {
"type": "string"
},
"message": {
"type": "string"
},
"path": {
"type": "string"
}
}
}
}
}
Mick
Well, first off you are executing two different queries here. The first is running in a query context whilst the second is essentially a match_all query executing in a filtered context. If your objective is simply to emulate the first query but by passing a JSON body you will need something like
GET 123.456.0.789:9200/stories/storyphrases/_search
{
"query" : {
"query_string" : {
"query" : "StoryText:'The boy sat quietly'"
}
}
}
To write this simple query using Nest you would use
var searchresults = client.Search<stories>(p => p.QueryString("StoryText:" + inputtext));
or in longer form
var searchresults = client.Search<stories>(p => p
.Query(q => q
.QueryString(qs => qs
.Query("StoryText:" + inputtext)
)
)
);
These both produce the same JSON body and send it to the _search endpoint. Assuming that storyphrases is your Elasticsearch type then you may also wish to include this in your C#.
var searchresults = client.Search<stories>(p => p
.Index("stories")
.Type("storyphrases")
.Query(q => q
.QueryString(qs => qs
.Query("StoryText:" + inputtext)
)
)
);
Having said all that and looking at your filtered query it should do what you expect according to my testing. Is your field definitely not analyzed? Can you post your mapping?
I stacked to build this Mongodb query in C# driver:
{
Location: { "$within": { "$center": [ [1, 1], 5 ] } },
Properties: {
$all: [
{ $elemMatch: { Type: 1, Value: "a" } },
{ $elemMatch: { Type: 2, Value: "b" } }
]
}
}
Something next:
var geoQuery = Query.WithinCircle("Location", x, y, radius);
var propertiesQuery = **?**;
var query = Query.And(geoQuery, propertiesQuery);
Addition:
The above query taken from my another question:
MongoDB: Match multiple array elements
You are welcome to take part in its solution.
Here's how if you want to get that exact query:
// create the $elemMatch with Type and Value
// as we're just trying to make an expression here,
// we'll use $elemMatch as the property name
var qType1 = Query.EQ("$elemMatch",
BsonValue.Create(Query.And(Query.EQ("Type", 1),
Query.EQ("Value", "a"))));
// again
var qType2 = Query.EQ("$elemMatch",
BsonValue.Create(Query.And(Query.EQ("Type", 2),
Query.EQ("Value", "b"))));
// then, put it all together, with $all connection the two queries
// for the Properties field
var query = Query.All("Properties",
new List<BsonValue> {
BsonValue.Create(qType1),
BsonValue.Create(qType2)
});
The sneaky part is that while many of the parameters to the various Query methods expect BsonValues rather than queries, you can create a BsonValue instance from a Query instance by doing something like:
// very cool/handy that this works
var bv = BsonValue.Create(Query.EQ("Type", 1));
The actual query sent matches your original request exactly:
query = {
"Properties": {
"$all": [
{ "$elemMatch": { "Type": 1, "Value": "a" }},
{ "$elemMatch": { "Type": 2, "Value": "b" }}
]
}
}
(I'd never seen that style of $all usage either, but apparently, it sounds like it's just not documented yet.)
While I can confirm that the query you posted works on my machine, the documentation of $all seems to indicate that it shouldn't accept expressions or queries, but only values:
Syntax: { field: { $all: [ <value> , <value1> ... ] }
(The documentation uses <expression> if queries are allowed, compare to $and). Accordingly, the C# driver accepts only an array of BsonValue instead of IMongoQuery.
However, the following query should be equivalent:
{
$and: [
{ "Location": { "$within": { "$center": [ [1, 1], 5 ] } } },
{ "Properties" : { $elemMatch: { "Type": 1, "Value": "a" } } },
{ "Properties" : { $elemMatch: { "Type": 2, "Value": "b" } } }
]
}
Which translates to the C# driver as
var query =
Query.And(Query.WithinCircle("Location", centerX, centerY, radius),
Query.ElemMatch("Properties", Query.And(Query.EQ("Type", 1), Query.EQ("Value", "a"))),
Query.ElemMatch("Properties", Query.And(Query.EQ("Type", 2), Query.EQ("Value", "b"))));