enter image description hereI am doing an aggregation on a field with the a guid value like c4b0c9ae-345c-4247-87e3-e9d9de67c01b but when elastic search returns the aggregations
it is becoming an array ["c4b0c9ae","345c","4247","87e3","e9d9de67c01b"].
How do you handle this on nest?
The field has been mapped as a text datatype which undergoes analysis at index time, by default using the Standard Analyzer. The result is that the GUID is tokenized into the constituent parts by splitting on hyphens. You can see this for yourself using the Analyze API (in Kibana Console)
GET _analyze
{
"analyzer": "standard",
"text": ["c4b0c9ae-345c-4247-87e3-e9d9de67c01b"]
}
yields
{
"tokens": [
{
"token": "c4b0c9ae",
"start_offset": 0,
"end_offset": 8,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "345c",
"start_offset": 9,
"end_offset": 13,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "4247",
"start_offset": 14,
"end_offset": 18,
"type": "<NUM>",
"position": 2
},
{
"token": "87e3",
"start_offset": 19,
"end_offset": 23,
"type": "<ALPHANUM>",
"position": 3
},
{
"token": "e9d9de67c01b",
"start_offset": 24,
"end_offset": 36,
"type": "<ALPHANUM>",
"position": 4
}
]
}
You probably don't want to analyze GUIDs at index time, so you should explicitly map them as keyword datatype.
Related
I need help with expression for sorting both ascending and descending for the same data.
I can have multiple expression separated by pipe but i can't seem to find any example that does both ASC and DESC in the same call.
So what i am looking for is equal to following sql query.
select *
from dummyData
order by id, code desc, update_only
Here is sample.
[
{
"id": 1,
"code": "y",
"update_only": 0.0
},
{
"id": 2,
"code": "a",
"update_only": 0.0
},
{
"id": 1,
"code": "z",
"update_only": 0.0
},
{
"id": 2,
"code": "b",
"update_only": 0.0
}
]
I can do following for the ordering
sort_by(array elements, expression->number|expression->string expr)
How can i do the following with one call rather than the two calls that i have?
sort_by(myarray, &id | &update_only)[];
reverse(sort_by(myarray, &code))[];
Or by doing multiple calls, don't want to do this as well.
result1 = sort_by(myarray, &id)[];
result2 = reverse(sort_by(result1, &code))[];
resilt3 = sort_by(myarray, &update_only)[];
Your actual expression is incorrect, you will realise, if you change a value of update_only that &id | &update_only is not a thing.
What is described in the documentation:
expression->number|expression->string expr
Is actually meaning that the expression can either be of type string or of type number, not that you can chain multiple sorting using the pipe sign.
An explanation of this is made here:
JMESPath has various built-in functions that operate on different data types, documented below. Each function below has a signature that defines the expected types of the input and the type of the returned output:
return_type function_name(type $argname)
return_type function_name2(type1|type2 $argname)
The list of data types supported by a function are:
number (integers and double-precision floating-point format in JSON)
string
boolean (true or false)
array (an ordered, sequence of values)
object (an unordered collection of key value pairs)
null
expression (denoted by &expression)
With the exception of the last item, all of the above types correspond to the types provided by JSON.
If a function can accept multiple types for an input value, then the multiple types are separated with |. If the resolved arguments do not match the types specified in the signature, an invalid-type error occurs.
Source: https://jmespath.org/specification.html#built-in-functions, emphasis, mine
Now, what want to achieve is pretty simple.
When you are sorting on multiple columns, it is like chaining the sorts, from the lower priority to the upper priority.
So that means that you can do (in pseudo code):
sort by id (
reverse (
sort by code (
reverse (
// ^--- this is the trick, so when you will reverse it
// again, to have the DESC sorting on `code`, you'll end
// up with the sorting of `update_only` in the "correct" order
sort by update_only
)
)
)
)
So, translated in a JEMSPath expression:
sort_by(reverse(sort_by(reverse(sort_by(#,&update_only)),&code)),&id)
This expression on the following JSON (added an extra case from your example to showcase the update_only sorting:
[
{
"id": 1,
"code": "y",
"update_only": 0.0
},
{
"id": 2,
"code": "a",
"update_only": 0.0
},
{
"id": 1,
"code": "z",
"update_only": 0.0
},
{
"id": 2,
"code": "b",
"update_only": 1.0
},
{
"id": 2,
"code": "b",
"update_only": 0.0
}
]
Will give:
[
{
"id": 1,
"code": "z",
"update_only": 0
},
{
"id": 1,
"code": "y",
"update_only": 0
},
{
"id": 2,
"code": "b",
"update_only": 0
},
{
"id": 2,
"code": "b",
"update_only": 1
},
{
"id": 2,
"code": "a",
"update_only": 0
}
]
The Fluent Assertions library puts quite some emphasis on its capabilities of comparing object graphs by means of the Should().BeEquivalentTo method and related methods. As the documentation points out:
Should().BeEquivalentTo is a very powerful feature, and one of the unique selling points of Fluent Assertions.
From reading the examples, I am getting the impression that the object graph has to be connected by means of object references.
Now, I have plenty of object graphs and trees whose content I would like to check, but they are usually loaded from files or from databases.
Therefore, the connection exists solely by ID property values that match across objects.
For instance, imagine this simple structure (here written in JSON format) - example A:
[{
"Id": 1,
"Name": "Foo",
"NextId": 2
}, {
"Id": 2,
"Name": "Bar"
}]
I would like to use Fluent Assertions to check the structure of such an object graph.
That is, no matter what values the fields Id and NextId actually have, NextId in the first object must have the same value as Id in the second object.
Thus, any set of two objects with two different Id values, where one object has a NextId value that equals the Id value of the other object, will do.
Example B:
[{
"Key": 5,
"LinkedTo": [3]
}, {
"Key": 10,
"LinkedTo": []
}, {
"Key": 3,
"LinkedTo": [5]
}]
Here, any set of three objects with three different Key values, where two objects refer to each other's key in their LinkedTo property and the remaining object has an empty array in its LinkedTo property should match.
Example C:
[{
"Id": 1,
"Owner": 4
}, {
"Id": 2,
"Owner": 4
}, {
"Id": 3,
"Owner": 4
}, {
"Id": 4
}]
Here, any set of four objects with three different Id values will match if three of those objects have an Owner property whose value matches the Id property of the remaining object.
Can this somehow be done with BeEquivalentTo?
Note that I do not want to walk through my objects and add actual object references before applying any assertions, because that operation in itself might introduce bugs of its own.
EDIT: As requested, here are a couple of matching and non-matching graphs for the above examples:
Example A:
Match:
[{
"Id": 5,
"Name": "Foo",
"NextId": -8
}, {
"Id": -8,
"Name": "Bar"
}]
Match:
[{
"Id": 200,
"Name": "Bar"
}, {
"Id": 30,
"Name": "Foo",
"NextId": 200
}]
Mismatch:
[{
"Id": 3,
"Name": "Bar",
"NextId": 6
}, {
"Id": 6,
"Name": "Foo"
}]
Example B:
Match:
[{
"Key": 20,
"LinkedTo": [7]
}, {
"Key": 8,
"LinkedTo": []
}, {
"Key": 7,
"LinkedTo": [20]
}]
Match:
[{
"Key": 9,
"LinkedTo": [100]
}, {
"Key": 100,
"LinkedTo": [9]
}, {
"Key": 3,
"LinkedTo": []
}]
Mismatch:
[{
"Key": 5,
"LinkedTo": [10]
}, {
"Key": 10,
"LinkedTo": []
}, {
"Key": 3,
"LinkedTo": [5]
}]
Example C:
Match:
[{
"Id": 1
}, {
"Id": 2,
"Owner": 1
}, {
"Id": 3,
"Owner": 1
}, {
"Id": 4,
"Owner": 1
}]
Match:
[{
"Id": 10,
"Owner": 20
}, {
"Id": 30,
"Owner": 20
}, {
"Id": 20
}, {
"Id": 40,
"Owner": 20
}]
Mismatch:
[{
"Id": 8,
"Owner": 2
}, {
"Id": 12,
"Owner": 8
}, {
"Id": 54,
"Owner": 2
}, {
"Id": 2
}]
As I understand your question, you have a list of objects and want to assert that NextId and Id of each consecutive pair matches.
class MyClass
{
public MyClass(int id, int nextId, string name)
{
Id = id;
NextId = nextId;
Name = name;
}
public int Id { get; set; }
public int NextId { get; set; }
public string Name { get; set; }
}
Fluent Assertions does not know about consecutiveness, instead we can create two lists such that each matching pair across the lists corresponds to consecutive items in the original list.
We can now compare the two lists to see if each pair across the lists has the desired relationship.
To compare the two lists by NextId and Id we need to instruct Fluent Assertions how to compare two instances of MyClass.
For BeEquivalentTo you can specify how to compare to instances of a given type using the Using<T>() + WhenTypeIs<T>() combination.
Note that per default will try to match the two lists in any order, but by specifying WithStrictOrdering() it will be pairwise comparison.
[TestMethod]
public void Lists_BeEquivalentTo()
{
var objects = new[]
{
new MyClass(1, 2, "Foo"),
new MyClass(2, 3, "Bar"),
new MyClass(3, 4, "Baz")
};
objects.Take(objects.Length - 1).Should().BeEquivalentTo(objects.Skip(1), opt => opt
.WithStrictOrdering()
.Using<MyClass>(e => e.Subject.NextId.Should().Be(e.Expectation.Id))
.WhenTypeIs<MyClass>());
}
If the comparison is as simple as comparing two ints, one could also use the Equal() assertion which just takes two lists and a predicate that specifies when two elements are equal.
[TestMethod]
public void Lists_Equal()
{
var objects = new[]
{
new MyClass(1, 2, "Foo"),
new MyClass(2, 3, "Bar"),
new MyClass(3, 4, "Baz")
};
objects.Take(objects.Length - 1).Should().Equal(objects.Skip(1),
(a, b) => a.NextId == b.Id);
}
I have a web application where each user also has GPS coordinates (SqlGeography). As of now, I calculate the distance between two GPS coordinates as Euclidean distance with the Distance function.
Some users have requested that I replace the approximated distance (in terms of air-line distance) with a more realistic distance. I'm thinking about using specific routing APIs (e.g., from Google, Here or Azure) to get a more realistic distance, for instance for a car ride.
Currently, I have approximately 5000 different GPS coordinates and I would like to calculate and store a pair-wise distance for all of these pairs. This would result in 5000 * 5000 queries / requests which is very expensive. Is there any way how I can reduce the quadratic complexity with a trick I haven't thought of?
Some APIs have an option to add waypoints/stops between start and end of a route( i.e. Sygic web API.
Maybe Google and Co. do something similar.
Response includes legs property, representing route/duration from start to waypoint1, wapoint1 to waypointN, waypointN to end of route ... etc.
{
"routes": [
{
"route": "oupjIa}w|#^ErCUPELEf##`A[nAjGlBnFfBtERl#~DdNdA~DdBjG|BhKfArELbAd#`EJdFKxF?rBVfKF~BFjH#jEChJC`DMtLCzEApB#bIRdOZlGdAtLr#tErAxGnIj\\fBnIrBxMr#|Gh#fHf#jJPvFDvAHxBBf#X|EA`Br#zLd#|Dz#hF`AzElBhIvClMnA|Ib#dGd#bKb#rYBbBJ|CDxGFjEx#tm#N`ID|CF`PStMI|B[`Hi#vIcA`KkC|PcBtIoBlIaF`RgN~c#oGpTqGrWaFtWyEd\\mAlKcBpQkBbX{#~PcAlYG`CClAEdBAj#KdEe#z`#AlBCjDC|XJ~WTrTh#rWZ`Pf#tUTfK`#fQ|#d]v#|O|BpYhBpOrE`\\bC`RDZf#fEvBlULtBZfFJhBNxCBZn#xQ\\~TZb`#JxQ#jNO~PeAfa#mDzdAo#jOsArSmDx_#eAnK}AlRs#zNg#rRChUB|ENzKZnR~B~tAvBthAv#nUtAdWbCl\\p[zbDrFhh#nDvZdB`Mb#~C|#xGrJ~n#fIzb#pBlJv#lDzAxGrDtOzEnQ|Rdr#|FzU|ArH|DfU~ArLx#xG^dDtBbQb#nDhAtIFf#\\`CZvBDXbAbHDXbBvKv#`E|BfJd#~BVlAJh#tA~Gb#tB\\~AtDbPrB~IxB~IzB`JvCdLzAdGjAvELb#Nn#|#xDR~#r#jDzAjI`#hCT~AhDbVF\\fAjIlBbOrAlKjAtIr#lFz#nG#JJv#v#`G`#|Cd#jDDhAJp#D^TzB#ZBpCStEYfB_#pAeAlCINo#~#oBbBoCrAaFfB_H`BsGhA}Ch#gMrB}Bd#oAR_APyOlCwAV}EdASF]JcErA_BXe#G]w#EsBNu#l#yBZe#j#]f#GjBTx#JXF#kApAc]ImEO_AWq#uAkAgAe#eCyA_CoDm#s#cA[eCM?mIYiBsAcEy#uBcA}A]MeAGu#?e#VsEVw#Ho#JiBHwAEa#SaAKy#Cs#PuAlAc#r#sChE{CjFkA`D}AbGIj#y#bEW`Ai#rAw#tA",
"eta": 1531834797,
"duration": {
"value": 1713,
"text": "28 minutes 33 seconds"
},
"distance": {
"value": 36430,
"text": "36.43 km"
},
"legs": [
{
"distance": {
"value": 36430,
"text": "36.43 km"
},
"duration": {
"value": 1713,
"text": "28 minutes 33 seconds"
},
"start_location": {
"latitude": 54.32168,
"longitude": 10.12193
},
"end_location": {
"latitude": 54.30731,
"longitude": 9.66195
},
"route": "oupjIa}w|#^ErCUPELEf##`A[nAjGlBnFfBtERl#~DdNdA~DdBjG|BhKfArELbAd#`EJdFKxF?rBVfKF~BFjH#jEChJC`DMtLCzEApB#bIRdOZlGdAtLr#tErAxGnIj\\fBnIrBxMr#|Gh#fHf#jJPvFDvAHxBBf#X|EA`Br#zLd#|Dz#hF`AzElBhIvClMnA|Ib#dGd#bKb#rYBbBJ|CDxGFjEx#tm#N`ID|CF`PStMI|B[`Hi#vIcA`KkC|PcBtIoBlIaF`RgN~c#oGpTqGrWaFtWyEd\\mAlKcBpQkBbX{#~PcAlYG`CClAEdBAj#KdEe#z`#AlBCjDC|XJ~WTrTh#rWZ`Pf#tUTfK`#fQ|#d]v#|O|BpYhBpOrE`\\bC`RDZf#fEvBlULtBZfFJhBNxCBZn#xQ\\~TZb`#JxQ#jNO~PeAfa#mDzdAo#jOsArSmDx_#eAnK}AlRs#zNg#rRChUB|ENzKZnR~B~tAvBthAv#nUtAdWbCl\\p[zbDrFhh#nDvZdB`Mb#~C|#xGrJ~n#fIzb#pBlJv#lDzAxGrDtOzEnQ|Rdr#|FzU|ArH|DfU~ArLx#xG^dDtBbQb#nDhAtIFf#\\`CZvBDXbAbHDXbBvKv#`E|BfJd#~BVlAJh#tA~Gb#tB\\~AtDbPrB~IxB~IzB`JvCdLzAdGjAvELb#Nn#|#xDR~#r#jDzAjI`#hCT~AhDbVF\\fAjIlBbOrAlKjAtIr#lFz#nG#JJv#v#`G`#|Cd#jDDhAJp#D^TzB#ZBpCStEYfB_#pAeAlCINo#~#oBbBoCrAaFfB_H`BsGhA}Ch#gMrB}Bd#oAR_APyOlCwAV}EdASF]JcErA_BXe#G]w#EsBNu#l#yBZe#j#]f#GjBTx#JXF#kApAc]ImEO_AWq#uAkAgAe#eCyA_CoDm#s#cA[eCM?mIYiBsAcEy#uBcA}A]MeAGu#?e#VsEVw#Ho#JiBHwAEa#SaAKy#Cs#PuAlAc#r#sChE{CjFkA`D}AbGIj#y#bEW`Ai#rAw#tA",
"eta": 1531834797
}
]
}
],
"status": "OK",
"copyright": "© 2018 Sygic a.s."
}
This question already has answers here:
How can I parse a JSON string that would cause illegal C# identifiers?
(3 answers)
Create a strongly typed c# object from json object with ID as the name
(1 answer)
Closed 5 years ago.
I might be silly, but I'm stuck on this one.
I'm coding in c# and attempting to parse a JSON string into an object.
The problem is that the string contains a data-field which I call a pseudo-array. It isn't a real array, because it is missing the []'s....
I really don't want to create an object with 500 properties just to cater for what I assume to be a bug in the API's JSON response - but how on earth do I deserialize this into c# objects?
Below is the sample JSON from the API's documentation page.
(I don't own the API - so I can't change the JSON)
Any advice will be appreciated.
Thanks
k
`
{
"data": {
"71489": { <---- This is supposed to be an item in an array - but it isn't
"air_by_date": 0,
"cache": {
"banner": 1,
"poster": 1
},
"language": "en",
"network": "USA Network",
"next_ep_airdate": "",
"paused": 0,
"quality": "HD720p",
"show_name": "Law & Order: Criminal Intent",
"status": "Ended",
"tvdbid": 71489,
"tvrage_id": 4203,
"tvrage_name": "Law & Order: Criminal Intent"
},
"140141": {
"air_by_date": 0,
"cache": {
"banner": 0,
"poster": 0
},
"language": "fr",
"network": "CBS",
"next_ep_airdate": "2012-01-15",
"paused": 0,
"quality": "Any",
"show_name": "Undercover Boss (US)",
"status": "Continuing",
"tvdbid": 140141,
"tvrage_id": 22657,
"tvrage_name": "Undercover Boss"
},
...
"194751": {
"air_by_date": 1,
"cache": {
"banner": 1,
"poster": 1
},
"language": "en",
"network": "TBS Superstation",
"next_ep_airdate": 2011-11-28",
"paused": 0,
"quality": "Custom",
"show_name": "Conan (2010)",
"status": "Continuing",
"tvdbid": 194751,
"tvrage_id": 0,
"tvrage_name": ""
},
"248261": {
"air_by_date": 0,
"cache": {
"banner": 1,
"poster": 1
},
"language": "en",
"network": "Cartoon Network",
"next_ep_airdate": "",
"paused": 1,
"quality": "HD",
"show_name": "NTSF:SD:SUV::",
"status": "Continuing",
"tvdbid": 248261,
"tvrage_id": 28439,
"tvrage_name": "NTSF:SD:SUV"
}
},
"message": "",
"result": "success"
}
`
The data property of your JSON object is an associative array. You need to access each element by the appropriate key (in this case a numeric string)
// convert from a String into a JObject
var data = JObject.Parse(json);
// access single property
Response.Write(data["data"]["71489"]);
Response.Write(data["data"]["140141"]);
// iterate all properties
foreach (JProperty prop in data["data"])
{
Response.Write(prop.Name + ": " + prop.Value);
// Also possible to access things like:
// - prop.Value["air_by_date"]
// - prop.Value["cache"]["banner"]
// - prop.Value["cache"]["poster"]
// - prop.Value["language"]
}
I've been using a service that returns a JSON file with an array inside and object in an array.
The problem with the second layer array is that I can't seem to use it.
The JSON looks something like this:
{
"section": "U.S.",
"subsection": "Politics",
"title": "To Understand Clinton’s Moment, Consider That It Came 32 Years After Ferraro’s",
"abstract": "A look back to when Geraldine A. Ferraro was on the Democratic ticket in 1984 can tell a lot about how the country has changed, and how it has not.",
"url": "urlLocation",
"byline": "By ALISON MITCHELL",
"item_type": "Article",
"updated_date": "2016-06-11T13:17:01-04:00",
"created_date": "2016-06-11T13:17:05-04:00",
"published_date": "2016-06-12T00:00:00-04:00",
"material_type_facet": "",
"kicker": "",
"des_facet": [
"Presidential Election of 2016",
"Women and Girls"
],
"org_facet": [],
"per_facet": [
"Clinton, Hillary Rodham",
"Ferraro, Geraldine A"
],
"geo_facet": [],
"multimedia": [
{
"url": "urlLocation",
"format": "Standard Thumbnail",
"height": 75,
"width": 75,
"type": "image",
"subtype": "photo",
"caption": "Geraldine A. Ferraro at a hearing of the Democratic national convention’s platform committee in 1984, the year she became the first woman selected to be a major party’s vice presidential nominee.",
"copyright": "George Tames/The New York Times"
},
{
"url": "urlLocation",
"format": "thumbLarge",
"height": 150,
"width": 150,
"type": "image",
"subtype": "photo",
"caption": "Geraldine A. Ferraro at a hearing of the Democratic national convention’s platform committee in 1984, the year she became the first woman selected to be a major party’s vice presidential nominee.",
"copyright": "George Tames/The New York Times"
},
{
"url": "urlLocation",
"format": "Normal",
"height": 127,
"width": 190,
"type": "image",
"subtype": "photo",
"caption": "Geraldine A. Ferraro at a hearing of the Democratic national convention’s platform committee in 1984, the year she became the first woman selected to be a major party’s vice presidential nominee.",
"copyright": "George Tames/The New York Times"
},
{
"url": "urlLocation",
"format": "mediumThreeByTwo210",
"height": 140,
"width": 210,
"type": "image",
"subtype": "photo",
"caption": "Geraldine A. Ferraro at a hearing of the Democratic national convention’s platform committee in 1984, the year she became the first woman selected to be a major party’s vice presidential nominee.",
"copyright": "George Tames/The New York Times"
},
{
"url": "urlLocation",
"format": "superJumbo",
"height": 1364,
"width": 2048,
"type": "image",
"subtype": "photo",
"caption": "Geraldine A. Ferraro at a hearing of the Democratic national convention’s platform committee in 1984, the year she became the first woman selected to be a major party’s vice presidential nominee.",
"copyright": "George Tames/The New York Times"
}
],
"short_url": "urlLocation"
}
I tried accessing it like this:
NewsArticle newsArticle = new NewsArticle();
newsArticle.category = data.results[counter].section;
newsArticle.title = data.results[counter].title;
newsArticle.fullText = data.results[counter].#abstract;
DynamicJsonArray multimedia = data.results[counter].multimedia;
newsArticle.image = multimedia[0].url;
return_value.Add(newsArticle);
I get an "Index was outside the bounds of the array." exception on the "multimedia" array. Is there something wrong with my syntax?
There seemed to be a problem in the return value of the service. The first result did not contain that array, that was why I got the index out of bounds exception.
Thanks Orel Eraki!