I'm new to LINQ and I need to write a LINQ query that returns each project's grade, called notet also the average of all notes.
Here is the query I have:
`var query = _context.Set<EvaluationResult>()
.Include(x => x.RatingLevel)
.Include(x => x.Skill)
.Include(x => x.Evaluation)
.ThenInclude(y => y.Project)
.ThenInclude(z => z.ProjectRatingLevels)
.ThenInclude(a => a.RatingLevel)
.Include(y => y.Evaluation.Project)
.ThenInclude(y => y.Degree)
.Where(x => x.Evaluation.Project.DegreeId == QueriedDegreeId)
.GroupBy(i => new { project = i.Evaluation.Project })
.Select(g => new
{
project = g.Select(y => y.Evaluation.Project.Label)
.Distinct().FirstOrDefault(),
note = Math.Round(((g.Sum(y => (double)y.Skill.Weight * (double)y.RatingLevel.Rate) /
g.Sum(y => (double)y.RatingLevel.Rate)) * 100) /
(double)g.Key.project.ProjectRatingLevels
.Select(z => z.RatingLevel.Rate)
.Max(), 2, MidpointRounding.AwayFromZero)
});
Here is the result:
[
{
"project": "Projet 1",
"note": 42.86
},
{
"project": "Projet 2",
"note": 41.67
},
{
"project": "Projet 3",
"note": 46.67
}
]
What I want is to add another value, average, which is just the Average of all "note" values, like so (the asterisks are just for emphasis):
[
{
"project": "Projet 1",
"note": 42.86,
**"average": 43.73**
},
{
"project": "Projet 2",
"note": 41.67,
**"average": 43.73**
},
{
"project": "Projet 3",
"note": 46.67,
**"average": 43.73**
}
]
MY PROBLEM
I'm stuck trying to calculate an Average of all the returned notes. I have no idea how to proceed. I tried to add an average key in my Select after note and project that reused the note query, like this:
average = g.Average(Math.Round(((g.Sum(... etc
But this gives me type errors: CS1929 'IGrouping<<anonymous type: Project project>, EvaluationResult>' does not contain a definition for average 'Average'. So I'm at an utter loss of what to do.
Can someone point me in the right direction? Am I missing something, like the need to use a Key or something?
Is your goal to do it all in one query ?
The only possible Resultset, with Aggregate and the Aggregated Items, is a GroupBy.
If you want to aggregate all, you want only one group, so you have to have a fictive key,the same one for each item
So Append:
.GroupBy(x => 1) /* makes one group for all, all have the same key */
.Select(g => new { average = g.Average(x => x.notes), items = g.Select(x => x)});
But this is really Forcing the SQL-Server to do the average. You Manifest the items in your memory anyway. So you can also take your existing Resultset, manifested with ToList or ToArray and just compute
var result = <yourquery>.ToList();
double average = result.Average(x => x.notes);
The only difference is, this is done on your CPU, not on the SQL-Server.
Related
Let consider this JSON
{
"data": "014",
"theme": "COLORADO CASUAL",
"family": "2163",
"category": "00",
"compo_groups": [
{
"title": "HEAD024",
"values": [
{
"perc": "100",
"desc": "COMP036"
}
]
},
{
"title": "HEAD035",
"values": [
{
"perc": "100",
"desc": "COMP042"
},
{
"perc": "50",
"desc": "COMP043"
}
]
}
],
"product_name": "D812",
"supplier_code": "1011"
}
I need to check that all my compositions are exactly 100pc. In this JSON I have 2 group of composition. The first one is correct. I have one element to 100pc. The second one is composed by 2 elements and total is 150pc. This is an error.
I need to write a code in C# that detect the error. I can write most part of this code. I just don't know how to transform this JSON in list of values I can manage with LinQ.
Assuming you are using a recent version of .NET (e.g. .NET6) then you can use the built-in System.Text.Json libraries to parse your JSON input. If you need to use other parts of the JSON, I would recommend deserialising in to concrete C# classes so you get proper validation, IntelliSense and all that good stuff.
However, if you simply want to check those percentages you can use the STJ library directly, something like this for example:
// Load JSON
var json = "{...}";
var doc = JsonDocument.Parse(json);
// Manually walk the document to get the values you need and summarise
var result = doc.RootElement
.GetProperty("compo_groups")
.EnumerateArray()
.Select(a => new
{
Title = a.GetProperty("title").ToString(),
Percentage = a.GetProperty("values")
.EnumerateArray()
.Select(v => double.Parse(v.GetProperty("perc").ToString()))
.Sum()
});
And you can iterate over that result like this:
foreach(var value in result)
{
Console.WriteLine($"Title '{value.Title}' has a percentage of {value.Percentage}");
}
Which will output this:
Title 'HEAD024' has a percentage of 100
Title 'HEAD035' has a percentage of 150
you don't need any classes to get the data you want
using System.Text.Json;
List<string> titles = JsonNode.Parse(json)["compo_groups"].AsArray()
.Select(x => x["values"].AsArray())
.Where(v => v.Select(x =>
Convert.ToInt32(x["perc"].GetValue<string>())).Sum() > 100)
.Select(v => v.Parent["title"].GetValue<string>())
.ToList(); // result ["HEAD035"]
or
using Newtonsoft.Json;
List<string> titles = JObject.Parse(json)["compo_groups"]
.Select(x => x["values"])
.Where(v => v.Select(x => (int)x["perc"]).Sum() > 100)
.Select(v => v.Parent.Parent)
.Select(p=> (string) p["title"]) // here you can select any data you need
.ToList(); // result ["HEAD035"]
I'm using Nest to build a query. I have a class called Event which has a property called Location and a property called Address (a string). So the indices look something like this:
"id" : "something1",
"location": {
"address": "The Club",
}
"id" : "something2",
"location": {
"address": "The Hole",
}
I want to create a query where the user types "The Club" (the location variable) and it only retrieves the first item that has "The Club" as the address
This is my current query
var skip = (page - 1) * rows;
var searchResponse = await _elasticClient.SearchAsync<Event>(s => s
.From(skip)
.Size(rows)
.Query(q => q
.QueryString(qs => qs
.Fields(fields => fields
.Field(f => f.Location.Address)
)
.Analyzer("keyword")
.Query("*" + (location ?? string.Empty) + "*")
)
));
Currently, the query retrieves both objects since it finds "The". Do I need to add something to my index previously or what should I do?
For the record I added this when creating the index but it has no effect
client.Indices.Create(eventsIndex, c => c.Map<Event>(m => m
.Properties(p => p
.Text(t => t
.Name(n => n.Location.Address)
.Analyzer("keyword"))
)
));
I have two suggestions:
1 - Add the stop filter (default language english) that way you won't produce tokens like the term "the", all stopwords will be removed. Now you will only match the term "hotel", the term "the" will be filtered out.
POST _analyze
{
"tokenizer": "standard",
"filter": ["stop"],
"text": ["the hotel"]
}
Token
{
"tokens": [
{
"token": "hotel",
"start_offset": 4,
"end_offset": 9,
"type": "<ALPHANUM>",
"position": 1
}
]
}
2 - Use the default_operator AND in Query String query.
The user who calls my Rest API must be able to specify the type of place and then have a placename suggestion.
The placetype must be exact while the placename search uses all the potential of the elaticsearch search.
I'm using NEST (latest nuget version) and Elastic 6.4.
My Api has 2 parameters:
1. query: search text search
2. placetypes: array to define the desired "categories" of documents in the suggestion
The placename suggestion with the Query parameter works well,
but I do not know how to add the condition of placetype.
Nest Mapping :
return map
.Dynamic(false)
.Properties(props => props
.Keyword(n => n
.Name(p => p.Id))
.Text(n => n
.Name(p => p.PlaceType))
.Completion(n => n
.Name(p => p.PlaceName)
.Analyzer("autocompletion_indexation"))
.Completion(n => n
.Name(p => p.Address)
.Analyzer("autocompletion_indexation"))
.GeoPoint(loc =>
{
loc.Name(location => location.Coordinates);
return loc;
}));
Nest AnalysisDescriptor :
return analysis
.CharFilters(c => c
.HtmlStrip("html_strip")
)
.Tokenizers(t => t
.EdgeNGram("custom_ngram", descriptor =>
{
descriptor.MinGram(2);
descriptor.MaxGram(10);
descriptor.TokenChars(new List<TokenChar> { TokenChar.Letter, TokenChar.Digit });
return descriptor;
}
))
.TokenFilters(tf => tf
.Lowercase("lowercase")
.WordDelimiter("word_delimiter", wd =>wd
.SplitOnNumerics()
.SplitOnCaseChange()
)
.AsciiFolding("asciifolding", af => af
.PreserveOriginal(false)
)
.Elision("elision", e => e
.Articles("l", "d", "o")
)
.Synonym("address_synonym", sy => sy
.Synonyms(GetSynonyms())
.Tokenizer("standard")
.Tokenizer("whitespace")
)
.Stop("french_stop", fs => fs
.StopWords("_french_"))
.Stemmer("french_stemmer", fs => fs
.Language("light_french")
)
)
.Analyzers(an => an
.Custom("autocompletion_indexation", c => c
.Tokenizer("custom_ngram")
.Tokenizer("standard")
.Tokenizer("whitespace")
.CharFilters("html_strip")
.Filters("address_synonym",
"lowercase",
"asciifolding",
"elision",
"word_delimiter",
"stop",
"french_stemmer",
"french_stop")
));
Suggest/Search function:
public Task<List<Place>> SuggestDocuments(CancellationToken cancellationToken, string query, params string[] placeTypes)
{
var search = new SearchDescriptor<Place>()
.From(0)
.Size(10)
.Index(PlaceDataService.DefaultPostalAddressIndexName)
.Query(q => q
.MultiMatch(mm => mm
.Query(query)
.Fuzziness(Fuzziness.Auto)
.Fields(fields => fields
.Field(f => f.PlaceName)
)));
var searchResults = _elasticClient.SearchAsync<Place>(search, cancellationToken);
return Task.Run(() => searchResults.Result.Documents.ToList(), cancellationToken);
}
You can combine the multi_match query with a terms query, by wrapping them in a bool query to achieve this
var query = "this is the place name";
var places = new [] { "Place 1", "Place 2" };
var search = new SearchDescriptor<Place>()
.From(0)
.Size(10)
.Index("index")
.Query(q => q
.MultiMatch(mm => mm
.Query(query)
.Fuzziness(Fuzziness.Auto)
.Fields(fields => fields
.Field(f => f.PlaceName)
)
) && +q
.Terms(t => t
.Field(f => f.PlaceType)
.Terms(places)
)
);
var searchResults = await _elasticClient.SearchAsync<Place>(search, cancellationToken);
The above makes use of operator overloading for queries, which you can read more about in the Writing Bool queries docs. The above produces the following request
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "this is the place name",
"fuzziness": "AUTO",
"fields": [
"placeName"
]
}
}
],
"filter": [
{
"terms": {
"placeType": [
"Place 1",
"Place 2"
]
}
}
]
}
}
}
Note that a terms query will OR the terms so that only at least one needs to match. If all inputs need to match, you can and multiple term queries together.
A few things don't look quite right with your mapping and analysis
PlaceName is mapped as a completion datatype but is being used in a multi_match search query, rather than a suggest search. I would have expected it to be mapped as a text datatype for this case.
If PlaceType fields need to be matched exactly as is, I would expect them to be mapped as keyword datatype
html_strip char filter is built in, so doesn't need to be re-specified, unless you need to customise how it works e.g. escaped_tags. Similarly for lowercase token filter.
The "autocompletion_indexation" custom analyzer can only have one tokenizer, so the last assigned wins ("whitespace" tokenizer). Check out the writing analyzers docs.
I have the query shown below:
var queryResult =
await
elastic.SearchAsync<CounterData>(s => s
.Query(q => q
.Bool(b => b
.Must(
m => m.ConstantScore(c => c
.Filter(f => f
.Term(x => x.CounterId, maxList))
),
m => m.ConstantScore(c => c.
Filter(f => f
.Term(x => x.Type, counterType))),
m => m.ConstantScore(c => c.
Filter(f => f.
DateRange(d => d.
GreaterThanOrEquals(lowerBound).Field(r => r.Date)))))))
.AllTypes()
.Scroll("1m")
.Size(10000));
Where maxList is a list of integers. I want to check if the term is in the list but looks like this does not work.
Any ideas how I can check if the term matches any of the elements in the list?
Something like the following will do it
var maxList = new[] { 1, 2, 3, 4, 5};
var counterType = "counter-type";
var lowerBound = DateTime.UtcNow.Date.AddDays(-7);
var queryResult = client.Search<CounterData>(s => s
.Query(q => +q
.DateRange(d => d
.Field(r => r.Date)
.GreaterThanOrEquals(lowerBound)
) && +q
.Term(x => x.Type, counterType) && +q
.Terms(t => t
.Field(x => x.CounterId)
.Terms(maxList)
)
)
.AllTypes()
.Scroll("1m")
.Size(10000)
);
A few things to highlight
+ unary operator applied to a QueryContainerDescriptor<T> is a shorthand for wrapping a query in a bool filter query. I think this is what you want in your case as you don't need to calculate scores, you just want to find matches to a predicate.
&& is overloaded for QueryContainer such that when applied to two QueryContainers, it is a shorthand for a bool must query with two must query clauses. However in this example, the queries all have the + unary operator applied so are bool filter queries, so will be &&'ed together as filter queries.
The value passed to Size() when using Scrolling (i.e. specifying a Scroll() time) is the number of documents to fetch from each shard per scroll, not total documents per scroll. So total documents will be Size() * number of shards. This might be a lot of documents per scroll.
Use the terms query to find documents that match on a field against any one of a list of terms (not analyzed).
The end query json looks like
POST http://localhost:9200/examples/_search?scroll=1m
{
"size": 10000,
"query": {
"bool": {
"filter": [
{
"range": {
"date": {
"gte": "2016-08-04T00:00:00Z"
}
}
},
{
"term": {
"type": {
"value": "counter-type"
}
}
},
{
"terms": {
"counterId": [
1,
2,
3,
4,
5
]
}
}
]
}
}
}
I'm developing an .NET application using Elastic Search. I used ES River to index the data.
Results (in Sense) look kinda like this:
{
"_index": "musicstore",
"_type": "songs",
"_id": "J2k-NjXjRa-mgWKAq0RMlw",
"_score": 1,
"_source": {
"songID": 42,
"tempo": "andante",
"metrum": "3/4 E8",
"intonation": "F",
"title": "Song",
"archiveSongNumber": "3684",
"Year": 2000,
"Place": "London"
}
},
To access the indexed data I use NEST queries similar to this:
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.title, "Song")));
I'm having a problem that the query doesn't return any results, when I search for a certain field.
For instance when I search for a title, songID, tempo or archiveSongNumber the query works fine and it returns the same results as Sense, but when I search for Year, Place, metrum, etc. the query doesn't return any results, but it should (Sense does and it should).
Queries like these work (and return the right results):
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.title, "Song")));
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.songID, 42)));
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.archiveSongNumber , "3684")));
Queries like these don't return any results (but they should):
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.Place, "London")));
var result = ElasticClient.Search<Song>(s => s.Query(q => q.Term(p => p.Year, 2000)));
What am I doing wrong? Did I mess up when I was indexing data?
Update:
Mapping looks like this:
{
"musicstore": {
"mappings": {
"songs": {
"properties": {
"Year": {
"type": "long"
},
"Place": {
"type": "string"
},
"archiveSongNumber": {
"type": "string"
},
"songID": {
"type": "long"
},
"intonation": {
"type": "string"
},
"metrum": {
"type": "string"
},
"title": {
"type": "string"
},
"tempo": {
"type": "string"
}
}
}
}
}
}
Update 2:
ES river request looks like this:
PUT /_river/songs_river/_meta
{
"type":"jdbc",
"jdbc": {
"driver":"com.microsoft.sqlserver.jdbc.SQLServerDriver",
"url":"jdbc:sqlserver://ip_address:1433;databaseName=database",
"user":"user",
"password":"password",
"strategy":"simple",
"poll":"300s",
"autocommit":true,
"fetchsize":10,
"max_retries":3,
"max_retries_wait":"10s",
"index":"musicstore",
"type":"songs",
"analysis": {
"analyzer" :{
"whitespace" :{
"type" : "whitespace",
"filter":"lowercase"
}
}
},
"sql":"some_sql_query"
}
}
ES client configuration looks like this:
private static ElasticClient ElasticClient
{
get
{
Uri localhost = new Uri("http://localhost:9200");
var setting = new ConnectionSettings(localhost);
setting.SetDefaultIndex("musicstore").MapDefaultTypeNames(d => d.Add(typeof(Song), "songs"));
setting.SetConnectionStatusHandler(c =>
{
if (!c.Success)
throw new Exception(c.ToString());
});
return new ElasticClient(setting);
}
}
From looking at your mapping, the issue here is most likely that all of your fields are being analyzed when indexing, but you are using term queries with NEST, which are not analyzed, meaning they will only find exact matches. If you don't explicitly specify an analyzer in your mappings, Elasticsearch defaults to the standard analyzer.
When you perform a search in Elasticsearch using a query string, like you're doing in Sense: GET _search/?q=Place:London, a query string query is what's being run by Elasticsearch, which is different than a term query.
From your examples though, it doesn't look like you are actually using query string syntax. You probably want a match query instead:
client.Search<Song>(s => s
.Query(q => q
.Match(m => m
.OnField(p => p.Place)
.Query("London")
)
)
);
If you do however want a query string query like the one you're performing with Sense, than you can use QueryString:
client.Search<Song>(s => s
.Query(q => q
.QueryString(qs => qs
.OnFields(p => p.Place)
.Query("London")
)
)
);
Hope that helps. I suggest checking out the getting started guide, specifically the section on exact values vs. full text.
Add "keyword" suffix to your Term Field :
var result = ElasticClient.Search<Song>(s => s
.Query(q => q
.Term(p => p
.Field(x => x.Year.Suffix("keyword")).Value(2000))));
Try it, it will work!