C# elastic search query, match multiple queries - c#

I have 2 elastic search queries that need to be matched by each fetched document in different ways. The "pools" query is a terms query. Each document has a list of pools attached to it, each of them being a string, and at least one of those pools must be in the provided list of pools in the "pools" query.
The other query is actually composed of multiple queries and at least 75% percent of them should be matched.
So in order for a document to be matched, the "pools" query must always be matched and from the other query, at least 75% must be matched.
I wrote my query like this:
var matchQuery = BuildQuery(searchCriteria);
var poolQuery = BuildPoolsQueryField(searchCriteria);
// prepare the data for elasticsearch
var result = await _elasticSearchClient.SearchAsync<ElasticPersonEntity>(
p => p.Query(q => q
.Bool(b => b.Must(poolQuery).Should(matchQuery.ToArray())
.MinimumShouldMatch(MinimumShouldMatch.Percentage(75))))).ConfigureAwait(false);
But I could not find anywhere on the internet if you can chain multiple Should and Must clauses and what it happens if you chain them like this.

According to your description, your query is wrong: you need to mustpoolQuery && matchQuery(75%) so
The .MinimumShouldMatch(MinimumShouldMatch.Percentage(75) should be inside your matchQuery :
I join an example (using my data, but this should solve your problem)
.Query(q => q
.Bool(b => b
.Must(
mu => mu.Term(te => te.CntCd, "FR"),
mu => mu.Bool(bo => bo
.Should(your should query).MinimumShouldMatch(75)
)
)
)
)

Related

Trouble with Linq Select Query

I am trying to query my Database for a single record ins three steps and I am having problems getting the result that I am looking for. These are the steps that I created:
client = client
.Where(s => s.CompanyName.Contains(name));
var res = client.Select(x => x.ID);
Tracker = Tracker
.Where(s => s.ClientId.Equals(client.Select(x => x.ID)));
Debugging the code indicated that steps one and two worked correctly and generated the data that I needed to run my third query, which should provide the whole record, utilizing the result of the second step.
The third and last steps generated the following error:
"The LINQ expression DbSet<TimeTrackerViewModel>().Where(t => t.ClientId.Equals(DbSet<ClientsViewModel>().Where(c => c.CompanyName.Contains(__name_0)).Select(c => c.ID))) could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information."
How do I query my database, utilizing the query result from the second step?
If you want to use the IDs in res to filter Tracker then you can use Any():
var res = client.Select(x => x.ID);
trackers = Tracker.Where(s => res.Any(r => r == s.ClientId));
The above query will return a collection.
I am trying to query my Database for a single record
If you want to return a single record then you could to use FirstOrDefault(), either in place of the Where clause (using the same predicate), or after the Where (you could consider Single() if you know there's exactly 1 matching record). But you should also consider what you expect to happen if multiple records match the name parameter in your first query and how you would handle that.
You should try this:
var trakers = (from c in client.Where(s => s.CompanyName.Contains(name))
join t in tracker
on c.ID
equals t.ClientId
select t).ToList();
So you do only a query on db.
After reading several related posts, I was able to combine thier ideas into a single working solution as posted below:
var client = _context.ClientsViewModel
.Where(s => s.CompanyName.Contains(name))
.Select(x => x.ID).ToList();
Tracker = Tracker
.Where(s => s.ClientId == client[0])
.OrderByDescending(x => x.Id);

ElasticSearch (NEST) Query to exclude specific term results

I'm trying to find a QueryContainer query that I can perform on an ElasticSearch result that will essentially filter out any "A" status Items in my resultset. My ProductIndex document contains a field named "StatusCode", and I don't want to return these "A" status' to my search resultsets... I'm having the hardest time finding a way to remove these items.
This query properly finds these "A" status':
.Match(qm => qm
.Field(f => f.StatusCode)
.Query("A"));
But I want to do the opposite (not get all the "A" status items, but exclude them)
Based upon other threads I read on here, I came up with the following query, but it's not filtering out these results:
.Bool(b => b
.MustNot(mn => mn
.Terms(t => t
.Field(f => f.StatusCode)
.Terms("A")
)
));
and
.Bool(b => b
.MustNot(mn => mn
.Term(t => t
.Field(f => f.StatusCode).Value("A")
)
));
But neither removes results that have an "A" statuscode
Kibana value of a result that is still being returned, but has the status "A" code:
The MustNot filter continues to not work for me - I ended up resolving this by performing two search queryies - one based upon the result set obtained from from the first query
.Match(qm => qm
.Field(f => f.StatusCode)
.Query("A"));
This properly filtered the items to only return the ones I was trying to remove from my primary query. I obtained this document list, then applied this list as an exclusion filter to my underlying request, thus filtering out the items I no longer wanted to return.
Not as elegant as I would like, but got the job done.

How to GroupBy data from many tables in dotnet core Entity Framework

I'm building some marketplace web app, let's say something like e-bay. Typical scenario is:
User makes offer which consists of one or more items and those items are of certain type.After that other users are bidding on that offer.
Here is simplified diagram.
On SQL Fiddle (here) you can see both CREATE TABLE and INSERT INTO statements
Sample data:
There are two offers. On one offer (Id 1) which consists of one item which is type of "watch". There is another offer, (Id 2), which has one item which is of type "headphone".
On both offers there are bids. On watch, there are two bis; one bid with 100 dollars and another with 120. On headphones, there are bids with 50 and 80 dollars.
What I want to achieve is to have average bid per type. In this sample, that means i want to get 110 as average bid for watch and 65 as average bid for headphone. To achieve that using T-SQL, I would write query like this:
SELECT t.name,
avg(amount)
FROM bid b
LEFT JOIN offer o ON b.OfferId = o.id
LEFT JOIN offeritem oi ON o.id = oi.OfferId
LEFT JOIN itemType t ON oi.itemtypeid = t.Id
GROUP BY t.name
So, my question is - how to achieve that in dotnet core 3.0 EntityFramework
Using GroupBy, like this:
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => b.Offer.OfferItems.First().ItemType.Name);
gives exception:
Client side GroupBy is not supported.
. When I try with projection, like this:
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => b.Offer.OfferItems.First().ItemType.Name)
.Select(g => new
{
Key = g,
Value = g.Average(b => b.Amount)
});
i get exception again.
Processing of the LINQ .... failed. This may indicate either a bug or
a limitation in EF Core.
EDIT:
This approach
_context.Bids
.Include(b => b.Offer)
.ThenInclude(o => o.OfferItems)
.ThenInclude(os => os.ItemType)
.GroupBy(b => new { b.Offer.OfferItems.First().ItemType.Name}, b => b.Amount)
.Select(g => new
{
Key = g.Key.Code,
Value = g.Average()
});
also threw an exception, but this time:
Cannot use an aggregate or a subquery in an expression used for the
group by list of a GROUP BY clause.
...
So, is there a way to group that data (get simple Average) or should I make another query and iterate throught collection and make calculation myself? That would lower performance for sure (I was hoping I can do server grouping, but as you can see, i got into mentioned issues). Any ideas? Thanks in advance.
In your case it is hard to hide subquery from grouping
You can try it in such way
var joined =
context.Bid
.SelectMany(x =>
x.Offer.OfferItem
.Select(y => new
{
Amount = x.Amount,
Name = y.ItemType.Name
})
.Take(1));
var grouped = from i in joined
group i by i.Name into groups
select new
{
Key = groups.Key,
Amount = groups.Average(x => x.Amount)
};
it gives me a query
SELECT [t].[Name] AS [Key], AVG([t].[Amount]) AS [Amount]
FROM [Bid] AS [b]
INNER JOIN [Offer] AS [o] ON [b].[OfferId] = [o].[Id]
CROSS APPLY (
SELECT TOP(1) [b].[Amount], [i].[Name], [o0].[Id], [i].[Id] AS [Id0], [o0].[OfferId]
FROM [OfferItem] AS [o0]
INNER JOIN [ItemType] AS [i] ON [o0].[ItemTypeId] = [i].[Id]
WHERE [o].[Id] = [o0].[OfferId]
) AS [t]
GROUP BY [t].[Name]

How to search multiple conditions in ElasticSearch

I'm using Nest to connect to ElasticSearch and perform some queries. Until now, I've been able to get the results I'm asking for given a single wildcard query as below:
var searchResults = _client.Search<MyIndexable>(
body =>
body
.Index("someIndex")
.Query(
query =>
query.QueryString(
qs => qs.
OnFields(f => f.Title, f => f.Description)
.Query(searchTerm + "*"))));
But now I need to add additional conditions to my query for a certain usecase.
What I need is not only to query by [searchTerm]* but also two
other fields matching:
searchTerm + "*"
field1 = "some string"
field2 = "some Guid"
I'm sure there must be a way to do this in elastic search (using Nest) but I haven't been able to find it.
I know I can add the two additional fields to the index (as in someIndex_field1_field2) and that is definitely an option for me since field1 and field2 are decent partitions (and in fact, it may be a much better option to allow better partitioning) but I just wanted to see if there was a way to avoid doing this and instead, if it was possible to add multiple conditions to the query.
Does anyone know how to achieve this?
Many thanks,
Assuming field1 and field2 are just another fields like Title and Description, you can use bool must query to achieve what you want.
var searchResults = _client.Search<MyIndexable>(s => s
.Query(q => q
.Bool(b => b
.Must(
subQuery => subQuery.QueryString(qs => qs.OnFields(f => f.Title, f => f.Description).Query(searchTerm + "*"")),
subQuery => subQuery.Term(f => f.field1, "some string"),
subQuery => subQuery.Term(f => f.field2 , "some Guid")))));

Combine two condition from two table with OR clause in orchard HQL query

In orchard cms i want to write a query containing two condition over two different ContentPartRecord combined with OR clause. one possible way to combine conditions is :
var query = _contentManager.HqlQuery()
.Where(a => a.ContentPartRecord<ProductPartRecord>(), a => a.Eq("Title", "myTitle"))
.Where(a => a.ContentPartRecord<TitlePartRecord>(), a => a.Eq("Price", 1000))
.List();
but this one combines two condition with And clause. i think something like following along with some changes (which would be the answer for this question) could be the case:
var query = _contentManager.HqlQuery()
.Join(a => a.ContentPartRecord<ProductPartRecord>())
.Join(a => a.ContentPartRecord<TitlePartRecord>())
.Where(a => a.ContentItem(),
a => a.Or(p => p.Eq("ProductPartRecord.Price",
"1000"), t => t.Eq("TitlePartRecord.Title", "myTitle")))
.List();
but i couldn't get it working.any body have any suggestion?
When you use where clause in HqlQuery you need to pass Alias (as first parameter). Alias means that you will be apply where clause to specific table (class that represent this table). And when you use OR clause that you definitely need to compare columns of the specific table in two parts of OR clause (left and right from the OR clause). And i think this is not possible to do you need a standard way.
But you can use:
Action<IAliasFactory> productPartRecordAlias = x => x.ContentPartRecord<ProductPartRecord>().Named("productPartRecord");
Action<IAliasFactory> titlePartRecordAlias = x => x.ContentPartRecord<TitlePartRecord>().Named("titlePartRecord");
var query = _contentManager.HqlQuery()
.Join(productPartRecordAlias)
.Join(titlePartRecordAlias)
.Where(a => a.ContentItem(), p => p.Gt("Id", "0 AND (productPartRecord.Price = 1000 OR titlePartRecord.Title = 'myTitle')"));

Categories