We Keep Coding

c# Nest and Elasticsearch Aggregations

Does anyone know how to do multiple aggregations with nest?
I have found quite a few examples unfortunately none of them work.
Here's what I have:
Vehicles fields = new Vehicles();
//create a terms query
var query = new TermsQuery
{
IsVerbatim = true,
Field = "VehicleOwnerId",
Terms = new string[] { 25 },
};
var aggregations = new Dictionary<string, IAggregationContainer>
{
{ "years", new AggregationContainer
{
Terms = new TermsAggregation(nameof(fields.Year))
{
Field = new Field(nameof(fields.Year))
}
}
}
//,
//{ "makes", new AggregationContainer
// {
// Terms = new TermsAggregation("Make")
// {
// Field = new Field(nameof(fields.Make))
// }
// }
//}
};
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 100,
Aggregations = aggregations
};
var result = client.SearchAsync<InventoryLiveView>(searchRequest).Result;
var years = result.Aggregations.Terms("years");
Dictionary<string, long> yearCounts = new Dictionary<string, long>();
foreach (var item in years.Buckets)
{
yearCounts.Add(item.Key, item.DocCount ?? 0);
}
If I just execute the code like this it works. Years returns the aggregates as expected. If I try to add another field (like the one commented out above) it fails and I get zero records.
How can I get multiple aggregates in one query? I see examples of it all over, but none of the examples I've tried seem to work and most seem to be outdated (including some in the Nest documentation).
I have also tried this approach which is pretty close to the documentation.
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 100,
//Aggregations = aggregations
Aggregations = new AggregationDictionary
{
{
"childAgg", new ChildrenAggregation("childAgg", typeof(Vehicles ))
{
Aggregations = new AggregationDictionary
{
{"years", new TermsAggregation(nameof(fields.VehicleYear))},
{"makes", new TermsAggregation(nameof(fields.VehicleMakeName))},
{"models", new TermsAggregation(nameof(fields.VehicleModelName))},
}
}
}
}
};
var result = client.SearchAsync<Vehicles>(searchRequest).Result;
This just produces a null reference exception.

I guess I'll never have too worry about getting to proud as a programmer :)
It's too often that the solution to the problem makes me feel stupid when it reveals itself.
So my issue was that the field I was trying to use in the aggregation was text and couldn't be used. I switched everything to the ID fields and multiple aggregations work as expected.
So this version of the code works like a champ:
Vehicle fields = new Vehicle ();
//create a terms query
var query = new TermsQuery
{
IsVerbatim = true,
Field = "VehicleOwnerId",
Terms = new string[] { "30" },
};
string[] Fields = new[]
{
nameof(fields.Year),
nameof(fields.MakeId),
nameof(fields.ModelId)
};
var aggregations = new Dictionary<string, IAggregationContainer>();
foreach (string sField in Fields)
{
var termsAggregation = new TermsAggregation(sField)
{
Field = sField
};
aggregations.Add(sField, new AggregationContainer { Terms = termsAggregation });
}
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 10,
Aggregations = aggregations
};
var result = client.SearchAsync<InventoryLiveView>(searchRequest).Result;
var years = result.Aggregations.Terms(nameof(fields.Year));
Dictionary<string, long> yearCounts = new Dictionary<string, long>();
foreach (var item in years.Buckets)
{
yearCounts.Add(item.Key, item.DocCount ?? 0);
}
The exact error from elasticsearch, which I saw using postman was:
Fielddata is disabled on text fields by default. Set fielddata=true on [MakeName] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

Here is my example using SearchDescriptors. My only problem is how to serialize returned results into a proper Key Value list. Is Looping through a fields list the best way to return results.
SearchDescriptor<Advert> agghDescriptor = new SearchDescriptor<Advert>();
agghDescriptor.Aggregations(ag => ag.Terms("make", a => a.Field(f => f.Make)) &&
ag.Terms("region", a => a.Field(f => f.Region)) &&
ag.Terms("city", a => a.Field(f => f.City)) &&
ag.Terms("category", a => a.Field(f => f.Category)) &&
ag.Terms("application", a => a.Field(f => f.Application)) &&
ag.Terms("portalId", a => a.Field(f => f.PortalId)) &&
ag.Terms("isActiveAuctionAdvert", a => a.Field(f => f.IsActiveAuctionAdvert)) &&
ag.Terms("isBargainAccount", a => a.Field(f => f.IsBargainAccount)) &&
ag.Terms("condition", a => a.Field(f => f.Condition))
);
agghDescriptor.Size(0);
var json2 = _client.RequestResponseSerializer.SerializeToString(agghDescriptor);
var aggregationResult = _client.Search<Advert>(agghDescriptor);
List<string> fields = new List<string>();
fields.Add("make");
fields.Add("category");
fields.Add("region");
List<Aggregation> aggregations = new List<Aggregation>();
foreach (var field in fields)
{
var aggrs = aggregationResult.Aggregations.Terms(field);
List<AggregateItem> aggregateItems = new List<AggregateItem>();
foreach (var item in aggrs.Buckets)
{
aggregateItems.Add(new AggregateItem()
{
Count = item.DocCount ?? 0,
Key = item.Key
});
}
aggregations.Add(new Aggregation()
{
Name = field,
Aggregates = aggregateItems
});
}

RavenDB Collection "in" Collection query

I need to preform a query that check if a collection is in given collection, just like the regular in operation but for collections.
class Post
{
public string[] Tags {get;set;}
}
session.Queury<Post>.Where(x=>x.Tags.in(new[]{".net","c#","RavenDB"})).ToList();
so if i have in my DB:
new Post{Tags= new[]{"C#",".net"}};
it will be returned
but if i have:
new Post{Tags= new[]{"C#",".net","SQLServer"}};
it will not be returned.
Update:
what i am trying to do is this:
session.Query<Post>()
.Where(x => x.Tags.All(y => y.In(new[] { "C#", ".net", "RavenDB" })))
.ToList();
but i got System.NotSupportedException.

I manage to find a solution:
static void Main(string[] args)
{
var sessionStore = new EmbeddableDocumentStore
{
RunInMemory = true,
UseEmbeddedHttpServer = true,
Conventions =
{
DefaultQueryingConsistency = ConsistencyOptions.AlwaysWaitForNonStaleResultsAsOfLastWrite
}
};
sessionStore.Initialize();
using (var session = sessionStore.OpenSession())
{
var allTags = new[] {"C#", ".net", "RavenDB", "Linux", "Mac"};
var tagsCollection = new[] {"C#", ".net", "RavenDB"};
var complementTagsCollection = allTags.Except(tagsCollection).ToList();
session.Store(new Post
{
Tags = new List<string>{"C#",".net"}
});
session.SaveChanges();
// Posts where all their tags are in tagsCollection
var result = session.Query<Post>().Where(x => !x.Tags.In(complementTagsCollection)).ToList();
}
}

The way IN works, it matches ANY of them.
If you want to match all you have to do a separate check for each.

ToDictionary error "Cannot apply indexing with [] to an expression of type 'method group'"

I have the following code inside my asp.net mvc web application :-
public void syncWithEX()
{
var EXresource = entities.Resources.Select(g => new
{
EXID = g.RESOURCEID,
CurrentEXSiteID = g.ResourceLocation.SITEID
}).ToDictionary(a => a.EXID, a => a.CurrentEXSiteID);
var technology = ITSys.Technologies.Select(g => new
{
ID = g.TechnologyID,
EXID = g.EXID
}).ToDictionary(a => a.ID, a => a.EXID);
var server = ITSys.ITSYSServers.Where(a => !a.Technology.IsDeleted && a.Technology.IsCompleted);
foreach (var s in server)
{
long? EXid = technology[s.ITSYSServerID];
if (EXresource.ContainsKey[EXid.Value] )
{
long? CurrentEXsiteid = EXresource[EXid.Value];
if (CurrentEXsiteid != s.EXSiteID)
{
s.EXSiteID = CurrentEXsiteid.Value;
ITSys.Entry(s).State = EntityState.Modified;
}
}
}
But i am getting the following error :
Cannot apply indexing with [] to an expression of type 'method group
on the following code:
if (EXresource.ContainsKey[EXid.Value] )

change to:
if (EXresource.ContainsKey(EXid.Value))
Dictionary.ContainsKey()

Alternate foreach output

I have this code which grabs the specified text from a webpage:
static void Main(string[] args)
{
using (var client = new WebClient())
{
var pageContent = client.DownloadString("http://www.modern-railways.com");
var regexTitle = new Regex(#"<span class='articleTitle'>(.+?)</span>");
var regexDate = new Regex(#"class='summaryText' data-ajax='false'>(.+?)</a></p><div");
foreach (Match title in regexTitle.Matches(pageContent))
{
var articleTitle = title.Groups[1].Value;
Console.WriteLine(articleTitle);
}
foreach (Match date in regexDate.Matches(pageContent))
{
var articleDate = date.Groups[1].Value;
Console.WriteLine(articleDate);
}
Console.ReadLine();
}
}
As it is now it prints all the articleTitle first and then all the articleDate. How can I get out 1st line ArticleTitle, second line articleDate and so on?

You can use LINQ and Zip method:
var titles = regexTitles.Matches(pageContent).Cast<Match>();
var dates = regexDate.Matches(pageContent).Cast<Match>();
var source = titles.Zip(dates, (t, d) => new { Title = t, Date = d })
foreach (var item in source)
{
var articleTitle = item.Title.Groups[1].Value;
var articleDate = item.Date.Groups[1].Value;
Console.WriteLine(articleTitle);
Console.WriteLine(articleDate);
}

Error in lambda: No implicit convertion between

I convert one list of to list and get error:
"Type of conditional expression cannot be determined because there is no implicit conversion between System.Collections.Generic.List and 'void'
return (topics.Select(c => new TopicUi()
{
Bookmarks = new List<Bookmark>().Add(new Bookmark { Id = c.BookmarkId, Name = c.BookmarkName })
})).ToList();
Why?

The Add method of List has a return type of void, this should work for you:
return (topics.Select(c => new TopicUi
{
Bookmarks = new List<Bookmark> {
new Bookmark { Id = c.BookmarkId, Name = c.BookmarkName }
}
})).ToList();

At the very leat, fix this line
Bookmarks = new List<Bookmark>().Add(new Bookmark { Id = c.BookmarkId, Name = c.BookmarkName })
Add is a void returning method. The line should be
Bookmarks = new List<Bookmark> { new Bookmark { Id = c.BookmarkId, Name = c.BookmarkName } }
In order to properly use collection initialization.

Rather than calling the Add method of List<T>, you can just use the object initialization syntax:
return (topics.Select(c => new TopicUi()
{
Bookmarks = new List<Bookmark>()
{ new Bookmark { Id = c.BookmarkId, Name = c.BookmarkName } }
)
})).ToList();

IList.Add has no return type. Try this;
Func<YourCType, IList<Bookmark>> createBookmarkListFunc = (c) =>
{
var list = new List<Bookmark>() { new Bookmark { Id = c.BookmarkId, Name = c.BookmarkName };
return list;
});
return (topics.Select(c => new TopicUi()
{
Bookmarks = createListFunc(c)
})).ToList();