I'm trying to return documents from the collection where id = 1 and date created is less than the current time.
I tried this, but it isn't working:
var collection = database.GetCollection("test");
var time = DateTime.Now;
var query2 = new QueryDocument
{
{ "id", 1},
{{"created_on", {"$lt",time}}
};
What's wrong with this query?
You need an embedded document for the $lt sub-object, but you forgot to create it:
var query = new QueryDocument {
{ "id", 1 },
{ "created_on", new BsonDocument { { "$lt", time } } }
}
Also consider using the Query builder, which might make things simpler:
var query = Query.And(
Query.EQ("id", 1),
Query.LT("created_on", time)
);
Related
Does anyone know how to do multiple aggregations with nest?
I have found quite a few examples unfortunately none of them work.
Here's what I have:
Vehicles fields = new Vehicles();
//create a terms query
var query = new TermsQuery
{
IsVerbatim = true,
Field = "VehicleOwnerId",
Terms = new string[] { 25 },
};
var aggregations = new Dictionary<string, IAggregationContainer>
{
{ "years", new AggregationContainer
{
Terms = new TermsAggregation(nameof(fields.Year))
{
Field = new Field(nameof(fields.Year))
}
}
}
//,
//{ "makes", new AggregationContainer
// {
// Terms = new TermsAggregation("Make")
// {
// Field = new Field(nameof(fields.Make))
// }
// }
//}
};
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 100,
Aggregations = aggregations
};
var result = client.SearchAsync<InventoryLiveView>(searchRequest).Result;
var years = result.Aggregations.Terms("years");
Dictionary<string, long> yearCounts = new Dictionary<string, long>();
foreach (var item in years.Buckets)
{
yearCounts.Add(item.Key, item.DocCount ?? 0);
}
If I just execute the code like this it works. Years returns the aggregates as expected. If I try to add another field (like the one commented out above) it fails and I get zero records.
How can I get multiple aggregates in one query? I see examples of it all over, but none of the examples I've tried seem to work and most seem to be outdated (including some in the Nest documentation).
I have also tried this approach which is pretty close to the documentation.
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 100,
//Aggregations = aggregations
Aggregations = new AggregationDictionary
{
{
"childAgg", new ChildrenAggregation("childAgg", typeof(Vehicles ))
{
Aggregations = new AggregationDictionary
{
{"years", new TermsAggregation(nameof(fields.VehicleYear))},
{"makes", new TermsAggregation(nameof(fields.VehicleMakeName))},
{"models", new TermsAggregation(nameof(fields.VehicleModelName))},
}
}
}
}
};
var result = client.SearchAsync<Vehicles>(searchRequest).Result;
This just produces a null reference exception.
I guess I'll never have too worry about getting to proud as a programmer :)
It's too often that the solution to the problem makes me feel stupid when it reveals itself.
So my issue was that the field I was trying to use in the aggregation was text and couldn't be used. I switched everything to the ID fields and multiple aggregations work as expected.
So this version of the code works like a champ:
Vehicle fields = new Vehicle ();
//create a terms query
var query = new TermsQuery
{
IsVerbatim = true,
Field = "VehicleOwnerId",
Terms = new string[] { "30" },
};
string[] Fields = new[]
{
nameof(fields.Year),
nameof(fields.MakeId),
nameof(fields.ModelId)
};
var aggregations = new Dictionary<string, IAggregationContainer>();
foreach (string sField in Fields)
{
var termsAggregation = new TermsAggregation(sField)
{
Field = sField
};
aggregations.Add(sField, new AggregationContainer { Terms = termsAggregation });
}
//create the search request
var searchRequest = new SearchRequest
{
Query = query,
From = 0,
Size = 10,
Aggregations = aggregations
};
var result = client.SearchAsync<InventoryLiveView>(searchRequest).Result;
var years = result.Aggregations.Terms(nameof(fields.Year));
Dictionary<string, long> yearCounts = new Dictionary<string, long>();
foreach (var item in years.Buckets)
{
yearCounts.Add(item.Key, item.DocCount ?? 0);
}
The exact error from elasticsearch, which I saw using postman was:
Fielddata is disabled on text fields by default. Set fielddata=true on [MakeName] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
Here is my example using SearchDescriptors. My only problem is how to serialize returned results into a proper Key Value list. Is Looping through a fields list the best way to return results.
SearchDescriptor<Advert> agghDescriptor = new SearchDescriptor<Advert>();
agghDescriptor.Aggregations(ag => ag.Terms("make", a => a.Field(f => f.Make)) &&
ag.Terms("region", a => a.Field(f => f.Region)) &&
ag.Terms("city", a => a.Field(f => f.City)) &&
ag.Terms("category", a => a.Field(f => f.Category)) &&
ag.Terms("application", a => a.Field(f => f.Application)) &&
ag.Terms("portalId", a => a.Field(f => f.PortalId)) &&
ag.Terms("isActiveAuctionAdvert", a => a.Field(f => f.IsActiveAuctionAdvert)) &&
ag.Terms("isBargainAccount", a => a.Field(f => f.IsBargainAccount)) &&
ag.Terms("condition", a => a.Field(f => f.Condition))
);
agghDescriptor.Size(0);
var json2 = _client.RequestResponseSerializer.SerializeToString(agghDescriptor);
var aggregationResult = _client.Search<Advert>(agghDescriptor);
List<string> fields = new List<string>();
fields.Add("make");
fields.Add("category");
fields.Add("region");
List<Aggregation> aggregations = new List<Aggregation>();
foreach (var field in fields)
{
var aggrs = aggregationResult.Aggregations.Terms(field);
List<AggregateItem> aggregateItems = new List<AggregateItem>();
foreach (var item in aggrs.Buckets)
{
aggregateItems.Add(new AggregateItem()
{
Count = item.DocCount ?? 0,
Key = item.Key
});
}
aggregations.Add(new Aggregation()
{
Name = field,
Aggregates = aggregateItems
});
}
I have .NET Application that uses MongoDB. Current driver I am using is 1.9.2. I am trying to upgrade it to 2.7.0.
I am having some difficulty in getting an Aggregate query to work in the new version:
The working code in version 1.9.2 of the driver is:
public IEnumerable<Car> GetCarsModifiedInPeriod(DateTimeOffset dateFrom, DateTimeOffset dateTo)
{
var matchRequestFromDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$gte", dateFrom.Ticks}}
}
}
}
};
var matchRequestToDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$lte", dateTo.Ticks}}
}
}
}
};
var cars = collection.Aggregate(new AggregateArgs
{
Pipeline = new[] { matchRequestFromDate, matchRequestToDate},
AllowDiskUse = true,
// Setting the OutputMode to Cursor allows us to return Mongo Doc Size > 16 MB - in the case when a large date
// range is used or a large number of cars were modified in a short period of time
OutputMode = AggregateOutputMode.Cursor
}).Select(r => r.Values.Select(c => c.AsObjectId.ToString()).First());
var returnData = collection.AsQueryable().Where(c => cars.Contains(c.Id)).Select(c => c);
return returnData;
}
With a breakpoint set on returnData for the two periods specified I am getting a count of 25 cars which is what I expect.
This is how I have attempted to re-write for 2.7.0 version of driver:
public IEnumerable<Car> GetCarsModifiedInPeriod(DateTimeOffset dateFrom, DateTimeOffset dateTo)
{
var matchRequestFromDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$gte", dateFrom.Ticks}}
}
}
}
};
var matchRequestToDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$lte", dateTo.Ticks}}
}
}
}
};
var pipeline = new[] {matchRequestFromDate, matchRequestToDate};
//var mongoPipeline = new AggregateArgs { Pipeline = pipeline, AllowDiskUse = true, OutputMode = AggregateOutputMode.Cursor };
var aggregate = collection.Aggregate(); //.Match(mongoPipeline);
aggregate.Options.AllowDiskUse = true;
aggregate.Options.UseCursor = true;
foreach (var pipe in pipeline)
{
aggregate.AppendStage<BsonDocument>(pipe);
}
var returnData = aggregate.ToList();
return returnData;
}
If I set a breakpoint in returnData in this method I am getting a count of around 10K cars so it doesnt look like I am correctly applying the same matches
Is there a reason you are doing everything in BsonDocuments? There are methods that would make your life a lot easier, for example something like this.
collection.Aggregate(new AggregateOptions() { AllowDiskUse = true, UseCursor = true })
.Match(Builders<BsonDocument>.Filter.Gte("LastUpdatedOn.0", dateFrom.Ticks) & Builders<BsonDocument>.Filter.Lte("LastUpdatedOn.0", dateFrom.Ticks))
.ToListAsync()
You could tidy the filtering up more as well by using the right class for the collection and the builders.
Looking at the query, I'm not sure you even need to be using an aggregate unless you are doing more than a match. It could simply be a find.
I'm playing around with the new driver of mongodb 2.0, and looking for adding some facetted searchs (Temporary move ,before using elastic search).
Here is some method where I created to build the agreggation. I guess that it should work.
As parameter I passed also a filterdefinition in the method.
But I don't find how to limit my agreggation to the filter.
Any Idea ???
private void UpdateFacets(SearchResponse response, FilterDefinition<MediaItem> filter, ObjectId dataTableId)
{
response.FacetGroups =new List<SearchFacetGroup>();
SearchFacetGroup group = new SearchFacetGroup()
{
Code = "CAMERAMODEL",
Display = "Camera model",
IsOptional = false
};
using (IDataAccessor da = NodeManager.Instance.GetDataAccessor(dataTableId))
{
var collection = da.GetCollection<MediaItem>();
var list = collection.Aggregate()
.Group(x => ((ImageMetaData) x.MetaData).Exif.CameraModel, g => new { Model = g.Key, Count = g.Count() })
.ToListAsync().Result;
foreach (var l in list)
{
group.Facets.Add(new SearchFacetContainer()
{
Code = l.Model,
Display = l.Model,
Hits = l.Count,
IsSelected = false
});
}
}
response.FacetGroups.Add(group);
}
I haven't used facet, but with Mongo driver Aggregate has .Match operation that accepts a filterdefinition.
collection1.Aggregate().Match(filter)
I started out with Mongo client doing some nifty queries and aggretations.. but now that I want to use it in .NET/C#, I see that I can't simply run the query as text field..
Furthermore, after resorting to building an Aggregation Pipeline, and running the collection.Aggregate() function, I'm getting a result set, but I have no idea how to traverse it..
Can anyone help guide me here?
Here's my code:
var coll = db.GetCollection("animals");
var match = new BsonDocument {
{ "$match", new BsonDocument {{"category","cats"}} }
};
var group = new BsonDocument{
{
"$group", new BsonDocument{
{"_id", "$species"},
{"AvgWeight", new BsonDocument{{"$avg", "$weight"}}} }
}
};
var sort = new BsonDocument{{"$sort", new BsonDocument{{"AvgWeight", -1}}}};
var pipeline = new[] { match, group, sort };
var args = new AggregateArgs { Pipeline = pipeline };
var res = coll.Aggregate(args);
foreach (var obj in res)
{
// WHAT TO DO HERE??
}
Also, I should say that I'm a little rusty with C# / ASP.NET / MVC so any room for simplification would be much appreciated.
Your result is IEnumerable of BsonDocument, you can Serialize them to C# objects using the BSonSerializer. And this code snippet just writes them to your console, but you can see that you have typed objects
List<Average> returnValue = new List<Average>();
returnValue.AddRange(documents.Select(x=> BsonSerializer.Deserialize<Average>(x)));
foreach (var obj in returnValue)
{
Console.WriteLine("Species {0}, avg weight: {1}",returnValue._Id,returnValue.AvgWeight);
}
And then have a class called Average, where the property name match the names in the BSonDocument, if you want to rename then (because _Id is not so nice in c# terms concerning naming conventions), you can add a $project BsonDocument to your pipeline.
public class Average
{
public string _Id { get; set; }
public Double AvgWeight {get; set; }
}
$project sample (add this in your pipeline just before sort
var project = new BsonDocument
{
{
"$project",
new BsonDocument
{
{"_id", 0},
{"Species","$_id"},
{"AvgWeight", "$AvgWeight"},
}
}
};
I'm writing a bit of code, and I'd like to play with doing it using the anonymous features of C#.
I'm writing a summary based on a DataTable returned from the SQL Server.
There are many ways I could write it already knowing Classical C# (???), but I'm interested in having a little fun.
So, here are the type of anonymous classes I want to have:
// Employee
var emp = new {
Badge = "000000",
Name = "No Name",
Parts = new List<Part>(),
Days = new List<DateTime>(),
};
// Part
var part = new {
SerialNumber = "N/A",
Date = DateTime.MinValue,
Badge = "000000",
};
Now, as I iterate over my DataTable entries, I want to sort my Parts by SerialNumber.
The first thing I have to do is break the data down into days.
private void TestMethod(DateTime minDate, DateTime maxDate, DataTable table) {
int days = 1;
var nextDay = minDate.AddHours(24);
foreach (DataRow row in table.Rows) {
var dateTime = (DateTime)row["Date_Time"];
var emp = new {
Badge = row["Badge"].ToString(),
Parts = new List<Part>(),
Days = new List<DateTime>(),
};
var part = new {
SerialNumber = row["Serial_Number"].ToString(),
Date = dateTime,
Badge = row["Badge"].ToString(),
};
if (nextDay < dateTime) {
days++;
nextDay = nextDay.AddHours(24);
}
}
Now, it is getting a little interesting.
I need a way to store Part information for the different days and the different employees found for the period.
How would I create and use an anonymous collection of my anonymous class items?
var parts = new List<typeof(part)>();
var emps = new List<typeof(emp)>();
Using typeof (above) does not work!
What does?
You need to use type inference:
new[] { part }.ToList()
(you probably want to clear the list afterwards)
You can also make a helper method:
public static List<T> ListOf<T>(T sample) {
return new List<T>();
}
var parts = ListOf(part);