I started out with Mongo client doing some nifty queries and aggretations.. but now that I want to use it in .NET/C#, I see that I can't simply run the query as text field..
Furthermore, after resorting to building an Aggregation Pipeline, and running the collection.Aggregate() function, I'm getting a result set, but I have no idea how to traverse it..
Can anyone help guide me here?
Here's my code:
var coll = db.GetCollection("animals");
var match = new BsonDocument {
{ "$match", new BsonDocument {{"category","cats"}} }
};
var group = new BsonDocument{
{
"$group", new BsonDocument{
{"_id", "$species"},
{"AvgWeight", new BsonDocument{{"$avg", "$weight"}}} }
}
};
var sort = new BsonDocument{{"$sort", new BsonDocument{{"AvgWeight", -1}}}};
var pipeline = new[] { match, group, sort };
var args = new AggregateArgs { Pipeline = pipeline };
var res = coll.Aggregate(args);
foreach (var obj in res)
{
// WHAT TO DO HERE??
}
Also, I should say that I'm a little rusty with C# / ASP.NET / MVC so any room for simplification would be much appreciated.
Your result is IEnumerable of BsonDocument, you can Serialize them to C# objects using the BSonSerializer. And this code snippet just writes them to your console, but you can see that you have typed objects
List<Average> returnValue = new List<Average>();
returnValue.AddRange(documents.Select(x=> BsonSerializer.Deserialize<Average>(x)));
foreach (var obj in returnValue)
{
Console.WriteLine("Species {0}, avg weight: {1}",returnValue._Id,returnValue.AvgWeight);
}
And then have a class called Average, where the property name match the names in the BSonDocument, if you want to rename then (because _Id is not so nice in c# terms concerning naming conventions), you can add a $project BsonDocument to your pipeline.
public class Average
{
public string _Id { get; set; }
public Double AvgWeight {get; set; }
}
$project sample (add this in your pipeline just before sort
var project = new BsonDocument
{
{
"$project",
new BsonDocument
{
{"_id", 0},
{"Species","$_id"},
{"AvgWeight", "$AvgWeight"},
}
}
};
Related
We have a data model with defined properties, but one of the properties allows for dynamic metadata (a list of maps or dictionaries). Using the document model, this property maps fine to a list of Document, however, when I'm having trouble getting this dynamic property to map to anything using DataModel. Is there a way to map dynamic data to documents inside a model class property?
Attempting to map it as a list of dictionaries (which matches the structure of the metadata) fails with the below error:
public List<Dictionary<string, object>> Events { get; set; }
Unable to convert [Amazon.DynamoDBv2.DocumentModel.Document] of type
Amazon.DynamoDBv2.DocumentModel.Document to
System.Collections.Generic.Dictionary`
Using a type of List<Document> got me the closest, which it now lists out 39 documents, but all the Documents have 0 keys, 0 values.
public List<Document> Events { get; set; }
Ex:
document["Events"].AsListOfDocument().First(); // works, contains the keys and values
datamodel.Events.First(); // does not work, it is an empty document
I know this is a quite old but hopefully this might help somebody else struggling with this.
Hi Devon,
In case you are trying to create the object to send it to your DynamoDB you can try mapping arbitrary data just as stated in AWS documentation for .NET
Here is the code from the documentation:
try
{
DynamoDBContext context = new DynamoDBContext(client);
// 1. Create a book.
DimensionType myBookDimensions = new DimensionType()
{
Length = 8M,
Height = 11M,
Thickness = 0.5M
};
Book myBook = new Book
{
Id = 501,
Title = "AWS SDK for .NET Object Persistence Model Handling Arbitrary Data",
ISBN = "999-9999999999",
BookAuthors = new List<string> { "Author 1", "Author 2" },
Dimensions = myBookDimensions
};
context.Save(myBook);
Once you have your data in the database you can append a new map to the list with something on the lines of:
var request = new UpdateItemRequest
{
TableName = "TableName",
Key = new Dictionary<string, AttributeValue>() { { "PartitionKey", new AttributeValue { S = "value" } } },
ExpressionAttributeNames = new Dictionary<string, string>()
{
{ "#A", "A" }
},
ExpressionAttributeValues = new Dictionary<string, AttributeValue>()
{
{
":val", new AttributeValue
{
L = new List<AttributeValue>()
{
{
new AttributeValue
{
M = new Dictionary<string, AttributeValue>()
{
{ "Address", new AttributeValue{S = "Value" } },
{ "Latitude", new AttributeValue { S = position.Latitude.ToString() } },
{ "Longitude", new AttributeValue { S = position.Longitude.ToString() } }
}
}
}
}
}
}
},
UpdateExpression = "SET #A = list_append(#A, :val)"
};
try
{
var response = await client.UpdateItemAsync(request);
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
This did the trick for me, I hope it does for someone else.
.
.
.
PS: BTW this is my first answer in Stackoverflow and it feels nice to try to contribute when I have came here multiple times for answers that saved me time jajajaja
I have .NET Application that uses MongoDB. Current driver I am using is 1.9.2. I am trying to upgrade it to 2.7.0.
I am having some difficulty in getting an Aggregate query to work in the new version:
The working code in version 1.9.2 of the driver is:
public IEnumerable<Car> GetCarsModifiedInPeriod(DateTimeOffset dateFrom, DateTimeOffset dateTo)
{
var matchRequestFromDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$gte", dateFrom.Ticks}}
}
}
}
};
var matchRequestToDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$lte", dateTo.Ticks}}
}
}
}
};
var cars = collection.Aggregate(new AggregateArgs
{
Pipeline = new[] { matchRequestFromDate, matchRequestToDate},
AllowDiskUse = true,
// Setting the OutputMode to Cursor allows us to return Mongo Doc Size > 16 MB - in the case when a large date
// range is used or a large number of cars were modified in a short period of time
OutputMode = AggregateOutputMode.Cursor
}).Select(r => r.Values.Select(c => c.AsObjectId.ToString()).First());
var returnData = collection.AsQueryable().Where(c => cars.Contains(c.Id)).Select(c => c);
return returnData;
}
With a breakpoint set on returnData for the two periods specified I am getting a count of 25 cars which is what I expect.
This is how I have attempted to re-write for 2.7.0 version of driver:
public IEnumerable<Car> GetCarsModifiedInPeriod(DateTimeOffset dateFrom, DateTimeOffset dateTo)
{
var matchRequestFromDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$gte", dateFrom.Ticks}}
}
}
}
};
var matchRequestToDate = new BsonDocument
{
{
"$match",
new BsonDocument
{
{
// Filter out those too recently modified
"LastUpdatedOn.0", new BsonDocument {{"$lte", dateTo.Ticks}}
}
}
}
};
var pipeline = new[] {matchRequestFromDate, matchRequestToDate};
//var mongoPipeline = new AggregateArgs { Pipeline = pipeline, AllowDiskUse = true, OutputMode = AggregateOutputMode.Cursor };
var aggregate = collection.Aggregate(); //.Match(mongoPipeline);
aggregate.Options.AllowDiskUse = true;
aggregate.Options.UseCursor = true;
foreach (var pipe in pipeline)
{
aggregate.AppendStage<BsonDocument>(pipe);
}
var returnData = aggregate.ToList();
return returnData;
}
If I set a breakpoint in returnData in this method I am getting a count of around 10K cars so it doesnt look like I am correctly applying the same matches
Is there a reason you are doing everything in BsonDocuments? There are methods that would make your life a lot easier, for example something like this.
collection.Aggregate(new AggregateOptions() { AllowDiskUse = true, UseCursor = true })
.Match(Builders<BsonDocument>.Filter.Gte("LastUpdatedOn.0", dateFrom.Ticks) & Builders<BsonDocument>.Filter.Lte("LastUpdatedOn.0", dateFrom.Ticks))
.ToListAsync()
You could tidy the filtering up more as well by using the right class for the collection and the builders.
Looking at the query, I'm not sure you even need to be using an aggregate unless you are doing more than a match. It could simply be a find.
I'm playing around with the new driver of mongodb 2.0, and looking for adding some facetted searchs (Temporary move ,before using elastic search).
Here is some method where I created to build the agreggation. I guess that it should work.
As parameter I passed also a filterdefinition in the method.
But I don't find how to limit my agreggation to the filter.
Any Idea ???
private void UpdateFacets(SearchResponse response, FilterDefinition<MediaItem> filter, ObjectId dataTableId)
{
response.FacetGroups =new List<SearchFacetGroup>();
SearchFacetGroup group = new SearchFacetGroup()
{
Code = "CAMERAMODEL",
Display = "Camera model",
IsOptional = false
};
using (IDataAccessor da = NodeManager.Instance.GetDataAccessor(dataTableId))
{
var collection = da.GetCollection<MediaItem>();
var list = collection.Aggregate()
.Group(x => ((ImageMetaData) x.MetaData).Exif.CameraModel, g => new { Model = g.Key, Count = g.Count() })
.ToListAsync().Result;
foreach (var l in list)
{
group.Facets.Add(new SearchFacetContainer()
{
Code = l.Model,
Display = l.Model,
Hits = l.Count,
IsSelected = false
});
}
}
response.FacetGroups.Add(group);
}
I haven't used facet, but with Mongo driver Aggregate has .Match operation that accepts a filterdefinition.
collection1.Aggregate().Match(filter)
How can I access the MongoCursor attributes in C#.
I have the following line of code:
MongoCursor results = collection.Find(searchQuery).SetLimit(10).SetFields(
Fields.Include("name1","name", "_id"));
MongoDB returns an array, each one with two attributes: name and name1. In the results View in the debugger,, I can see an array, each item in the array contains a MongoDB.Bson.BsonDocument.
I want to have a dot notation access to the attributes of each BsonDocument in the Array. How can I achieve this.?
Came across this issue when I had to work with raw BsonDocuments. This solution does allow you to use dot notation.
Not fully tested or anything, but may be useful.
void Main()
{
var example = new BsonDocument{
{ "customer", new BsonDocument { { "email" , "homerjay#simpsons.com" } }}
};
var email = example.GetPath("customer.email").AsString;
Console.WriteLine(email);
}
public static class MongoDbHelper
{
public static BsonValue GetPath(this BsonValue bson, string path)
{
if (bson.BsonType != BsonType.Document)
{
throw new Exception("Not a doc");
}
var doc = bson.AsBsonDocument;
var tokens = path.Split(".".ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
if (tokens.Length == 0 )
{
return doc;
}
if (!doc.Contains(tokens[0]))
{
return BsonNull.Value;
}
if (tokens.Length > 1)
{
return GetPath(doc[tokens[0]], tokens[1]);
}
return doc[tokens[0]];
}
}
To get values out of a BsonDocument you can use the GetValue/TryGetValue methods or the indexer:
foreach (var document in results)
{
var name1 = document.GetValue("name1");
var name = document["name"];
}
I am working on below code, and what I want to do is query by object itself.
For example: I have a search form, that populates objects fields such as below. Then what I want to do is to search Elastic search based on whatever user filled the form with.
ie: below, I want to query the index by searchItem object. How can I do it easily?
class Program
{
static void Main(string[] args)
{
var p = new Program();
var item1 = new Announcement() {Id=1, Title = "john", ContentText = "lorem", Bar = false, Num = 99, Foo = "hellow"};
//p.Index(item1, "add");
var searchItem = new Announcement() {Title="john",Num=99};
ElasticClient.Search<Announcement>();
Console.Read();
}
public void Index(Announcement announcement, String operation)
{
var uriString = "http://localhost:9200";
var searchBoxUri = new Uri(uriString);
var settings = new ConnectionSettings(searchBoxUri);
settings.SetDefaultIndex("test");
var client = new ElasticClient(settings);
if (operation.Equals("delete"))
{
client.DeleteById("test", "announcement", announcement.Id);
}
else
{
client.Index(announcement, "test", "announcement", announcement.Id);
}
}
private static ElasticClient ElasticClient
{
get
{
try
{
var uriString = "http://localhost:9200";
var searchBoxUri = new Uri(uriString);
var settings = new ConnectionSettings(searchBoxUri);
settings.SetDefaultIndex("test");
return new ElasticClient(settings);
}
catch (Exception)
{
throw;
}
}
}
}
You can't :)
NEST cannot infer how to best query only based on a partially filled POCO. Should it OR or AND should it do a nested term query or a term query wrapped in a has_child? You catch my drift.
Nest does have a slick feature called conditionless queries that allow you the write out to entire query like so:
ElasticClient.Search<Announcement>(s=>s
.Query(q=>
q.Term(p=>p.Title, searchItem.Title)
&& q.Term(p=>p.Num, searchItem.Num)
//Many more queries use () to group all you want
)
)
When NEST sees that the argument passed to Term is null or empty it simply wont render that part of the query.
Read more here on how this feature works http://nest.azurewebsites.net/concepts/writing-queries.html