We Keep Coding

Select all _id from mongodb collection with c# driver

I have large document collection in mongodb and want to get only _id list. Mongodb query is db.getCollection('Documents').find({},{_id : 0, _id: 1}). But in C# query
IMongoCollection<T> Collection { get; set; }
...
List<BsonDocument> mongoResult = this.Collection.FindAsync(FilterDefinition<T>.Empty, new FindOptions<T, BsonDocument>() { Projection = "{ _id: 0, _id: 1 }" }).Result.ToList();
throw exeption InvalidOperationException: Duplicate element name '_id'.
I want to get only _id list, other fileds not needed. Documents may have different structures and exclude all other fileds manualy difficult.
What C# query corresponds to the specified mongodb query db.getCollection('Documents').find({},{_id : 0, _id: 1}?
UPDATE: Do not offer solutions related query large amounts of data from the server, for example like
this.Collection.Find(d => true).Project(d => d.Id).ToListAsync().Result;

Since your using C# driver I would recommend to use the AsQueryable and then use linq instead.
In my opinion it is better since you wouldn't need the magic strings and you would benefit from your linq knowledge. Then it would look something like this
database.GetCollection<T>("collectionname").AsQueryable().Select(x => x.Id);

Alexey is correct, solutions such as these
var result = (await this.Collection<Foos>
.Find(_ => true)
.ToListAsync())
.Select(foo => foo.Id);
Will pull the entire document collection over the wire, deserialize, and then map the Id out in Linq To Objects, which will be extremely inefficient.
The trick is to use .Project to return just the _id keys, before the query is executed with .ToListAsync().
You can specify the type as a raw BsonDocument if you don't want to use a strongly typed DTO to deserialize into.
var client = new MongoClient(new MongoUrl(connstring));
var database = client.GetDatabase(databaseName);
var collection = database.GetCollection<BsonDocument>(collectionName);
var allIds = (await collection
.Find(new BsonDocument()) // OR (x => true)
.Project(new BsonDocument { { "_id", 1 } })
.ToListAsync())
.Select(x => x[0].AsString);
Which executes a query similar to:
db.getCollection("SomeCollection").find({},{_id: 1})

C# MongoDb Project with DateTime.date

Given:
I've a mongodb collection with some data. One field is a DateTime field .
Now I want to aggregate the data for each day.
for this I created this aggregation
var result = collection
.Aggregate()
.Project(i => new {i.Key,date = i.Timestamp.Date})
.Group(k => k.date,l => new { l.Key,count =l.Count()})
.ToList();
Problem:
Now Mongo db is telling mit that i.TimeStamp.Date is not supported. I assume that it just cant translate it to a "ToDate" function.
What would be the correct way to group such data by date?

I was checking in cshardriver project and there is no extension like that yet,
that means we need to use bson document to get this working.
var project = BsonDocument.Parse("{Key:1, Timestamp:1,year:{$year:'$Timestamp'}, dayOfYear:{$dayOfYear:'$Timestamp'}}");
var group = BsonDocument.Parse("{_id:{year:'$year', dayOfYear:'$dayOfYear'}, count:{$sum:1}}");
var result = collection
.Aggregate()
.Project(project)
.Group(group)
.ToList();
We could have typed object Project<ournewClassrepresentingProjectShape>(project) and then we can use typed group.
Any comments welcome!

Best Practices: Adding properties to a LINQ-to-Entities query result?

I'm writing an ASP.NET Web Pages application and in it, I have a massive LINQ to Entities query. This query pulls data from a table in the database, filters it, groups the data twice, and adds extra properties to the result set. I then loop through the table, outputting the rows.
The query is quite big, sorry:
accountOrders = db.EventOrders
.Where(order => order.EventID == eventID)
.OrderBy(order => order.ProductCode)
.GroupBy(order => new { order.AccountNum, order.Exhibitor, order.Booth })
.Select(orders =>
new {
Key = orders.Key,
ProductOrders = orders
.GroupBy(order => new { order.ProductCode, order.Product, order.Price })
.Select(productOrders =>
new {
Key = productOrders.Key,
Quantity = productOrders.Sum(item => item.Quantity),
HtmlID = String.Join(",", productOrders.Select(o => (o.OrderNum + "-" + o.OrderLine))),
AssignedLines = productOrders.SelectMany(order => order.LineAssignments)
})
})
.Select(account =>
new {
Key = account.Key,
// Property to see whether a booth number should be displayed
HasBooth = !String.IsNullOrWhiteSpace(account.Key.Booth),
HasAssignedDigitalLines = account.ProductOrders.Any(order => order.AssignedLines.Any(line => line.Type == "digital")),
// Dividing the orders into their respective product group
PhoneOrders = account.ProductOrders.Where(prod => ProductCodes.PHONE_CODES.Contains(prod.Key.ProductCode)),
InternetOrders = account.ProductOrders.Where(prod => ProductCodes.INTERNET_CODES.Contains(prod.Key.ProductCode)),
AdditionalOrders = account.ProductOrders.Where(prod => ProductCodes.ADDITIONAL_CODES.Contains(prod.Key.ProductCode))
})
.ToList();
I use the added properties to help style the output. For example, I use HasBooth property to check whether or not I should output the booth location in brackets beside the exhibitor name. The problem is I have to save this big query as an IEnumerable, meaning I get the error: Cannot use a lambda expression as an argument to a dynamically dispatched operation without first casting it to a delegate or expression tree type. Should I even be manipulating the query this way?
Any advice is much appreciated!

At some point, you are passing in a dynamic datatype to the method, which in turn changes the return type to simply dynamic. You can either cast the dynamic type to a type that is recognised at compile time or explicitly set the return type instead of using var.
You can read more about this issue here: http://www.mikesdotnetting.com/Article/198/Cannot-use-a-lambda-expression-as-an-argument-to-a-dynamically-dispatched-operation

Select Contains With CSV But Keep Sort Order

I have the scenario where I have a IList<Guid> in a variable called csv which are also in a specific order that I need to keep. I am then doing a select contains like so I can get back all my topics based in the list of guids I have.
The guids are from a lucene search which are ordered by the original score from each LuceneResult. Which is why I need to keep them in this order.
var results = _context.Topic
.Where(x => csv.Contains(x.Id));
However. I lose the order the guids came in as soon as I do this. Any idea how I can do this but keep the same order I hand the list of guids to the context and get the topics back in the same order based on the topid.Id?
I have tried the following as mentioned below, by doing a join but they still come out in the same order? Please note that I am paging these results too.
var results = _context.Topic
.Join(csv,
topic => topic.Id,
guidFromCsv => guidFromCsv,
(topic, guidFromCsv) => new { topic, guidFromCsv }
)
.Where(x => x.guidFromCsv == x.topic.Id)
.Skip((pageIndex - 1)*pageSize)
.Take(pageSize)
.Select(x=> x.topic);
** UPDATE **
So I have moved away from just using and guid and am attempting to pass in my lucene model which has the score property that I want to order by. Here is what I have
public PagedList<Topic> GetTopicsByLuceneResult(int pageIndex, int pageSize, int amountToTake, List<LuceneSearchModel> luceneResults)
{
var results = _context.Topic
.Join(luceneResults,
topic => topic.Id,
luceneResult => luceneResult.Id,
(topic, luceneResult) => new { topic, luceneResult }
)
.Where(x => x.luceneResult.Id == x.topic.Id)
.OrderByDescending(x => x.luceneResult.Score)
.Skip((pageIndex - 1) * pageSize)
.Take(pageSize)
.Select(x => x.topic);
var topicResults = results.ToList();
// Return a paged list
return new PagedList<Topic>(topicResults, pageIndex, pageSize, topicResults.Count);
}
However I am now getting the following error? Is what I am doing possible?
Unable to create a constant value of type 'LuceneSearchModel'. Only primitive types or enumeration types are supported in this context.

If I understand the question correctly, you want to filter the Topics based on the csv and you want to get back the results in the same order as the csv. If so:
var results = csv
.GroupJoin(_context.Topic, guid => guid, topic => topic.Id,
(guid, topics) => topics)
.SelectMany(topics => topics);
It is important to note that this treats the _context.Topic as an IEnumerable<T>; therefore, it will fetch all topics from the database and perform the GroupJoin on the client side, not on the database.
EDIT: Based on the comment below, this answer is NOT what you want. I'll just leave the answer here for documentation.

How to query and sort by filtered Max query in EF4?

Context: EF4, C#, .NET4, SQL2008/R2
Tables/entities to repro problem:
Account (long Id, string Name, etc.)
Order (long Id, DateTime
DateToExecute, int OrderStatus, etc.)
AccountOrder (long Id, long
AccountId, long OrderId) <- Yes, one account may have many orders and, likewise, one order may be associated with many accounts.
OrderedItem (long Id, long OrderId, long
ItemId, etc) <- One order may have many items, and we want to eager-load these items (I realize this has performance/data size implications).
Pseudocode (nearly real code) that would be ideal to work:
DateTime startDateInclusive = xxxx;
DateTime stopDateExclusive = yyy;
var query = Db.Accounts.Include(a => a.AccountOrders.Select(ao => ao.Order.Ordereditems.Select(oi => oi.Item)))
.Where(account =>
account.AccountOrders.Where(ao => ao.OrderStatus != 42)
.Max(ao => ao.DateToExecute).IsBetween(startDateInclusive, stopDateExclusive))
.OrderBy(account =>
account.AccountOrders.Where(ao => ao.OrderStatus != 42)
.Max(ao => ao.DateToExecute));
var results = query.Take(5).ToList();
In English, this is looking for the next 5 accounts that have their last order to be executed within a date range. However, there are also Orders that can be cancelled, so we must exclude OrderStatus of 42 when performing that Max.
The problem revolves around this filtered Max date across many-to-many tables. An added complexity is that we need to sort by that filtered max value and we must do all of the above without breaking our eager loading (i.e. joins must be done via projection in the Where and not using a .Join). I’m not sure how to do this query without the result being 10x’s more complex than it should be. I’d hate to do the joins to filter the ao.OrderStatus/Max the DateToExecute 3 times (once for startDate, once for stopDate, and once for the sort). And clearly the IsBetween isn’t functional.
Any ideas on how to perform this query, sorted this way, in a fairly-efficient way for the generated SQL?

It may be helpful to use an anonymous type here:
DateTime startDateInclusive = xxxx;
DateTime stopDateExclusive = yyy;
var query = Db.Accounts
.Select(account => new {
Account = account,
MaxDate = account.AccountOrders.Select(ao => ao.Order).Where(o => o.OrderStatus != 42).Max(o => o.DateToExecute)
})
.Where(a => a.MaxDate >= startDateInclusive && a.MaxDate < stopDateExclusive)
.OrderBy(a => a.MaxDate)
.Select(a => a.Account)
.Include(a => a.AccountOrders.Select(ao => ao.Order.Ordereditems.Select(oi => oi.Item)));
var results = query.Take(5).ToList();
This is untested as I don't have any datasource to test against. But it's probably the simplest approach for what you need to do.