We Keep Coding

C# Mongodb select rows by their sequence number in one query

I have an object in the db which contains an array of strings.
{
"_id": ObjectId("abc123..."),
"Lines":[
"string 1",
"string 2",
"string 3",
"...",
"string 100"
]
}
I need to select rows by their sequence number in one query.
For example, I need lines 1, 7 and 15.
var projection = Builders<ContentEntity>.Projection
.Slice(e => e.Lines2, 1, 1)
.Slice(e => e.Lines2, 7, 1)
.Slice(e => e.Lines2, 15, 1);
var entity = await GetContext(categoryId).ContentEntities
.Find(e => e.Id == contentId)
.Project<ContentEntity>(projection)
.FirstOrDefaultAsync(cancellationToken);
return entity;
If I use the slice operator like this, then I only get row #15.
I need to select rows by their sequence number in one query from Mongodb with С#.

Think that the last .Slice() overwrite the previous .Slice().
Pass the query as BsonDocument instead of using the MongoDB Driver Fluent API which is difficult to implement for your scenario.
Query
db.collection.find({},
{
Lines2: {
$filter: {
input: "$Lines",
cond: {
$in: [
{
$indexOfArray: [
"$Lines",
"$$this"
]
},
[
0,
6,
14
]
]
}
}
}
})
Demo # Mongo Playground
MongoDB .NET Driver syntax
var projection = new BsonDocument
{
{
"Lines", new BsonDocument
{
{
"$filter", new BsonDocument
{
{ "input", "$Lines" },
{
"cond", new BsonDocument
{
{
"$in", new BsonArray
{
new BsonDocument("$indexOfArray", new BsonArray
{
"$Lines",
"$$this"
}),
new BsonArray { 0, 6, 14 }
}
}
}
}
}
}
}
}
};

MongoDB.NET Driver query for $lte and $gte throwing error: An object representing an expression must have exactly one field

FilterDefinition<DM.Content> filterDefinition = Builders<DM.Content>.Filter.Empty;
filterDefinition &= Builders<DM.Content>.Filter.Eq(x => x.IsDeleted, false);
if (contentTypeId > 0)
{
if (contentTypeId == 4)// Photo Video recipes
{
filterDefinition &= Builders<DM.Content>.Filter.In(x => x.ContentTypeId, new List<int>() { 1, 2 });// Video Photo recipes Recipes
}
else
{
filterDefinition &= Builders<DM.Content>.Filter.Eq(x => x.ContentTypeId, contentTypeId);
}
}
// rating
if (!string.IsNullOrEmpty(rating))
{
//filterDefinition &= Builders<DM.Content>.Filter.Gte(x => x.ContentAverageRating, rating);
filterDefinition &= new BsonDocument("$expr", new BsonDocument("$gte",
new BsonArray {
new BsonDocument("$toDouble", "$ContentAverageRating"),
Convert.ToDouble(rating)
}));
}
// cookTime
if (!string.IsNullOrEmpty(cookTime))
{
filterDefinition &= new BsonDocument("$expr", new BsonDocument("$lte",
new BsonArray {
new BsonDocument("$toDouble", "$ContentTime"),
Convert.ToDouble(cookTime)
}));
}
SortDefinition<DM.Content> sortDefinition = Builders<DM.Content>.Sort.Descending(x => x.UpdatedDate);
var results = await contentDocument.AggregateByPage(filterDefinition, sortDefinition, pageIndex, pageSize);
return new DM.CustomModels.ContentSearchResult() { ContentItems = results.data.ToList(), TotalPages = results.totalPages };
public static async Task<(int totalPages, IReadOnlyList<TDocument> data)> AggregateByPage<TDocument>(
this IMongoCollection<TDocument> collection,
FilterDefinition<TDocument> filterDefinition,
SortDefinition<TDocument> sortDefinition,
int page,
int pageSize)
{
var countFacet = AggregateFacet.Create("count",
PipelineDefinition<TDocument, AggregateCountResult>.Create(new[]
{
PipelineStageDefinitionBuilder.Count<TDocument>()
}));
var dataFacet = AggregateFacet.Create("data",
PipelineDefinition<TDocument, TDocument>.Create(new[]
{
PipelineStageDefinitionBuilder.Sort(sortDefinition),
PipelineStageDefinitionBuilder.Skip<TDocument>((page - 1) * pageSize),
PipelineStageDefinitionBuilder.Limit<TDocument>(pageSize),
}));
var aggregation = await collection.Aggregate()
.Match(filterDefinition)
.Facet(countFacet, dataFacet)
.ToListAsync();
var count = aggregation.First()
.Facets.First(x => x.Name == "count")
.Output<AggregateCountResult>()?
.FirstOrDefault()?
.Count ?? 0;
var totalPages = (int)Math.Ceiling((double)count / pageSize);
var data = aggregation.First()
.Facets.First(x => x.Name == "data")
.Output<TDocument>();
return (totalPages, data);
}
I am trying to execute the above query to check ContentAverageRating, ContentTime with certain conditions. But throwing an exception:
Command aggregate failed: An object representing an expression must have exactly one field: { $gte: [ { $toDouble: "$ContentAverageRating" }, 3.0 ], $lte: [ { $toDouble: "$ContentTime" }, 15.0 ] }.
Can anyone let me know what is wrong with the above query?

Issue & Concern
From what I suspect, MongoDB .Net driver had placed both $gte and $lte under the same $expr value, which make the whole BsonDocument failed.
Expected generated Bson Document
{
$and: [
{ $expr: { $gte: [/* Match Condition */] } },
{ $expr: { $lte: [/* Match Condition */] } }
]
}
Actual generated Bson Doccument
{
$and: [
{ $expr: { $gte: [/* Match Condition */], $lte: [/* Match Condition */] } }
]
}
Solution
After some trial and error, I suggest that remove the BsonDocument with $expr from both $lte and $gte. Then you create another FilterDefinition, rootFilterDefinitionto place $expr at top level, then append the filterDefinition as below:
filterDefinition &=
new BsonDocument("$gte",
new BsonArray {
new BsonDocument("$toDouble", "$ContentAverageRating"),
Convert.ToDouble(rating)
}
);
filterDefinition &=
new BsonDocument("$lte",
new BsonArray {
new BsonDocument("$toDecimal", "$ContentTime"),
Decimal.Parse(cookTime)
}
);
FilterDefinition<BsonDocument> rootFilterDefinition = new BsonDocument("$expr",
filterDefinition.ToBsonDocument());
var results = await contentDocument.AggregateByPage(rootFilterDefinition , sortDefinition, pageIndex, pageSize);
Equivalent to this MongoDB query
db.collection.aggregate([
{
$match: {
$expr: {
$and: [
{
$gte: [
{
$toDouble: "$ContentAverageRating"
},
3.0
]
},
{
$lte: [
{
$toDouble: "$ContentTime"
},
15.0
]
}
]
}
}
}
])
Sample Mongo Playground

Group by day using $dateToString in C# MondoDB Driver 2

I want to group an aggregate by day (not dayOfMonth or Year, just absolute day). Therefore I want to use the $dateToString operator as shown here: $dateToString: { format: "%Y-%m-%d", date: "$date" }. Is there a way to use an expression for this like:
var groups = await collection.Aggregate()
.Match(job => /*...*/)
.Group(job => job.Created.ToString(),
group => /*...*/)
.ToListAsync();
I get this error:
ToString of type System.DateTime is not supported in the expression tree {document}{created}.ToString...

With the help of #Blakes Sevens comment, I solved the original problem with another grouping key.
var groups = await collection.Aggregate()
.Match(job => /*...*/)
.Group(job => new
{
Year = job.Created.Year,
Month = job.Created.Month,
Day = job.Created.Day
},
group => new { Key = group.Key, Value = group.Count() })
.ToListAsync();
Edit
To support other periods than day, I had to use a BsonDocument.
var groupBy = new BsonDocument
{
{
"_id", new BsonDocument
{
{ "$add", new BsonArray
{
new BsonDocument
{
{ "$subtract", new BsonArray
{
new BsonDocument { { "$subtract", new BsonArray
{
"$created",
new DateTime(0) }
}
new BsonDocument { { "$mod", new BsonArray
{
new BsonDocument
{
{ "$subtract", new BsonArray
{
"$created",
new DateTime(0)
}
}
},
msPerPeriod
}
} }
}
}
},
new DateTime(0)
}
}
}
},
{ "count", new BsonDocument("$sum", 1) } };
var groups = await collection.Aggregate()
.Match(job => job.Created >= since && regex.IsMatch(job.Definition))
.Group(groupBy)
.Sort(Builders<BsonDocument>.Sort.Ascending(doc => doc["_id"]))
.ToListAsync();
See this answer: https://stackoverflow.com/a/32137234/498298

Can't do multiple groups in MongoDB C# Driver

I am trying to show a statistics count, per month, per manufacturer from a MongoDB collection using Linq.I would like to see my data as:
[{
Manufacturer: '',
Statistics: [{
Month: '',
Counts: 0
}]
}]
So I am trying the following linq query:
var result = _statisticsRepository
.All()
.GroupBy(g => g.ManufacturerId)
.Select(s => new
{
Manufacturer = s.Key,
Statistics = s
.GroupBy(a => a.Values["DATA_MONTH"])
.Select(a => new
{
Month = a.Key,
Counts = a.Sum(d => d.Count)
})
});
When I try this code I get this error:
NotSupportedException: The method GroupBy is not supported in the expression tree: {document}.GroupBy(a => a.Values.get_Item("DATA_MONTH")).
Am I doing this the wrong way or is this a limitation I have in the C# driver? Or is there another way to do this?

You could use the aggregation framework instead LINQ to get the data that you need:
var group1 = new BsonDocument
{
{ "_id", new BsonDocument{{"ManufacturerId", "$ManufacturerId"},{"DATA_MONTH", "$DATA_MONTH"} }},
{ "Count", new BsonDocument
{
{
"$sum", $Count }
} }
};
var group2 = new BsonDocument
{
{ "_id", "$_id.ManufacturerId" },
{ "Statistics ", new BsonDocument { { "$addToSet",new BsonDocument{{"Month", "$_id.DATA_MONTH"},{"Count","$Count"} } } }},
};
var result = database.GetCollection<BsonDocument>("Manufacturers").Aggregate()
.Group(group1)
.Group(group2)
.ToList();

Using Linq to remove from set where key exists in other set?

What is the proper way to do set subtraction using Linq? I have a List of 8000+ banks where I want to remove a portion of those based on the routing number. The portion is in another List and routing number is the key property to both. Here is a simplification:
public class Bank
{
public string RoutingNumber { get; set; }
public string Name { get; set; }
}
var removeThese = new List<string>() { "111", "444", "777" };
var banks = new List<Bank>()
{
new Bank() { RoutingNumber = "111", Name = "First Federal" },
new Bank() { RoutingNumber = "222", Name = "Second Federal" },
new Bank() { RoutingNumber = "333", Name = "Third Federal" },
new Bank() { RoutingNumber = "444", Name = "Fourth Federal" },
new Bank() { RoutingNumber = "555", Name = "Fifth Federal" },
new Bank() { RoutingNumber = "666", Name = "Sixth Federal" },
new Bank() { RoutingNumber = "777", Name = "Seventh Federal" },
new Bank() { RoutingNumber = "888", Name = "Eight Federal" },
new Bank() { RoutingNumber = "999", Name = "Ninth Federal" },
};
var query = banks.Remove(banks.Where(x => removeThese.Contains(x.RoutingNumber)));

This should do the trick:
var toRemove = banks.Where(x => removeThese.Contains(x.RoutingNumber)).ToList();
var query = banks.RemoveAll(x => toRemove.Contains(x));
The first step is to make sure that you don't have to re-run that first query over and over again, whenever banks changes.
This should work too:
var query = banks.Except(toRemove);
as your second line.
EDIT
Tim Schmelter pointed out that for Except to work, you need to override Equals and GetHashCode.
So you could implement it like so:
public override string ToString()
{
... any serialization will do, for instance JSON or CSV or XML ...
... OR any serialization that identifies the object quickly, such as:
return "Bank: " + this.RoutingNumber;
}
public override bool Equals(System.Object obj)
{
return ((obj is Bank) && (this.ToString().Equals(obj.ToString()));
}
public override int GetHashCode()
{
return this.ToString().GetHashCode();
}

Generally it's less work to just pull out the ones you need rather than deleting the ones you don't i.e.
var query = myList.Where(x => !removeThese.Contains(x.RoutingNumber));

Filtering of this type is generally done with generic LINQ constructs:
banks = banks.Where(bank => !removeThese.Contains(bank.RoutingNumber)).ToList();
In this specific case you can also use List<T>.RemoveAll to do the filtering in-place, which will be faster:
banks.RemoveAll(bank => removeThese.Contains(bank.RoutingNumber));
Also, for performance reasons, if the amount of routing numbers to remove is large you should consider putting them into a HashSet<string> instead.

Either use the Linq extension methods Where and ToList to create a new list or use List.RemoveAll which is more efficient since it modifies the original list:
banks = banks.Where(x => !removeThese.Contains(x.RoutingNumber)).ToList();
banks.RemoveAll(x => removeThese.Contains(x.RoutingNumber));
Of course you have to reverse the condition since the former keeps what Where leaves and the latter removes what the predicate in RemoveAll returns.

Have you tried using RemoveAll()?
var query = banks.RemoveAll(p => removeThese.Contains(p.RoutingNumber));
This will remove the any values from banks where a matching record is present in removeThese.
query will contain the number of records removed from the list.
Note: The orginal variable banks will be updated directly by this query; a reassignment is not required.

You can use RemoveAll()
var removedIndexes = banks.RemoveAll(x => removeThese.Contains(x.RoutingNumber));
or
banks = banks.Where(bank => !removeThese.Contains(bank.RoutingNumber)).ToList();