We Keep Coding

Single Element Using Lookup operator

I am stuck in an issue with MongoDB. I am using C# driver for MongoDB.
So basically I am trying to join Categories document with Users document. Each category document has one property:
public Guid UserId { get; set; }
This property is an ObjectId behind the scenes. I have another property:
public UserDoc User { get; set; }
Now I am trying to fill this User property with all the user details based on the UserId property. This is the code I am trying to achieve this:
categoriesCollection.Aggregate()
.Lookup<CategoryDoc, UserDoc, CategoryDoc>(
usersCollection,
x => x.UserId,
x => x.Id,
x => x.User)
As expected, 'Lookup' is expecting an array of User documents but I have a property referencing a single user object and thus, it throws an error:
An error occurred while deserializing the User property of class TatSat.API.Documents.CategoryDoc: Expected a nested document representing the serialized form of a TatSat.API.Documents.UserDoc value, but found a value of type Array instead.
Can someone help me with this? I am new to Mongo so this is a bit of a pain for me. Kindly note that I am looking for a strongly typed solution and don't want to mess with BsonDocuments if that can be avoided.
Thanks in advance.

So I finally figured out in some ways. Basically I decided to have another class:
public class CategoryDocWithTemporaryData : CategoryDoc
{
public UserDoc[] Users { get; set; }
public static Expression<Func<CategoryDocWithTemporaryData, CategoryDoc>> ToCategoryDoc => c =>
new CategoryDoc
{
Id = c.Id,
//other properties
User = c.Users.First()
};
}
Then I use the lookup as:
categoriesCollection.Aggregate()
.Lookup<CategoryDoc, UserDoc, CategoryDocWithTemporaryData>(
usersCollection,
c => c.UserId,
c => c.Id,
c => c.Users)
.Project(CategoryDocWithTemporaryData.ToCategoryDoc)
This, however needs me to use additional projection where I have to manually select all properties which is something I wanted to avoid. But I think I can live with it until I come across a better approach.

Lookup will always return an array, even if it only has 1 element. If you need just the first (or only) document in that array, use a project/set/addFields stage with arrayElemAt.

How to compute object property in database with C# code?

As an example let's say my database has a table with thousands of ships with every ship potentially having thousands of passengers as a navigation property:
public DbSet<Ship> Ship { get; set; }
public DbSet<Passenger> Passenger { get; set; }
public class Ship
{
public List<Passenger> passengers { get; set; }
//properties omitted for example
}
public class Passenger
{
//properties omitted for example
}
The example use case is that someone is fetching all ships per API and would like to know for each ship whether it is empty (0 passengers), so the returned JSON will contain a list of ships each with a bool whether it is empty.
My current code seems very inefficient (including all passengers just to determine if a ship is empty):
List<Ship> ships = dbContext.Ship
.Include(x => x.passengers)
.ToList();
and later when the ships are serialized to JSON:
jsonShip.isEmpty = !ship.passengers.Any();
I would like a more performant (and not bloated) alternative to including all passengers. What options do I have?
I have looked at computed columns but they only seem to support sql as string. If possible I would like to stay in the C# code world, so for example having a property which is set correctly by being automatically woven in the SQL query would be optimal.

Create a Data Transfer Object for Ship that reflects the shape of your JSON result, like -
public class ShipDto
{
public int Id { get; set; }
public string Name { get; set; }
public bool IsEmpty { get; set; }
}
Then use projection in your query -
var ships = dbCtx.Ships
.Select(p => new ShipDto
{
Id = p.Id,
Name = p.Name,
IsEmpty = !p.Passengers.Any()
})
.ToList();
Usually, APIs need to produce responses of various shapes and DTOs give you well defined models to represent the shape of your API response. Domain entities are not always suitable for this.
If your domain entity (Ship) has a lot of properties, then copying all those properties in the .Select() method might be cumbersome. You can use AutoMapper to map them for you. AutoMapper has a ProjectTo<T>() method that can generate the SQL and return the projected result. For example, you can achieve the above result with a mapping configuration -
CreateMap<Ship, ShipDto>()
.ForMember(d => d.IsEmpty, opt => opt.MapFrom(s => !s.Passengers.Any()));
and a query -
var ships = Mapper.ProjectTo<ShipDto>(dbCtx.Ships).ToList();
assuming all other properties in ShipDto are named similar as in Ship entity.
EDIT :
If you don't want a DTO model -
you can add a NotMapped property in Ship model -
public class Ship
{
public int Id { get; set; }
public string Name { get; set; }
[NotMapped]
public bool IsEmpty { get; set; }
public List<Passenger> passengers { get; set; }
}
and then do the query like -
var ships = dbCtx.Ships
.Select(p => new Ship
{
Id = p.Id,
Name = p.Name,
IsEmpty = !p.Passengers.Any()
})
.ToList();
You can return an anonymous type -
var ships = dbCtx.Ships
.Select(p => new
{
Id = p.Id,
Name = p.Name,
IsEmpty = !p.Passengers.Any()
})
.ToList();

If I understand your intention correctly...
One way is to store the number of passengers inside each Ship entity. This can work well if you use Domain Driven Design, treat the Ship as an aggregate root, and only add or remove passengers through methods exposed on the given Ship entity, e.g. RegisterPassenger() / RemovePassenger(). Inside these methods, increment or decrement the passenger number along with adding or removing the passenger.
Then, obviously, you can query the Ships dbset with a PassengerCount < 0 projection to the bool you need. And, again obviously, it won't even touch the Passengers table.
In traditional anemic domain ASP.NET systems this sort of data redundancy might be a bit more risky, because properties are usually publicly mutable, and you have multiple services that 'massage' the entities, which is a potential source of data integrity loss.

"There is already an open DataReader associated with this Command which must be closed first" error on Entity Framework linq query

I regularly (but not always) get this error
There is already an open DataReader associated with this Command which must be closed first
when running a query shown in GetRecentClients. After I looked it up, I've found out that it has to do something with 2 connections being opened at the same time and that turning MARS on should help, but I would prefer not to do it.
Is there any other way around it, and where exactly this second connection comes from? There's no includes or anything like that going on, the meeting is a pretty straightforward entity with only basic type properties.
public static async Task<RecentClientsModel> GetRecentClients(int managerId, IUnitOfWork unitOfWork)
{
var recentViews = (await unitOfWork.GetRepository<Meeting>().Get(
source => source
.Where(a => a.Type == StatusType.Viewed && a.ManagerId == managerId)
.GroupBy(c => c.ClientId)
.Select(gr => gr.OrderByDescending(g => g.Date).FirstOrDefault())
.OrderByDescending(a => a.Date)
.Take(10))).ToArray();
//...
}
public class Meeting
{
public int Id { get; set; }
public DateTime Date { get; set; }
public StatusType Type { get; set; }
public int? ClientId { get; set; }
public int? ManagerId { get; set; }
}
public async Task<IEnumerable<T>> Get(Func<IQueryable<T>, IQueryable<T>> queryBuilder)
{
return await queryBuilder(_context.Set<T>()).ToListAsync();
}

Found this answer: Entity Framework: There is already an open DataReader associated with this Command
Which says that it's probably triggered due to nested queries or lazy loading. It sounds like the only way to fix this is to not use lazy loading or to enable MARS (Multiple Active Result Sets).
In your case, it's probably not an actual second connection, but another result set coming from the same connection. I also wonder if it could be due to the ToListAsync().
Actually, now that I look at it again, it could be something with the gr.OrderByDescending() call in the middle of the query? That might be attempting to enumerate the result set in the middle of the outer result set. Maybe try to rewrite the query (even incorrectly) to avoid something like that, and see if the problem goes away.

C# EntityFramework 6.0 - How to use Where Statement in EntityTypeConfiguration?

I have 2 classes like this:
Parent.cs
public class Parent
{
public int Id {get;set;}
public virtual ICollection<Child> Children { get; set; }
}
Child.cs
public class Child
{
public int Id {get;set;}
public ItemStatusType ItemStatusTyp { get; set; }
public int ParentId {get;set;}
[ForeignKey("ParentId")]
public virtual Parent Parent { get; set; }
}
ItemStatusType.cs
public enum ItemStatusType
{
Active = 1,
Deactive = 2,
Deleted = 3
}
What I want is to somehow retrieve always the active ones and not the deleted ones. Since I am not deleting the record physically, I'm merely updating the ItemStatusType to Deleted status.
So, when I say ParentObj.Children I only wish to retrieve the active ones without further using Where condition.
Here is so far what I've done but giving an exception on runtime that I stated afterwards:
public class ParentConfiguration : EntityTypeConfiguration<Parent>
{
public ParentConfiguration()
{
HasMany(c => c.Children.Where(p => p.ItemStatusTyp != ItemStatusType.Deleted).ToList())
.WithRequired(c => c.Parent)
.HasForeignKey(c => c.ParentId)
;
}
}
Runtime Exception:
The expression 'c => c.Children.Where(p => (Convert(p.ItemStatusTyp)
!= 3)).ToList()' is not a valid property expression. The expression
should represent a property: C#: 't => t.MyProperty' VB.Net:
'Function(t) t.MyProperty'.
I had to use ToList after the expression, otherwise it does not compile.
What is the proper what to do what I want?
Thanks in advance,

You cannot use Where or any other logic in fluent property mapping - that's just configuration.
Basically you cannot solve what you need in declarative way.
There are some workarounds which you can use for first-level entities, like implement your own extension(s) MySet<T> which will return .Set<T>.Where(x => x.ItemStatusType != ItemStatusType.Deleted) and use it everywhere, but that won't solve filtering issue for child collections.
You can go hard way and prepare a separate set of entities to use for selecting data, which basically should be based on database views or stored procedures; you will have to create separate view for every entity, so you will be able to combine selecting in any relations based on these views-entities.
For inserting though you will need to have entities mapped over "real" tables. No sure if it worth it but it might in some cases.

Class with multiple List Properties of same type but with different restrictions

Here's my problem: I have a class that have 2 list properties of the same class type (but with some different restriction as on how to be filled), let's say:
public class Team
{
[Key]
public int IDTeam { get; set; }
public string TeamName { get; set; }
public List<Programmer> Members { get; set; }
public List<Programmer> Leaders { get; set; }
public LoadLists(MyProjectDBContext db)
{
this.Members = db.Programmers.Where(p => p.IDTeam = this.IDTeam
&& (p.Experience == "" || p.Experience == null)).ToList();
this.Leaders = db.Programmers.Where(p => p.IDTeam = this.IDTeam
&& (p.Experience != null && p.Experience != "")).ToList();
}
}
public class Programmer
{
[Key]
public int IDProgrammer { get; set; }
[ForeignKey("Team")]
public int IDTeam { get; set; }
public virtual Team Team { get; set; }
public string Name { get; set; }
public string Experience { get; set; }
}
At some point, I need to take a list of Teams, with it's members and leaders, and for this I would assume something like:
return db.Teams
.Include(m => m.Members.Where(p => p.Experience == "" || p.Experience == null)
.Include(l => l.Leaders.Where(p => p.Experience != null && p.Experience != "")
.OrderBy(t => t.TeamName)
.ToList();
And, of course, in this case I would be assuming it wrong (cause it's not working at all).
Any ideas on how to achieve that?
EDIT: To clarify a bit more, the 2 list properties of the team class should be filled according to:
1 - Members attribute - Should include all related proggramers with no experience (proggramer.Experience == null or "");
2 - Leaders attribute - Should include all related proggramers with any experience (programmer.Experiente != null nor "");
EDIT 2: Here's the MyProjectDbContext declaration:
public class MyProjectDBContext : DbContext
{
public DbSet<Team> Teams { get; set; }
public DbSet<Programmer> Programmers { get; set; }
}

You are talking about EntityFramework (Linq to entities) right? If so, Include() is a Method of Linq To Entities to include a sub-relation in the result set. I think you should place the Where() outside of the Inlcude().
On this topic you'll find some examples on how to use the Include() method.
So I suggest to add the Include()'s first to include the relations "Members" and "Leaders" and then apply your Where-Statement (can be done with one Where()).
return db.Teams
.Include("Team.Members")
.Include("Team.Leaders")
.Where(t => string.IsNullOrWhitespace(t.Members.Experience) ... )
What is unclear to me is your where criteria and your use-case at all as you are talking of getting a list of Teams with Leaders and Members. May above example will return a list of Teams that match the Where() statement. You can look though it and within that loop you can list its members and leaders - if that is the use-case.
An alternative is something like this:
return db.Members
.Where(m => string.IsNullOrWhitespace(m.Experience))
.GroupBy(m => m.Team)
This get you a list of members with no experience grouped by Team. You can loop the groups (Teams) and within on its members. If you like to get each team only once you can add a Distinct(m => m.Team) at the end.
Hope this helps. If you need some more detailed code samples it would help to understand your requirements better. So maybe you can say a few more words on what you expect from the query.
Update:
Just read our edits which sound interesting. I don't think you can do this all in one Linq-To-Entities statement. Personally I would do that on the getters of the properties Members and Leaders which do their own query (as a read-only property). To get performance for huge data amount I would even do it with SQL-views on the DB itself. But this depends a little on the context the "Members" and "Leaders" are used (high frequent etc).
Update 2:
Using a single query to get a table of teams with sublists for members and leaders I would do a query on "Programmers" and group them nested by Team and Experience. The result is then a list of groups (=Teams) with Groups (Experienced/Non-experience) with Programmers in it. The final table then can be build with three nested foreach-Statements. See here for some grouping examples (see the example "GroupBy - Nested").

Whenever you fetch entities, they will be stored in the context -- regardless of the form they are "selected" in. That means you can fetch the teams along with all the necessary related entities into an anonymous type, like this:
var teams =
(from team in db.Teams
select new {
team,
relatedProgrammers = team.Programmers.Where(
[query that gets all leaders OR members])
}).ToList().Select(x => x.team);
It looks like we're throwing away the relatedProgrammers field here, but those Programmer entities are still in memory. So, when you execute this:
foreach (var team in teams) team.LoadLists(db);
...it will populate the lists from the programmers that were already fetched, without querying the database again (assuming db is the same context instance as above).
Note: I haven't tested this myself. It's based on a similar technique shown in this answer.
EDIT - Actually, it looks like your "leaders" and "members" cover all programmers associated with a team, so you should be able to just do Teams.Include(t => t.Programmers) and then LoadLists.