We Keep Coding

Using Queryable Extension to get back Foreign Key Data Value

I have my Unit of Measure which users fill in and save, they can then save a list of Unit Sizes which has its own table and is a foreign key to the Unit Of Measure. When I am fetching all the data back, the Unit Size value is coming back blank.
I have read a half dozen ways to do this and I am not comprehending them. The one that makes the most sense to me is using a Queryable extension so I am trying to go that route but my code still hasn't quite gotten there.
Here is where I am at - these are my entities:
namespace Mudman.Data.Entities
{
[Table("UnitOfMeasure")]
public class UnitOfMeasure : IEntityBase, IAuditBase
{
[Key]
[Column("UnitOfMeasureId")]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public string Id { get; set; }
[Required]
[ForeignKey("TenantId")]
public string TenantId { get; set; }
[JsonIgnore]
public virtual Tenant Tenant { get; set; }
public string Name { get; set; }
public virtual IEnumerable<UnitOfMeasureSize> UnitSize { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime CreateDate { get; set; } = DateTime.UtcNow;
[StringLength(255)]
public string CreateUserId { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime UpdateDate { get; set; }
[StringLength(255)]
public string UpdateUserId { get; set; }
}
}
Unit Of Measure size entity:
namespace Mudman.Data.Entities
{
[Table("UnitOfMeasureSize")]
public class UnitOfMeasureSize : IEntityBase, IAuditBase
{
[Key]
[Column("UnitOfMeasureSize")]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public string Id { get; set; }
[Required]
[ForeignKey("TenantId")]
public string TenantId { get; set; }
[JsonIgnore]
public virtual Tenant Tenant { get; set; }
[Required]
[ForeignKey("UnitOfMeasureId")]
public string UnitOfMeasureId { get; set; }
public virtual UnitOfMeasure UnitOfMeasure { get; set; }
[Required]
public int UnitSize { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime CreateDate { get; set; } = DateTime.UtcNow;
[StringLength(255)]
public string CreateUserId { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime UpdateDate { get; set; }
[StringLength(255)]
public string UpdateUserId { get; set; }
}
}
Unit Of Measure Repository including Unit Size:
namespace Mudman.Repository
{
public class UnitOfMeasureRepository : EntityBaseRepository<UnitOfMeasure>,
IUnitOfMeasureRepository
{
MudmanDbContext context;
public UnitOfMeasureRepository(MudmanDbContext context) : base(context)
{
{ this.context = context; };
}
public IEnumerable<UnitOfMeasure> GetAllUnitsOfMeasure(string TenantId)
{
var result = context.UnitOfMeasure
.Where( uom => uom.TenantId == TenantId)
.Include(uom => uom.UnitSize);
return result;
}
}
}
My GetAllAsync method in my service:
public Task<IEnumerable<UnitOfMeasureViewModel>> GetAllAsync()
{
var result = _unitOfMeasureRepo.GetAllUnitsOfMeasure(TenantId);
result.OrderBy(r => r.Name);
return _mapper.Map<List<UnitOfMeasure>, List<UnitOfMeasureViewModel>>(result.ToList());
}
AutoMapper Code:
CreateMap<UnitOfMeasure, UnitOfMeasureViewModel>().ReverseMap()
.ForMember(dest => dest.UnitSize, uos => uos.Ignore())
.ForMember(uom => uom.UnitSize, src => src.MapFrom(uom => uom.UnitSize));

There are a few issues with your attempts so far.
Firstly, your GetAllAsync looks like it wants to be an async method but you have it making entirely synchronous calls, and hasn't been marked as async. I would avoid diving into asynchronous methods until you have the fundamentals of retrieving your data down.
What we cannot see from your example is the mapping between your unit of measure entity and the view model. The entity has a one-to-many relationship between unit of measure and UnitSizes, so what gets updated depends on how the view model is laid out and configured for mapping. This is most likely the root of your problem where your view model mapping from the entity is likely relying on a convention that isn't pairing up with the data you expect.
Performance wise, this approach will run into problems as your data model grows in terms of entities and rows. The fundamental problem with using a repository like this is that a method like this:
IEnumerable<UnitOfMeasure> GetAllUnitsOfMeasure(string TenantId)
will load all data into memory and you explicitly need to include related entities, whether the consumer will want them or not, which adds to the amount of work the queries need to do and the memory required. If TenantId is for something like a multi-tenant database such as in a SaaS application with multiple tenants using a single data source, this is a good reason to adopt a Repository pattern, but I would not pass tenantIds around as parameters. Instead, have the repository accept a dependency that can validate and resolve the current TenantId from the session. This way the repository can always ensure that the current tenant rules are validated and applied for every query without worrying about where the caller might have gotten a TenantId from. (I.e accepting a TenantId from a POST request would be bad as that value could easily be tampered with)
To address performance and probably touch on what you had read about IQueryable extensions, rather than returning IEnumerable<TEntity> from a repository, you can return IQueryable<TEntity>. The advantages here are that you can still have the repository add base filtering rules like the tenantID, and allow the consumer to handle things like sorting and projection.
For example, the repository looks more like:
public class UnitOfMeasureRepository : IUnitOfMeasureRepository
{
private readonly MudmanDbContext _context;
private readonly ICurrentUserLocator _currentUserLocator;
public UnitOfMeasureRepository(MudmanDbContext context, ICurrentUserLocator currentUserLocator )
{
_context = context ?? throw new ArgumentNullException("context");
_currentUserLocator = currentUserLocator ?? throw new ArgumentNullException("currentUserLocator");
}
public IQueryable<UnitOfMeasure> GetUnitsOfMeasure()
{
var tenantId = _currentUserLocator.CurrentUserTenantId; // Checks session for current user and retrieves a tenant ID or throws an exception. (no session, etc.)
var query = _context.UnitOfMeasure
.Where( uom => uom.TenantId == tenantId)
return query;
}
}
The changes to note here is that we do away with the base generic repository class. This was confusing as you were passing the context to a base class then setting a local context instance as well. Generic repositories with EF are a bad code smell as they lead to either very complex code, very poor performing code, or both. There is a CurrentUserLocator with the container can inject which is a simple class that can verify that a user is currently authenticated and can return their Tenant ID. From there we will return an IQueryable<UnitOfMeasure> which has a base filter for the TenantID which will allow our consumers to make up their own minds how they want to consume it. Note that we do not need to use Include for related entities, again the consumers can decide what they need.
Calling the new repository method and projecting your view models looks fairly similar to what you had. It looks like you are using Automapper, rather than using .Map() we can use .ProjectTo() with the IQueryable and Automapper can essentially build a Select() expression to pull back only the data that the view model will need. To use ProjectTo extension method we do need to provide it with the MappingConfiguration that was used to create your mapper and that will tell it how to build the ViewModel. (So rather than having a dependency of type 'Mapper' you will need one for the MapperConfiguration you set up for that mapper.)
public IEnumerable<UnitOfMeasureViewModel> GetAll()
{
var models = _unitOfMeasureRepo.GetUnitsOfMeasure()
.OrderBy(r => r.Name)
.ProjectTo<UnitOfMeasureViewModel>(_mapperConfiguration)
.ToList();
}
What this does is call our repository method to get the IQueryable, which we can then append the ordering we desire, and call ProjectTo to allow Automapper to populate the view models before executing the query with ToList(). When using Select or ProjectTo we don't need to worry about using Include to eager load related data that might be mapped, these methods take care of loading data related entities if/when needed automatically.
Even in cases where we want to use a method like this to update entities with related entities, using IQueryable works there to:
public void IncrementUnitSize(string unitOfMeasureId)
{
var unitOfMeasure = _unitOfMeasureRepo.GetUnitsOfMeasure()
.Include(r => r.UnitSizes)
.Where(r => r.Id == unitOfMeasureId)
.Single();
foreach(var unitSize in unitOfMeasure.UnitSizes)
unitSize.UnitSize += 1;
_context.SaveChanges();
}
Just as an example of fetching related entities as needed, versus having a method that returns IEnumerable and needs to eager load everything just in case some caller might need it.
These methods can very easily be translated into an asyncronous method without touching the repository:
public async Task<IEnumerable<UnitOfMeasureViewModel>> GetAll()
{
var models = await _unitOfMeasureRepo.GetAllUnitsOfMeasure(TenantId)
.OrderBy(r => r.Name)
.ProjectTo<UnitOfMeasureViewModel>(_mapperConfiguration)
.ToListAsync();
}
... and that is all! Just remember that async doesn't make the call faster, if anything it makes it a touch slower. What it does is make the server more responsive by allowing it to move the request handling to a background thread and free the request thread to pick up a new server request. That is great for methods that are going to take a bit of time, or are going to get called very frequently to avoid tying down all of the server request threads leading to timeouts for users waiting for a response from the server. For methods that are very fast and aren't expected to get hammered by a lot of users, async doesn't add a lot of value and you need to ensure every async call is awaited or you can end up with whacky behaviour and exceptions.

How to stop Entity Framework from including properties I don't want it to?

When reading an entity type from my DataContext, I get all the associated objects when I don't want them. How do I set EF up so I only do explicit loading?
Reading up on msdn info like from here:
https://devblogs.microsoft.com/dotnet/announcing-entity-framework-core-2-2/
It seems like I should get nothing for free, so explicit loading is the way, however I'm using the code below but my results are more than I would expect.
public class TalesContext : DbContext
{
public TalesContext()
{
}
public TalesContext(DbContextOptions<TalesContext> options) : base(options)
{
}
protected internal DbSet<Story> Stories { get; set; }
protected internal DbSet<Event> Events { get; set; }
protected internal DbSet<StoryEventMention> EventMentions { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (optionsBuilder.IsConfigured) return;
optionsBuilder.UseInMemoryDatabase("TalesTesting");
}
}
public class Event
{
[Required]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
public IList<StoryEventMention> EventMentions { get; set; }
[MaxLength(128)]
public string Title { get; set; }
}
var query = from e in TalesContext.Events select e;
// various query.Where
query = query.Skip((pageNumber - 1) * pageSize).Take(pageSize);
return query.ToList();
I would expect to get a list of Events with the Ids and Titles populated and EventMentions as null. However I get EventMentions populated along with all further navigation properties. Pretty much the entire test database.
I get this when I run a unit test and when I expose this through an API.

I found my mistake.
My EventFetcher class is a singleton, registered through IoC. It has a reference to TalesContext also registered as a singleton through IoC. So I had one DataContext through the application. So when the first request came in, it seeded the database - and so had everything in it. Thus all the references between the objects were already built and when I requested one without explicitly including it, the context returned the data it already had with everything attached.
I did a restructure on this so that a new context is injected into the controller each request and the behaviour is exactly what you would expect. So the lesson here is to be mindful of the age and persistence of your data context if you start getting unexpected results when querying.
Thanks for the assistance!

To answer your question. I believe what you need to do is
inside your TalesContext call this :
this.Configuration.LazyLoadingEnabled = false;
Sources:
explains lazy loading:
https://www.entityframeworktutorial.net/lazyloading-in-entity-framework.aspx
Explains the difference between eager loading and lazy loading
https://learn.microsoft.com/en-us/aspnet/mvc/overview/getting-started/getting-started-with-ef-using-mvc/reading-related-data-with-the-entity-framework-in-an-asp-net-mvc-application

Losing connection when returning entity with navigational properties using Include(...)

After quite some digging and confusing error messages, I've arrived at this sample (which I believe to be the smallest example reproducing the issue). I can almost certainly conclude that the issue appears due to the entity linked to the one that I'm returning.
[OperationContract]
[WebGet(UriTemplate = "Stations")]
List<Station> GetStations();
public List<Station> GetStations()
{
List<Station> stations = new List<Station>();
using (Context context = new Context())
foreach (Station station in context.Stations.Include(element => element.Records))
//stations.Add(station.Copy());
stations.Add(station);
return stations;
}
It works if I activate the line with Copy() (which creates a new instance of a station and copies over all the properties except for the records, which it creates by itself). However, when I just add the stations without creating a copy (regardless of whether I keep the records, nullify them or set en empty list), it doesn't roll well.
Since I used Include(), the objects being disposed isn't the issue anymore. The error message I get says in the console like so.
http://localhost:25760/MyService.svc/Stations net::ERR_CONNECTION_RESET
Googling that gave me a lot of references to Apache (I'm running it on IIS), PHP (I'm building it on .NET) and security issues with certificates (I'm not using any and the other calls work well).
So my suspicion is either that the error message is misleading coming from a confused computer or that I'm missing something in my setup. The autogenerated classes reflect the foreign key I added to the tables and look like this.
alter table Records
add constraint FkStationId
foreign key (StationId)
references Stations (Id)
public partial class Record
{
public System.Guid Id { get; set; }
public Nullable<System.Guid> StationId { get; set; }
...
public virtual Station Station { get; set; }
}
public partial class Station
{
[System.Diagnostics.CodeAnalysis.SuppressMessage(
"Microsoft.Usage", "CA2214:DoNotCallOverridableMethodsInConstructors")]
public Station() { this.Records = new HashSet<Record>(); }
public System.Guid Id { get; set; }
...
[System.Diagnostics.CodeAnalysis.SuppressMessage(
"Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
public virtual ICollection<Record> Records { get; set; }
}
I see nothing that leads me to any idea on how to trouble-shoot it. This error shouldn't happen. On the other hand - Fukushima shouldn't happen neither. But it did.

Finally I got this running.
Turn off proxy creation.
Turn off lazy loading.
Make virtual property ignorable to serialization.
Load in the linked entity explicitly.
The first two items are done in the constructor of the context.
public partial class Context : DbContext
{
public Context() : base("name=ContextConnection")
{
Configuration.LazyLoadingEnabled = false;
Configuration.ProxyCreationEnabled = false;
}
...
}
The third is done by attributing one of the virtual properties that lead to circular dependency as not valid for serialization.
public partial class Station
{
...
[IgnoreDataMemeber]
public virtual ICollection<Record> Records { get; set; }
}
The last one is including or omitting the navigational property to be (or not to be displayed). In this case, it made sense to jam in the information about stations into each record. The stations can be presented without the records, though.
public List<Station> GetStations()
{
using (Context context = new Context())
return context.Stations
.ToList();
}
public List<Record> GetRecords()
{
using (Context context = new Context())
return context.Records
.Include(record => record.Station)
.ToList();
}
Having said that, there will be dragons. This approach leads to a lot of work as it requires to manually re-edit the auto-generated files each time they're re-created. So i went with Code First, instead.

eager loading with dbcontext and projection

We are converting our project from ObjectContext to dbContext.
Our current problem is with the difference in how eager loading is handled.
Example context
public class Person
{
public virtual ICollection<Email> Emails { get; set; }
public virtual ICollection<Post> Posts { get; set; }
}
public class Email
{
public string Address{ get; set; }
}
public class Post
{
public string Content{ get; set; }
}
we have many pieces of code throughout the enterprise that expect the emails to be loaded and therefore call person.Emails.First() without thinking about it.
So we need to make sure that Emails are eagerly loaded.
Sometimes we can just us Include
However, when we use projections in our data layer we are running into problems. i.e.
return context.Persons.Select(p=> new Top5VM {
Person = p,
TopPosts = p.Posts.Take(5)
};
We have a lot of code that relies on Top5VM and expects Person.Emails to be loaded.
No mater what we've tried we can not figure out where to put the Include (or Load) function call where it will actually make a difference .
With the ObjectContext we would just have a dummy property on the Top5VM called Emails. Once that was loaded, the ObjectContext had references to all of those Entities an therefore never needed to go back to the server even when we accessed them through the person object. But that no longer works with the DbContext

Figured it out. I need to set context.Configuration.LazyLoadingEnabled = false; any time I want to use projection to accomplish eager loading. Unless of course I want to access the loaded entities directly from the projection.

You can do this by always using your own DbContext in between:
public class MyDbContext : DbContext
{
public MyDbContext() : base()
{
Configuration.LazyLoadingEnabled = false;
}
}
Then use this everywhere instead of a DbContext.
Alternately, if you have control of all the POCOs, you could just remove the virtual keyword from the related collections ... the virtual keyword is the magic hookup Microsoft uses for lazy loading.
EDIT
I also tend to do this as an extension method:
static class DbContextExtensions
{
public static DbContext AsEagerLoadingContext(this IDbContext context)
{
context.Configuration.LazyLoadingEnabled = false;
//context.Configuration.AutoDetectChangesEnabled = false;
return context;
}
}
Used as so, allowing lazy loading to be used or not as needed:
using (var context = new DbContext().AsEagerLoadingContext())
context.Stuff.Select(s => s.AllTheThings);

Why are my navigational properties null when retrieved from the database in EF 4.2 POCO?

I have a exceedingly simplistic data model (below). I am having trouble figuring out how I am to get my navigational properties to load from the database. I have no trouble getting them in, but the navigational property does not get set by EF it appears. I have seen several related questions, but they are slightly different or rather involved. I am looking for information on how navigational properties are treated by EF 4.2 (POCO). In the reading I've done, I got the impression that I would be able to access objects with foreign keys using navigational properties. Instead, my properties are coming back as either null or empty depending on if I instantiate my collection in the constructor.
public class AnimalDb : DbContext
{
public static AnimalDb Create(string fileName)
{
Database.DefaultConnectionFactory = new SqlCeConnectionFactory("System.Data.SqlServerCe.4.0");
return new AnimalDb(fileName);
}
private AnimalDb(string fileName) : base(fileName) { }
public DbSet<Animal> Animals { get; set; }
}
public class Animal
{
public Animal()
{
Id = Guid.NewGuid();
Traits = new ObservableCollection<Trait>();
}
public Guid Id { get; set; }
public string Species { get; set; }
public string Name { get; set; }
public ObservableCollection<Trait> Traits { get; set; }
}
public class Trait
{
public Trait()
{
Id = Guid.NewGuid();
}
public Guid Id { get; set; }
public string Name { get; set; }
}
And here is some (simple) code that uses it:
foreach (var animal in db.Animals)
{
foreach (var trait in animal.Traits)
{
//animal.Traits count is 0, so this does not run.
//However there are traits in the database, as my populate
//function is working fine.
Console.WriteLine("{0} is {1}", animal.Name, trait.Name);
}
}
----Edit Answer Summary----
Using the article and information provided in the answers below, I was able to discover I could either eagerly load using db.Animals.Include() or enable lazy loading. There is a trick to enabling lazy loading and being able to use it though. First to enable lazy loading I added:
db.Configuration.LazyLoadingEnabled = true;
Next I changed my Traits collection in the following manner:
public virtual ObservableCollection<Trait> Traits { get; set; }
Making it virtual allows the automatically generated proxy to lazily load Traits. That's it! IMHO I think the MSDN docs should shout this load and clear in the POCO EF 4.2 coding conventions. Again thanks for the help.

There are a few reasons that your wire-up methods may appear to have no data. To load related data you need to :
explicity load the data
meet the lazy loading requirements, or
use eager loading using Include()
My guess is that you turned off the virtual proxies. There is more on the requirements here:
http://msdn.microsoft.com/en-us/library/dd456855.aspx

If you don't use lazy loading you have to explicitly tell EF to load the relation with the Include method:
foreach (var animal in db.Animals.Include(a => a.Traits))
{
foreach (var trait in animal.Traits)
{
//...
}
}
You can read more about eager loading in this article.