Let's say I have a method like this in one of my controllers:
[Route("api/Products")]
public IQueryable<Product> GetProducts() {
return db.Products
.Include(p => p.Category);
}
Using this I can get a product from the database and include its Category property.
In my CategoryControllerI have this method:
[Route("api/Categories")]
public IQueryable<Category> GetCategories() {
return db.Categories
.Include(c => c.Parent)
.Include(c => c.Products)
.Include(c => c.SubCategories);
}
When I send a GET request to the CategoryController this works as intended, I get the category, its parent, its products and its sub-categories. But when I send a GET request to the ProductController I don't want to include all the products in the category of the requested product, I just need the basic information about that category.
So, how can I make GetProducts() return the products in the database, including the Category property of each product, but excluding the Products list property of the category, still keeping the other properties like id, title and so on?
Thank you.
As said in the comments, the first step is to disable lazy loading. You can either do that by removing the virtual modifier from the collection properties, which is permanent, or by disabling it per context instance, which is temporary:
context.Configuration.ProxyCreationEnabled = false;
(disabling proxy creation also disables lazy loading, but keeps the generated objects more light-weight).
In disconnected scenarios, like web API, people often prefer to disable lazy loading by default, because of this serializer-lazy-loading cascade.
However, you can't stop Entity Framework from executing relationship fixup. Loading a Productattaches it to the context. Include()-ing its categories attaches those to the context and EF populates their Products collections with the attached product, whether you like it or not. Circular references will still be a problem.
You can somewhat reduce this effect by fetching the products with AsNoTracking (which prevents entities to get attached, i.e. change-tracked):
return db.Products.AsNoTracking()
.Include(p => p.Category);
Now categories will only have their Products filled with the Product of which they are the category.
By the way, in disconnected scenarios, also using AsNoTracking is preferred. The entities won't ever be saved by the same context instance anyway and it increases performance.
Solutions
Return DTOs, not entity types
By using DTO objects you take full control over the object graph that will be serialized. Lazy loading won't surprise you. But yeah, the amount of required DTO classes can be overwhelming.
Return anonymous types.
This will raise some eyebrows because we should never return anonymous types from methods, right? Well, they leave an action method as a Json string, just as named types, and the javascript client doesn't know the distinction. You might say that it only brings the weakly typed javascript environment one step closer. The only thing is that a named DTO type serves as a data contract (of sorts) and anonymous types can be changed (too) easily and break client-side code.
Tweak the serializer.
You can tell the Json.Net serializer to ignore reference loops. Using JsonConvert directly, it looks like so:
var products = db.Products.AsNoTracking().Include(p => p.Category);
var setting = new JsonSerializerSettings
{
Formatting = Newtonsoft.Json.Formatting.Indented, // Just for humans
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
};
var json = JsonConvert.SerializeObject(products, setting);
In combination with AsNoTracking() this will serialize the categories with empty Products arrays ("Products": []), because Product - Category - Product is a reference loop.
In Web API there are several ways to configure the built-in Json.Net serializer, you may want to do this per action method.
Personally, I prefer using DTOs. I like to be in control (also over the properties that cross the wire) and I don't particularly like to rely on a serializer to solve for me what I neglected to do.
I believe it's not the best practice but it will work, you could create new object and fill it.
[Route("api/Products")]
public IQueryable<Product> GetProducts() {
return db.Products
.Include(p => p.Category);
.Select(x => new Product{
Name = x.Name,
Price = x.Price,
Category = new Category{
Name = x.Category.Name}})
}
Related
I have issues with DTO mapping. I'm using OData WebAPI and I need to use IQueryable due to paging, sorting, filtering...
When I use this code (simplified for this purpose) in my WebAPI controller, it works
return Ok(_dataService.GetEntities(idUser).Select(e => new EntityDTO
{
ID = e.ID,
Name = e.Name
}));
but when I have separate method for DTO mapping it does not work.
return Ok(_dataService.GetEntities(idUser).Select(e => _dataService.MapToEntityDTO(e)));
Methods in my _dataService object (simplified)
public IQueryable<Entity> GetEntities(long idUser)
{
return from z in _context.Entities select z;
}
public EntityDTO MapToEntityDTO(Entity entity) {
return new EntityDTO {
ID = entity.ID,
Name = entity.Name
};
}
Could you please someone explain me what is wrong with that ? Thanks for help.
It would appear that GetEntities is returning an IQueryable that EF would be deferring execution until the results need to be materialized. This would fail if you try calling some arbitrary C# method (MapToEntity) because EF cannot translate that down into SQL.
What you should consider using is Automapper which contains a method called ProjectTo which can integrate with EF's IQueryable to project your DTOs through a configured mapper.
var dtos = _dataService.GetEntities(idUser)
.ProjectTo<EntityDTO>(config)
.ToList();
return Ok(dtos);
Where "config" is an instance of the Automapper Configuration class containing the mapping config to convert the Entity(ies) to DTO(s).
The alternative with a custom mapper that doesn't integrate with IQueryable is that you would have to materialize the entities first, then perform the mapping in memory:
var dtos = _dataService.GetEntities(idUser)
.ToList() // Materializes the entities
.Select(e => _dataService.MapToEntityDTO(e))
.ToList();
return Ok(dtos);
The disadvantages of this approach are that it requires more memory and time from the server to load the entities into memory first, then perform the mapping Select. This also requires that the GetEntities() method ensures that any/all related entities that might be mapped are eager loaded, otherwise they would either trigger lazy loads or be left #null. This can blow out time and memory usage where little of this related data might actually be needed.
With the ProjectTo projection, the queries will automatically fetch whatever related details are needed without the overhead of needing to eager load relations or tripping lazy loads.
Recently I've started learning Entity Framework Core and I'm curious if it's fine to use DbContext instance inside the entity class.
Sample code:
class User {
public int Id { get; set; }
public string Name { get; set; }
public ICollection<Order> Orders { get; set; }
public void LoadOrders() {
using (var context = new StoreContext()) {
Orders = context.Orders
.Where(x => x.UserId == Id)
.ToList();
}
}
}
User entity has a relation with Order class, both of them have appropriate tables in the DataBase created using migrations from Entity Framework.
The purpose of LoadOrders() method is simply to load related entities for current user when it's necessary.
Now I wanted to know if that's a valid approach?
Or maybe I should always load related entities at the same time when I'm loading the parent object? (E.g. .Include().ThenInclude())
Or maybe the code of LoadOrders() method should be located in some additional class like UserHelper that would be used along with the User entity.
You should avoid using an approach like this because the User will be loaded by one DbContext, while it's orders would be associated to another, disposed context. When you go to update a User, you would be facing errors or duplicate orders, or a messy business of reassociating the order (and other child entities) to contexts before saving. Down the road there will undoubtedly be confusion if orders are mapped to users and someone goes and writes code to .Include(x => x.Orders) If you completely detach related entities from EF and rely on load on demand, you lose out on a lot of the capability that EF gives you.
Issues like this typically stem from mixing up the scope/lifespan of entities vs. the scope of the context they are loaded from. For example loading entities in one method with a DbContext, returning them, then later deciding you want to access related entities but the DbContext was disposed. The simplest method I can recommend using is adopting POCO view models and ensuring that entities never exit the scope of their DbContext, only view models do. That way you can sculpt a view model structure to represent the data you need, then use entities and their references to populate those view models using .Select() without worrying about lazy loading or eager loading.
For instance:
using (var context = new StoreContext())
{
var userViewModel = context.Users.Where(x => x.UserId == userId)
.Select(x => new UserViewModel
{
UserId = x.UserId,
UserName = x.UserName,
Orders = x.Orders
.Where(o => o.IsActive)
.Select( o => new OrderViewModel
{
OrderId = o.OrderId,
OrderNumber = o.OrderNumber
Price = o.OrderItems.Sum(i => i.Price)
}).ToList()
}).SingleOrDefault();
return userViewModel;
}
Automapper can assist with mapping entities to view models. It's not a one-to-one tree structure map, but rather aligning the view model to represent the data the view needs, then filling that with the entity structure. You just need to be a bit careful to only pull data and supported aggregate methods from the entities because these will be passed to SQL, so no .Net or custom functions in the .Select. Let the view models accept raw values and provide alternate properties to perform formatting, or use .Select() to fetch anonymous types, get EF to materialize those into POCO instances with .ToList()/.Single()/etc. and then populate your view models from those using Linq2Object.
Working with entities on demand and view models / DTOs for the to-and-fro of data avoids a lot of hassle with entities. Done right, EF can pull this data extremely fast and it avoids performance pitfalls such as tripping lazy loads during serialization. It means that when you're done with the View Model you will need to re-load the entity to apply changes. It may seem to make more sense to simply use entities then have EF magically re-attach them and persist changes, but your view model will have all the info needed to quickly fetch that entity by ID if needed, and you will need to consider cases where the data may have changed between the time you first retrieved the entity, and the time you are prepared to alter it.
Assuming that I have an entity with virtual nagivation property like this:
public class School
{
public long Id { get; set; }
public virtual ICollection<Students> Students { get; set; }
}
As I understand, EF6 uses proxy to enable lazy-loading of Students, but do the following LINQ queries:
var myStudent = this.Students.Single(x => x.Id == id);
var studentsCount = this.Students.Count();
var bestStudents = this.Students
.OrderByDescending(x => x.GPA)
.Take(5)
.ToArray();
result into a SQL query (just like IQueryable<T> does)? Or is it just a lazy-loading collection and will load all students into memory after the first request, and then perform simple IEnumerable<T> LINQ behaviour?
When you query for an entity in Entity Framework, the objects that get returned are not (always) the type of object you think they are. Behind the scenes, it creates a brand new class that inherits from the your class. Because OOP allows a subclass to be stored in a variable typed as the superclass, you never really notice. This is the "proxy" that you mention. That's why the virtual function allows lazy loading. The subclass overrides your virtual method, and contains the code to lazy load the extra data before returning it.
That overridden property call will then check the context to see if the navigation properties are already loaded. If they are, it just returns them. If they are not, it will make additional SQL calls to load them, storing them in the DbContext for later.
In your updated question, my understanding is that running those lines of code would result in 3 separate queries being executed.
Public virtual properties are lazy loading in EF6.
You can disable lazy loading for a DbContext, or use the .Include() method on the IQueryable to include the property in the first query.
http://www.entityframeworktutorial.net/EntityFramework4.3/lazy-loading-with-dbcontext.aspx
https://msdn.microsoft.com/en-us/library/jj574232(v=vs.113).aspx
Once you "iterate" through the list (e.g. by calling the .Single(), .Count() or .ToArray() method), the query is executed and you have a in-memory list of your students. See https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ef/language-reference/query-execution for more details about query execution.
The first example would result in 3 queries, where the first returns the student with the given Id, the second returns the student count and the third returns the first 5 students ordered desc by their GPA property.
Regarding DDD => You can use some Layered architecture, with ApplicationServices and DomainServices, where DomainServices perform the domain logic and ApplicationServices load / transform the data.
The https://aspnetboilerplate.com/ template is for example a good starting point for domain driven design.
Assuming that the second code block is executed somewhere INSIDE the scope of an instance of 'School' (so 'this' is an instance of 'School' - in my example below school19) then the following scenarios are possible:
A) You have loaded your instance of 'School' like this (Lazy loading enabled):
var school19 = dbContext.Set<School>()
.FirstOrDefault(school => school.Id == 19)
Then your 3 lines of code for accessing the property 'Students' will trigger a single additional database hit when
var myStudent = this.Students.Single(x => x.Id == id);
is executed, but no more database hits will occur with the subsequent two statements.
B) In case you have loaded your instance of 'School' like this (Lazy loading enabled):
var school19 = dbContext.Set<School>()
.Include(school => school.Students)
.FirstOrDefault(school => school.Id == 19)
Then your 3 lines of code for accessing the property 'Students' will not trigger an additional database hit.
C) If lazy loading was disabled, then
A) would result in a Null Reference Exception
B) would behave the same
As a last remark, if 'this' was a reference to an instance of the DBContext class, which has a property
public Set<School> Schools { get; set; }
then it would trigger 3 different database calls. But the result is different as this would be executed in the context of ALL schools, while my assumptions above only apply to a single school.
I am using EF 5, and ASP.NET Web API to return JSON from my controllers. My example is for a pet service. For this question the entities are Client, Pet, and Veterinarian. The relationships are:
Client => one-to-many => Pet => one-to-one => Veterinarian
and, reversing the hierarchy:
Veterinarian => one-to-many => Pet => one-to-one => Client
In my PetsController, the GetPets action looks like this:
var pets= db.Pets
.Where(p => p.IsActive == true)
.Include(p => p.Client)
.Include(p => p.Veterinarian);
return Mapper.Map<IEnumerable<Pet>, IEnumerable<PetDTO>>(pets); // Using Automapper to map to a DTO
The resulting JSON has Pets, for each Pet a Client, for each Pet a Veterinarian AND for each Veterinarian, a collection of Pets.
I understand why the collection of pets is showing up in the JSON, but it's not relevant (at least not now) to my list of Pets from the controller action. I'm looking for an elegant way to remove the collection of pets. Right now I'm using what feels like a hack:
var pets= db.Pets
.Where(p => p.IsActive == true)
.Include(p => p.Client)
.Include(p => p.Veterinarian);
foreach (Pet pet in pets) {
pet.Veterinarian.Pets = null; // Remove the Pets collection before serializing
}
return Mapper.Map<IEnumerable<Pet>, IEnumerable<PetDTO>>(pets); // Using Automapper to map
I've tried some things with the serializer and with some IQuerable methods (many of which I am not familiar) but to no avail.
You can disable lazy loading to prune the graph. But that still doesn't solve the problem always. If you have a bidirectional relationship (like pet <=> Veterinarian), fetching one side of the relationship using Include will automatically build the other side too. So, after your query pet.Veterinarian.Pets will not be null even though you haven't asked it explicitly in the Include.
Using DTO's is the best way that I could think of. You have full control of the object graph.
Also, OData protocol has this mechanism already defined through $select & $expand and gives control to the client to specify the depth and breadth of the object graph that it needs. For example, for your scenario the OData url would look like, ~/Pets?$expand=Client,Veterinarian. Have you considered OData for your needs?
If you don't want to create DTO's every time, you can do it on the fly using anonymous objects. For example, for your scenario, you can do,
db.Pets.Select(p => new
{
Id = p.ID,
.... other pet properties you want,
Veterinarian = new
{
ID = p.Veterinarian.ID
... other veterinarian properties you want
},
Client =
{
... client properties you need.
}
});
This is just moving the mapping into the select clause though.
I have a many to many relationship in EF Code First between Contacts and Lists. ProxyCreation and LazyLoading are disabled to allow serialization of the entities.
I have a query that is meant to return contacts that are in a given list.
// GET api/Contacts
[Queryable]
public IQueryable<Contact> GetContacts(int bulkListId)
{
var bulkList = db.BulkLists.Include(c => c.Contacts).Where(c => c.ID == bulkListId).SingleOrDefault();
if (bulkList == null)
throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.NotFound));
return bulkList.Contacts.AsQueryable().OrderBy(c => c.ID).Include(c => c.AddressBookType).Include(c => c.BulkLists);
}
Although this works, it doesn't work as intended. It results in the correct set of contacts who are in a given list but these contacts only have that list populated in their Lists property of the relationship. So when this is serialized and gets back to the client it hides the other lists that the contacts are members of.
I can't see how the query is filtering it in this way and how I might change it to include the full set of lists. Any advice would be very much appreciated.
You're cheating! :)
By adding AsQueryable() to bulkList.Contacts you make it possible to continue with Include without making the compiler complain. BUT...
As per MSDN on DbExtensions.Include:
This extension method calls the Include(String) method of the IQueryable source object, if such a method exists. If the source IQueryable does not have a matching method, then this method does nothing.
(emphasis mine)
And EntityCollection does not have an Include method, so nothing happens.
You'll have to expand the include list in your first statement, probably like so:
db.BulkLists
.Include(c => c.Contacts.Select(c => c.AddressBookType))
.Include(c => c.Contacts.Select(c => c.BulkLists))