EF tables with Many to many relationship, Will this consume memory? - c#

I have 2 tables that saved family members, when I use include to retrieve the family members, the generated T-SQL is what I'm expected, but when I see the result from VS, like image below, it's look like never ending.
My questions:
It's this normal?
Should I avoid include when the relationship becomes complex?
If it is normal, will this very memory consumption?
POCO
public class Cust_ProfileTbl
{
[Key]
public long bintAccountNo { get; set; }
public string nvarCardName { get; set; }
public string varEmail { get; set; }
public virtual ICollection<Cust_ProfileFamilyTbl> profileFamilyParents { get; set; }
public virtual ICollection<Cust_ProfileFamilyTbl> profileFamilyChildren { get; set; }
}
public class Cust_ProfileFamilyTbl
{
[Key]
public int intProfileFamily { get; set; }
public long bintAccountNo { get; set; }
public long bintAccountNoMember { get; set; }
public virtual Cust_ProfileTbl custProfileParent { get; set; }
public virtual Cust_ProfileTbl custProfileChild { get; set; }
}
LINQ
var rs = from family in context.member.Include("profileFamilyParents.custProfileChild")
select family;
rs = rs.Where(x => x.bintAccountNo.Equals(1));
var result = rs.ToList();
In onModelCreating
modelBuilder.Entity<Cust_ProfileFamilyTbl>()
.HasRequired(m => m.custProfileParent)
.WithMany(t => t.profileFamilyParents)
.HasForeignKey(m => m.bintAccountNo)
.WillCascadeOnDelete(false);
modelBuilder.Entity<Cust_ProfileFamilyTbl>()
.HasRequired(m => m.custProfileChild)
.WithMany(t => t.profileFamilyChildren)
.HasForeignKey(m => m.bintAccountNoMember)
.WillCascadeOnDelete(false);

When people use an ORM like EF in their application, many times the application design gets driven by this ORM and the entities defined in its model. When the app is a simple "CRUD" application, that's not a problem, but an advantage, because you spare a lot of time.
However when things start to get more complicated, an "ORM guided design" becomes a problem. This looks to be the case.
There are at least two problems, recovered from the comments:
the data retrieved from the DB is more than needed
in this case, because of some particular relationships between entities, there is a circular reference, which creates an endless loop and a stack overflow when trying to show the model in the view
When this kind of situation shows up, the most advisable is to break the tight tie between the ORM and the rest of the app, which can be dine by defining a new class, and projecting the data into it. Let's give a generic ProfileDto name.
public class ProfileDto { ... }
DTO is a generic name for this kind of classes: Data Transfer Objects - but, when they have specific purposes, they can get other names like view models, when they're going to be used as the model sent to an MVC view
And then, what you need to do is to project the result of the query into the DTO:
var model = theQuery.Select(i => new ProfileDto { a = i.a, b = i.b...}).ToList();
With a good design of the Dto you'll only recover the needed data from the DB, and you'll avoid the loop problem (by not including the navigation property that creates the loop).
NOTE: many times people uses mappers, like AutoMapper or ValueInjecter to make the mapping, or part of the mapping, automatic
Code standardization is a very good idea until it becomes a source of problems. The main purpose of writing code is implementing the business logic. If code standardization, technology, or whatever, makes it harder to implement business logic, instead of contributing to the solution, they become a problem, so you need to avoid them.

Mapping you created is Normal but use of Include depends upon its usage
Use of Include depends on situation of use for example if you want to cache it in memory then you may use include, Where as if you are using only showing properties of Cust_ProfileTbl
class in some grid and on click you want show details of Cust_ProfileFamilyTbl then you might don't want to use include. But be careful if you are using Automapper or something because when It will try to map related properties it will query database.
It will consume memeory when you execute ToList() as doing so you are Loading query result into List collection. Where as If you again want to query the result then you can use ToQueryable() or just want to iterate the you can don't load them to List.

Related

One to many relationship doesn`t retrieve data in entity framework

I`m in process of learning C# & .NET and EF (with aspnetboilerplate) and I came up with idea to create some dummy project so I can practice. But last 4 hour Im stuck with this error and hope someone here can help me.
What I create( well at least I think I create it correctly ) is 2 class called "Ingredient" and "Master"
I want to use it for categorize Ingredient with "Master" class.
For example ingredient like
Chicken breast
chicken drumstick
Both of them belong to Meat ( witch is input in "Master" database ) and here is my code
Ingredient.cs
public class Ingrident : Entity
{
public string Name { get; set; }
public Master Master { get; set; }
public int MasterId { get; set; }
}
Master.cs
public class Master : Entity
{
public string Name { get; set; }
public List<Ingrident> Ingridents { get; set; } = new();
}
IngridientAppService.cs
public List<IngridientDto> GetIngWithParent()
{
var result = _ingRepository.GetAllIncluding(x => x.Master);
//Also I try this but doesn`t work
// var result = _ingRepository.GetAll().Where(x => x.MasterId == x.Master.Id);
return ObjectMapper.Map<List<IngridientDto>>(result);
}
IngridientDto.cs
[AutoMap(typeof(IndexIngrident.Entities.Ingrident))]
public class IngridientDto : EntityDto
{
public string Name { get; set; }
public List<MasterDto> Master { get; set; }
public int MasterId { get; set; }
}
MasterDto.cs
[AutoMap(typeof(IndexIngrident.Entities.Master))]
public class MasterDto : EntityDto
{
public string Name { get; set; }
}
When I created ( for last practice ) M -> M relationship this approach with .getAllIncluding work but now when I have One -> Many it won`t work.
Hope someone will be able to help me or at least give me some good hint.
Have a nice day !
Straight up the examples you are probably referring to (regarding the repository etc.) are overcomplicated and for most cases, not what you'd want to implement.
The first issue I see is that while your entities are set up for a 1-to-many relationship from Master to Ingredients, your DTOs are set up from Ingredient to Masters which definitely won't map properly.
Start with the simplest thing. Get rid of the Repository and get rid of the DTOs. I'm not sure what the base class "Entity" does, but I'm guessing it exposes a common key property called "Id". For starters I'd probably ditch that as well. When it comes to primary keys there are typically two naming approaches, every table uses a PK called "Id", or each table uses a PK with the TableName suffixed with "Id". I.e. "Id" vs. "IngredientId". Personally I find the second option makes it very clear when pairing FKs and PKs given they'd have the same name.
When it comes to representing relationships through navigation properties one important detail is ensuring navigation properties are linked to their respective FK properties if present, or better, use shadow properties for the FKs.
For example with your Ingredient table, getting rid of the Entity base class:
[Table("Ingredients")]
public class Ingredient : Entity
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int IngredientId { get; set; }
public string Name { get; set; }
public int MasterId { get; set; }
[ForeignKey("MasterId")]
public virtual Master Master { get; set; }
}
This example uses EF attributes to aid in telling EF how to resolve the entity properties to respective tables and columns, as well as the relationship between Ingredient and Master. EF can work much of this out by convention, but it's good to understand and apply it explicitly because eventually you will come across situations where convention doesn't work as you expect.
Identifying the (Primary)Key and indicating it is an Identity column also tells EF to expect that the database will populate the PK automatically. (Highly recommended)
On the Master side we do something similar:
[Table("Masters")]
public class Master : Entity
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int MasterId { get; set; }
public string Name { get; set; }
[InverseProperty("Master")]
public virtual ICollection<Ingredient> Ingredients { get; set; } = new List<Ingredient>();
}
Again we denote the Primary Key, and for our Ingredients collection, we tell EF what property on the other side (Ingredient) it should use to associate to this Master's list of Ingredients using the InverseProperty attribute.
Attributes are just one option to set up the relationships etc. The other options are to use configuration classes that implement IEntityConfiguration<TEntity> (EF Core), or to configure them as part of the OnModelCreating event in the DbContext. That last option I would only recommend for very small projects as it can start to become a bit of a God method quickly. You can split it up into calls to various private methods, but you may as well just use IEntityConfiguration classes then.
Now when you go to fetch Ingredients with it's Master, or a Master with its Ingredients:
using (var context = new AppDbContext())
{
var ingredients = context.Ingredients
.Include(x => x.Master)
.Where(x => x.Master.Name.Contains("chicken"))
.ToList();
// or
var masters = context.Master
.Include(x => x.Ingredients)
.Where(x => x.Name.Contains("chicken"))
.ToList();
// ...
}
Repository patterns are a more advanced concept that have a few good reasons to implement, but for the most part they are not necessary and an anti-pattern within EF implementations. I consider Generic repositories to always be an anti-pattern for EF implementations. I.e. Repository<Ingredient> The main reason not to use repositories, especially Generic repositories with EF is that you are automatically increasing the complexity of your implementation and/or crippling the capabilities that EF can bring to your solution. As you see from working with your example, simply getting across an eager load through to the repository means writing in complex Expression<Func<TEntity>> parameters, and that just covers eager loading. Supporting projection, pagination, sorting, etc. adds even more boiler-plate complexity or limits your solution and performance without these capabilities that EF can provide out of the box.
Some good reasons to consider studying up on repository implementations /w EF:
Facilitate unit testing. (Repositories are easier to mock than DbContexts/DbSets)
Centralizing low-level data rules such as tenancy, soft deletes, and authorization.
Some bad (albeit very common) reasons to consider repositories:
Abstracting code from references or knowledge of the dependency on EF.
Abstracting the code so that EF could be substituted out.
Projecting to DTOs or ViewModels is an important aspect to building efficient and secure solutions with EF. It's not clear what "ObjectMapper" is, whether it is an Automapper Mapper instance or something else. I would highly recommend starting to grasp projection by using Linq's Select syntax to fill in a desired DTO from the models. The first key difference when using Projection properly is that when you project an object graph, you do not need to worry about eager loading related entities. Any related entity / property referenced in your projection (Select) will automatically be loaded as necessary. Later, if you want to leverage a tool like Automapper to help remove the clutter of Select statements, you will want to configure your mapping configuration then use Automapper's ProjectTo method rather than Map. ProjectTo works with EF's IQueryable implementation to resolve your mapping down to the SQL just like Select does, where Map would need to return everything eager loaded in order to populate related data. ProjectTo and Select can result in more efficient queries that can better take advantage of indexing than Eager Loading entire object graphs. (Less data over the wire between database and server/app) Map is still very useful such as scenarios where you want to copy values back from a DTO into a loaded entity.
Do it like this
public class Ingrident:Entity
{
public string Name { get; set; }
[ForeignKey(nameof(MasterId))]
public Master Master { get; set; }
public int MasterId { get; set; }
}

Use AutoMapper to map two VM to one Entity object

I'm using AutoMapper to map a lot of Entity models to View Model that I use in my controllers and views (.Net MVC)
There is a lot of relations in the DB and so our VM have a lot of childs (who have childs, and so and so)
public class InvoiceVMFull : VMBase
{
public int Id { get; set; }
public InvoiceType InvoiceType { get; set; }
public string Reference { get; set; }
//.... shortened code for readability
// list all entity fields
public List<string> InvoiceMainAddress { get; set; }
public List<string> InvoiceDlvAddress { get; set; }
}
It works just fine, but is very slow and always load from the DB all relations whereas I usually need only a few datas...
So I created some light VM that I want to use for the majority of our pages.
public class InvoiceVMLite : VMBase
{
public int Id { get; set; }
public string Reference { get; set; }
//.... shortened code for readability
// list only some of the entity fields (most used)
public StoredFileVM InvoiceFile { get; set; }
}
The problem is I can't find how :
to map one Entity object to the two VMs and how to choose the right one (to load from DB) using the context (the page or event called)
to map two VMs to one entity and save (on the DB) only the fields that are present in the VM used and don't erase the absent ones
I tried to create the mapping both VM :
Mapper.CreateMap<Invoice, InvoiceVMLite>();
Mapper.CreateMap<Invoice, InvoiceVMFull>();
But when I try to call the mapping for Lite, it doesn't exist (have been overridden by Full) :
Mapper.Map(invoice, InvoiceEntity, InvoiceVMLite)
Correct Use of Map function
It looks like you are calling map incorrectly. Try these instead
var vmLite = Mapper.Map<Invoice, InvoiceVMLite>(invoice);
var vmFull = Mapper.Map<Invoice, InvoiceVMFull>(invoice);
var vmLite = Mapper.Map(invoice); // would work if it were not ambiguous what the destination was based on the input.
Entity to two view models
You would usually create two mappings, one for each view model from the one entity. I'd suggest the cleanest is to have two separate views (separate Actions in a controller) for each view model. This may involve a quick redirect after you've decided on context which one to use.
View models to entity
Automapper is not meant for mapping from view models to Entities for many reasons, including the challenge you'd face. Instead you would pass specific parameters. The author of Automapper, Jimmy Bogard, wrote a good article on why this is the case.
I couldnt manage to do that with AutoMapper, and so I created my own convert methods (Entity <=> VM) with a lot of reflexivity, and with specific cases handled in each of the VM classes.
Now I can easily get a full or lite VM from an Entity, and also specify the depth in relation I want to go. So it's A LOT faster and more adaptable than AutoMapper
And I can save a VM to an entity (only saving modified fields if I want) that I create or that i got from base. So it's A LOT faster and adaptable than AutoMapper
In conclusion : Don't use autoMapper, it seem easy but create so many performance issues that it isn't worth it

Telling EF 6 to Ignore a Private Property

I'm using Entity Framework 6.0.2 to map some simple hand-coded models to an existing database structure. The primary model at the moment is:
public class Occurrence
{
public int ID { get; set; }
public Guid LegacyID { get; set; }
public string Note { get; set; }
public virtual ICollection<OccurrenceHistory> History { get; set; }
}
(The OccurrenceHistory model isn't really relevant to this, but that part is working fine whereby EF loads up the child records for this model.)
The mapping is simple, and I try to be as explicit as I can be (since as the application grows there will be some less-intuitive mapping):
public class OccurrenceMap : EntityTypeConfiguration<Occurrence>
{
public OccurrenceMap()
{
ToTable("Occurrence");
HasKey(o => o.ID);
Property(o => o.ID).IsRequired().HasColumnName("ID");
Property(o => o.LegacyID).IsRequired().HasColumnName("LegacyID");
Property(o => o.Note).IsUnicode().IsOptional().HasColumnName("Note");
}
}
But if I add a private property to the model, EF tries to map it to the database. Something like this:
private OccurrenceHistory CurrentHistory { get; set; }
(Internal to the model I would have some logic for maintaining that field, for other private operations.) When EF generates a SELECT statement it ends up looking for a column called CurrentHistory_ID which of course doesn't exist.
I can make the property public and set the mapping to ignore it:
Ignore(o => o.CurrentHistory);
But I don't want the property to be public. The model is going to internally track some information which the application code shouldn't see.
Is there a way to tell EF to just ignore any and all private members? Even if it's on a per-map basis? I'd particularly like to do this without having to add EF data annotations to the models themselves, since that would not only be a bit of a leaky abstraction (persistence-ignorant models would then have persistence information on them) but it would also mean that the domain core assembly which holds the models would carry a reference to EntityFramework.dll everywhere it goes, which isn't ideal.
A colleague pointed me to a blog post that led to a very practical approach.
So what I have is a private property:
private OccurrenceHistory CurrentHistory { get; set; }
The core of the problem is that I can't use that in my mapping:
Ignore(o => o.CurrentHistory);
Because, clearly, the property is private and can't be accessed in this context. What the blog post suggests is exposing a public static expression which references the private property:
private OccurrenceHistory CurrentHistory { get; set; }
public static readonly Expression<Func<Occurrence, OccurrenceHistory>> CurrentHistoryExpression = o => o.CurrentHistory;
I can then reference that in the mapping:
Ignore(Occurrence.CurrentHistoryExpression);
As with anything, it's a mix of pros and cons. But in this case I think the pros far outweigh the cons.
Pros:
The domain core assembly doesn't need to carry a reference to EntityFramework.dll.
The persistence mapping is entirely encapsulated within the DAL assembly.
Cons:
Models need to expose a little information about their inner workings.
The con breaks encapsulation, but only slightly. Consuming code still can't access that property or its value on instances, it can only see that the property exists statically. Which, really, isn't a big deal, since developers can see it anyway. I feel that the spirit of encapsulation is still preserved on any given instance of the model.

Filtering navigation properties in EF Code First

I'm using Code First in EF. Let's say I have two entities:
public class Farm
{
....
public virtual ICollection<Fruit> Fruits {get; set;}
}
public class Fruit
{
...
}
My DbContext is something like this:
public class MyDbContext : DbSet
{
....
private DbSet<Farm> FarmSet{get; set;}
public IQueryable<Farm> Farms
{
get
{
return (from farm in FarmSet where farm.owner == myowner select farm);
}
}
}
I do this so that each user can only see his farms, and I don't have to call the Where on each query to the db.
Now, I want to filter all the fruits from one farm, I tried this (in Farm class):
from fruit in Fruits where fruit .... select fruit
but the query generated doesn't include the where clause, which is very important because I have dozens of thousands of rows and is not efficient to load them all and filter them when they're Objects.
I read that lazy loaded properties get filled the first time they're accessed but they read ALL the data, no filters can be applied UNLESS you do something like this:
from fruits in db.Fruits where fruit .... select fruit
But I can't do that, because Farm has no knowledge of DbContext (I don't think it should(?)) but also to me it just loses the whole purpose of using navigation properties if I have to work with all the data and not just the one that belongs to my Farm.
So,
am I doing anything wrong / making wrong assumptions?
Is there any way I can apply a filter to a navigation property that gets generated to the real query? (I'm working with a lot of data)
Thank you for reading!
Unfortunately, I think any approach you might take would have to involve fiddling with the context, not just the entity. As you've seen, you can't filter a navigation property directly, since it's an ICollection<T> and not an IQueryable<T>, so it gets loaded all at once before you have a chance to apply any filters.
One thing you could possibly do is to create an unmapped property in your Farm entity to hold the filtered fruit list:
public class Farm
{
....
public virtual ICollection<Fruit> Fruits { get; set; }
[NotMapped]
public IList<Fruit> FilteredFruits { get; set; }
}
And then, in your context/repository, add a method to load a Farm entity and populate FilteredFruits with the data you want:
public class MyDbContext : DbContext
{
....
public Farm LoadFarmById(int id)
{
Farm farm = this.Farms.Where(f => f.Id == id).Single(); // or whatever
farm.FilteredFruits = this.Entry(farm)
.Collection(f => f.Fruits)
.Query()
.Where(....)
.ToList();
return farm;
}
}
...
var myFarm = myContext.LoadFarmById(1234);
That should populate myFarm.FilteredFruits with only the filtered collection, so you could use it the way you want within your entity. However, I haven't ever tried this approach myself, so there may be pitfalls I'm not thinking of. One major downside is that it would only work with Farms you load using that method, and not with any general LINQ queries you perform on the MyDbContext.Farms dataset.
All that said, I think the fact that you're trying to do this might be a sign that you're putting too much business logic into your entity class, when really it might belong better in a different layer. A lot of the time, it's better to treat entities basically as just receptacles for the contents of a database record, and leave all the filtering/processing to the repository or wherever your business/display logic lives. I'm not sure what kind of application you're working on, so I can't really offer any specific advice, but it's something to think about.
A very common approach if you decide to move things out the Farm entity is to use projection:
var results = (from farm in myContext.Farms
where ....
select new {
Farm = farm,
FilteredFruits = myContext.Fruits.Where(f => f.FarmId == farm.Id && ...).ToList()
}).ToList();
...and then use the generated anonymous objects for whatever you want to do, rather than trying to add extra data to the Farm entities themselves.
Just figured I'd add another solution to this having spent some time trying to append DDD principles to code first models. After searching around for some time I found a solution like the one below which works for me.
public class FruitFarmContext : DbContext
{
public DbSet<Farm> Farms { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Farm>().HasMany(Farm.FruitsExpression).WithMany();
}
}
public class Farm
{
public int Id { get; set; }
protected virtual ICollection<Fruit> Fruits { get; set; }
public static Expression<Func<Farm, ICollection<Fruit>>> FruitsExpression = x => x.Fruits;
public IEnumerable<Fruit> FilteredFruits
{
get
{
//Apply any filter you want here on the fruits collection
return Fruits.Where(x => true);
}
}
}
public class Fruit
{
public int Id { get; set; }
}
The idea is that the farms fruit collection is not directly accessible but is instead exposed through a property that pre-filters it.
The compromise here is the static expression that is required to be able to address the fruit collection when setting up mapping.
I've started to use this approach on a number of projects where I want to control the access to an objects child collections.
Lazy loading doesn't support filtering; use filtered explicit loading instead:
Farm farm = dbContext.Farms.Where(farm => farm.Owner == someOwner).Single();
dbContext.Entry(farm).Collection(farm => farm.Fruits).Query()
.Where(fruit => fruit.IsRipe).Load();
The explicit loading approach requires two round trips to the database, one for the master and one for the detail. If it is important to stick to a single query, use a projection instead:
Farm farm = (
from farm in dbContext.Farms
where farm.Owner == someOwner
select new {
Farm = farm,
Fruit = dbContext.Fruit.Where(fruit => fruit.IsRipe) // Causes Farm.Fruit to be eager loaded
}).Single().Farm;
EF always binds navigation properties to their loaded entities. This means that farm.Fruit will contain the same filtered collection as the Fruit property in the anonymous type. (Just make sure you haven't loaded into the context any Fruit entities that should be filtered out, as described in Use Projections and a Repository to Fake a Filtered Eager Load.)

How to use Fluent NHibernate Validator when using auto mapping?

I've just modelled a small database using Fluent nHibernate and the auto mapping feature. Now I'm wondering how I work with validation. In the past I've decorated my classes with attributes but the purpose of this by-convention automapping is to keep things clean.
I do have a couple override files which look like this:
public class EventMappingOverride : IAutoMappingOverride<Event>
{
public void Override(AutoMapping<Event> mapping)
{
mapping.Map(x => x.EventType, "TypeID").CustomType(typeof(EventType));
mapping.Map(x => x.EventStatus, "StatusID").CustomType(typeof(EventStatus));
mapping.HasMany(x => x.EventDates).KeyColumn("EventID");
}
}
Is this where I would put my validation rules? If so, what does that look like and is there really even a point to using the auto mapping (if my override files are going to be elaborate anyway)?
Thanks.
To clarify further:
My entities look like this as of now:
namespace Business.Data
{
public class Event
{
public virtual int Id { get; set; }
public virtual string Title { get; set; }
public virtual EventStatus EventStatus { get; set; }
public virtual EventType EventType { get; set; }
public virtual IList<EventDate> EventDates { get; set; }
}
}
I would like to keep them looking that like. Just plain objects so in the future we can potentially switch out or upgrade the ORM and still have these nice clean objects.
However, when it comes to using nHibernate Validator (part of NHContrib) I'm not sure how to incorporate it without littering the properties with attributes. I guess this is more of a question architecture. I could use a different validation framework as well but I want it to be tied in with nHibernate so that it won't insert/update invalid records. Any opinions appreciated!
My opinion is :
Validation is part of the business at it depend from it and then the database scale to this need. So if you need a email string column in your db you should not rely on a db framework to do that especially as you said that may be later you will switch ORM then you will loose your work.
Keep validation in the business/high layer, and leave the db do simple query/insertion, remember NHibernate is already a bit complicate to hand on so keep it simple.
To answer your question, if you don't want to littering your entities use the xml validation as describe here.
http://nhforge.org/wikis/validator/nhibernate-validator-1-0-0-documentation.aspx

Categories