I just started to learn entity framework and i am facing the problem related to serialization of generated Models. I have tables with one to many relation which are Country and State as one country have many states. I am using the DB first approach and when i create the entities using Entity Framework, the class Country has one property of ICollection. I read and found that this is the navigation property. Let me show the generated class first which is below:
//This is the generated class.
public class Country
{
public Country()
{
States = new HashSet<States>();
}
public int Id { get; set; }
public string ContryCode { get; set; }
public string ContryName { get; set; }
public virtual ICollection<States> States{ get; set; }
}
I generated Models and then i step forwarded. Then i got the problem of serialization when i request via ajax to get the list for Country. I googled and found some terms like lazy loading, eager loading and n+1 problem. I read about it in detail. I found a solution which was turning off the lazy loading. Now the question is How i can serialize my Model with lazy loading?
I have created MetaData class and use some attribute like ignoreXml etc but nothing helped. By the way i am using Asp.Net MVC 5 and i want to serialize my Model with lazy loading. Can any one explain?
When you use lazy loading, execution is deferred until the value of the property is actually needed. You are probably encountering an error because the context has been disposed by the time the property is accessed. Consider the following scenario:
You have an object called Country with a lazy-loaded property called States.
You get this object from a context.
The context is disposed.
You call the States property.
The property goes looking for the context it came from.
An error is thrown because "the context has been disposed".
Code Sample:
using(var context = new SomeEntityContext())
{
var country = context.Countries.First();
}
//This will throw an error because the context was disposed of.
var states = country.States;
The serializer would also throw an error because it will go through the properties of the object, it will find States and it will try to get its value.
Even if the context is still alive, you could run into a loop with navigational properties during serialization. this is because both objects hold a reference to each other. Consider the following scenario:
You have an object called Country with a lazy-loaded property called States.
The serializer attempts to serializer the object of type Country.
It reads the States collection.
It attempts to serialize every object of type State.
It reads a property of type Country.
It attempts to serialize the object of type Country.
It reads the States collection.
It attempts to serialize every object of type State.
It reads a property of type Country.
It reads the States collection.
It attempts to serialize every object of type State.
It reads a property of type Country.
It reads the States collection.
It attempts to serialize every object of type State.
It reads a property of type Country.
.... (endless loop, well, at least until you run out of stack frames).
Alternatively, you can create a custom serializer that will avoid the pitfalls of navigational properties, but that is probably more work than it's worth. This approach is best suited for situations where the serialized version differs significantly from the object.
This is why you are better off using a Data Transfer Object (DTO). Map the data to this object and send that over the wire. There are components out there that can do the mapping for you, if you keep the structures as similar as possible.
Check out AutoMapper: http://automapper.org/
Related
I had seen some books(e.g programming entity framework code first Julia Lerman) define their domain classes (POCO) with no initialization of the navigation properties like:
public class User
{
public int Id { get; set; }
public string UserName { get; set; }
public virtual ICollection<Address> Address { get; set; }
public virtual License License { get; set; }
}
some other books or tools (e.g Entity Framework Power Tools) when generates POCOs initializes the navigation properties of the the class, like:
public class User
{
public User()
{
this.Addresses = new IList<Address>();
this.License = new License();
}
public int Id { get; set; }
public string UserName { get; set; }
public virtual ICollection<Address> Addresses { get; set; }
public virtual License License { get; set; }
}
Q1: Which one is better? why? Pros and Cons?
Edit:
public class License
{
public License()
{
this.User = new User();
}
public int Id { get; set; }
public string Key { get; set; }
public DateTime Expirtion { get; set; }
public virtual User User { get; set; }
}
Q2: In second approach there would be stack overflow if the `License` class has a reference to `User` class too. It means we should have one-way reference.(?) How we should decide which one of the navigation properties should be removed?
Collections: It doesn't matter.
There is a distinct difference between collections and references as navigation properties. A reference is an entity. A collections contains entities. This means that initializing a collection is meaningless in terms of business logic: it does not define an association between entities. Setting a reference does.
So it's purely a matter of preference whether or not, or how, you initialize embedded lists.
As for the "how", some people prefer lazy initialization:
private ICollection<Address> _addresses;
public virtual ICollection<Address> Addresses
{
get { return this._addresses ?? (this._addresses = new HashSet<Address>());
}
It prevents null reference exceptions, so it facilitates unit testing and manipulating the collection, but it also prevents unnecessary initialization. The latter may make a difference when a class has relatively many collections. The downside is that it takes relatively much plumbing, esp. when compared to auto properties without initialization. Also, the advent of the null-propagation operator in C# has made it less urgent to initialize collection properties.
...unless explicit loading is applied
The only thing is that initializing collections makes it hard to check whether or not a collection was loaded by Entity Framework. If a collection is initialized, a statement like...
var users = context.Users.ToList();
...will create User objects having empty, not-null Addresses collections (lazy loading aside). Checking whether the collection is loaded requires code like...
var user = users.First();
var isLoaded = context.Entry(user).Collection(c => c.Addresses).IsLoaded;
If the collection is not initialized a simple null check will do. So when selective explicit loading is an important part of your coding practice, i.e. ...
if (/*check collection isn't loaded*/)
context.Entry(user).Collection(c => c.Addresses).Load();
...it may be more convenient not to initialize collection properties.
Reference properties: Don't
Reference properties are entities, so assigning an empty object to them is meaningful.
Worse, if you initiate them in the constructor, EF won't overwrite them when materializing your object or by lazy loading. They will always have their initial values until you actively replace them. Worse still, you may even end up saving empty entities in the database!
And there's another effect: relationship fixup won't occcur. Relationship fixup is the process by which EF connects all entities in the context by their navigation properties. When a User and a Licence are loaded separately, still User.License will be populated and vice versa. Unless of course, if License was initialized in the constructor. This is also true for 1:n associations. If Address would initialize a User in its constructor, User.Addresses would not be populated!
Entity Framework core
Relationship fixup in Entity Framework core (2.1 at the time of writing) isn't affected by initialized reference navigation properties in constructors. That is, when users and addresses are pulled from the database separately, the navigation properties are populated.
However, lazy loading does not overwrite initialized reference navigation properties.
In EF-core 3, initializing a reference navigation property prevents Include from working properly.
So, in conclusion, also in EF-core, initializing reference navigation properties in constructors may cause trouble. Don't do it. It doesn't make sense anyway.
In all my projects I follow the rule - "Collections should not be null. They are either empty or have values."
First example is possible to have when creation of these entities is responsibility of third-part code (e.g. ORM) and you are working on a short-time project.
Second example is better, since
you are sure that entity has all properties set
you avoid silly NullReferenceException
you make consumers of your code happier
People, who practice Domain-Driven Design, expose collections as read-only and avoid setters on them. (see What is the best practice for readonly lists in NHibernate)
Q1: Which one is better? why? Pros and Cons?
It is better to expose not-null colections since you avoid additional checks in your code (e.g. Addresses). It is a good contract to have in your codebase. But it os OK for me to expose nullable reference to single entity (e.g. License)
Q2: In second approach there would be stack overflow if the License class has a reference to User class too. It means we should have one-way reference.(?) How we should decide which one of the navigation properties should be removed?
When I developed data mapper pattern by myself I tryed to avoid bidirectional references and had reference from child to parent very rarely.
When I use ORMs it is easy to have bidirectional references.
When it is needed to build test-entity for my unit-tests with bidirectional reference set I follow the following steps:
I build parent entity with emty children collection.
Then I add evey child with reference to parent entity into children collection.
Insted of having parameterless constructor in License type I would make user property required.
public class License
{
public License(User user)
{
this.User = user;
}
public int Id { get; set; }
public string Key { get; set; }
public DateTime Expirtion { get; set; }
public virtual User User { get; set; }
}
It's redundant to new the list, since your POCO is depending on Lazy Loading.
Lazy loading is the process whereby an entity or collection of entities is automatically loaded from the database the first time that a property referring to the entity/entities is accessed. When using POCO entity types, lazy loading is achieved by creating instances of derived proxy types and then overriding virtual properties to add the loading hook.
If you would remove the virtual modifier, then you would turn off lazy loading, and in that case your code no longer would work (because nothing would initialize the list).
Note that Lazy Loading is a feature supported by entity framework, if you create the class outside the context of a DbContext, then the depending code would obviously suffer from a NullReferenceException
HTH
The other answers fully answer the question, but I'd like to add something since this question is still relevant and comes up in google searches.
When you use the "code first model from database" wizard in Visual Studio all collections are initialized like so:
public partial class SomeEntity
{
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2214:DoNotCallOverridableMethodsInConstructors")]
public SomeEntity()
{
OtherEntities = new HashSet<OtherEntity>();
}
public int Id { get; set; }
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
public virtual ICollection<OtherEntity> OtherEntities { get; set; }
}
I tend to take wizard output as basically being an official recommendation from Microsoft, hence why I'm adding to this five-year-old question. Therefore, I'd initialize all collections as HashSets.
And personally, I think it'd be pretty slick to tweak the above to take advantage of C# 6.0's auto-property initializers:
public virtual ICollection<OtherEntity> OtherEntities { get; set; } = new HashSet<OtherEntity>();
Q1: Which one is better? why? Pros and Cons?
The second variant when virtual properties are set inside an entity constructor has a definite problem which is called "Virtual member call in a constructor".
As for the first variant with no initialization of navigation properties, there are 2 situations depending on who / what creates an object:
Entity framework creates an object
Code consumer creates an object
The first variant is perfectly valid when Entity Framework creates a object,
but can fail when a code consumer creates an object.
The solution to ensure a code consumer always creates a valid object is to use a static factory method:
Make default constructor protected. Entity Framework is fine to work with protected constructors.
Add a static factory method that creates an empty object, e.g. a User object, sets all properties, e.g. Addresses and License, after creation and returns a fully constructed User object
This way Entity Framework uses a protected default constructor to create a valid object from data obtained from some data source and code consumer uses a static factory method to create a valid object.
I use the answer from this Why is my Entity Framework Code First proxy collection null and why can't I set it?
Had problems with constructor initilization. Only reason I do this is to make test code easier. Making sure collection is never null saves me constantly initialising in tests etc
I'm using EF6 code first. There are two tables, Lesson and LessonSections. The LessonSections table has a foreign key to Lesson.Id
Here is the Lesson class with none important fields removed:
public partial class Lesson
{
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2214:DoNotCallOverridableMethodsInConstructors")]
public Lesson()
{
LessonSections = new HashSet<LessonSection>();
}
[StringLength(50)]
public string Id { get; set; }
[System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
public virtual ICollection<LessonSection> LessonSections { get; set; }
}
Here is how I'm initiating my data model:
var db = new Touch_Type_Trainer_DB.DataModel();
db.Configuration.ProxyCreationEnabled = false;
db.Configuration.LazyLoadingEnabled = false;
Just after my first call to the database to retrieve the first lesson in the database, the resulting object has no LessonSections
Then I make a second call to retrieve the sections into a separate object. (They must be in a separate objects since I want to serialize them to a JSON string and the serializer halts on the circular reference between Lesson and LessonSections if I use the standard EF LazyLoading.)
Now my original object has two sections loaded from the database even though I never accessed the LessonSections property and even though LazyLoadingEnabled is set to False!
Why do the LessonSections get loaded?
Edit:
I'm using Newtonsoft to serialize my object into a JSON string. Maybe there is a configuration setting in Newtonsoft that I should be setting so it doesn't get caught in the circular reference problem?
Also, I do want LazyLoading enabled for the majority of the code, just not for the serializing part.
is this a problem for you, or are you just curious as to why its happening?
The DBContext keeps track of all references for you. When you load the sections, it knows that the lessons have references to them, and wires it up for you.
you could stop this by disconnecting the objects , or by loading the sections from a different dbcontext
myDbContext.Entry(someLesson).State=Detached;
For the serialization issue, see this Q&A How Do You "Really" Serialize Circular Referencing Objects With Newtonsoft.Json?
or
http://johnnycode.com/2012/04/10/serializing-circular-references-with-json-net-and-entity-framework/
This is, in my opinion, a bad behaviour of EF.
EF works in this way (like probably you already noted): when you disable lazy loading relationships are not resolved (properties are not loaded even if you access the property).
In your case you can access to les.LessonSections and you see that is null or (in your case that you initialize it in Lesson constructor) empty.
If you, with the same context access to an object or to a collection not loaded with lazy load (build a query and materialize it) EF automatically try to solve relationships like during lazy load (also if the object does not have a proxy). This is your behavior, in a totally different query you access to LessonSections and EF solves relationships in Lesson.
At first look is a good behaviour but at the end you can have an inconsistent context (some objects with relationships resolved and some other not) and it can result in a buggy app. IMHO, the behaviour could be more consistent if EF (with LazyLoad disabled) doesn't resolve relationships at all and if you need to solve it you do it by hand. But is just my opinion.
About your question you can detach the object like suggested in another answer or use two different contexts (with the same or a different connection) in the same unit of work. I prefere (usually I use) this second approach disposing the context with Lazy Load disabled as soon as possible.
I have the following class generated by entity framework:
public partial class Branch
{
public short Id { get; set; }
public short CompanyId { get; set; }
public string Code { get; set; }
public string Title { get; set; }
public virtual Company Ts_Companies { get; set; }
}
I have the following method which takes all of the branches out of the database:
public Branch[] LoadBranches(int companyId, int page, int limit, string search, string sort, string sortOrder)
{
using (var dbContext = new TimeShedulerEntities())
{
var _branches = (from ct in dbContext.Branches
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
return _branches.ToArray();
}
}
In my model designer I see that the Lazy Loading is set to true, but when I iterate over the branches, the property Ts_Companies is null. Also I get the following exception:
An exception of type 'System.ObjectDisposedException' occurred in
EntityFramework.dll but was not handled in user code
Additional information: The ObjectContext instance has been disposed
and can no longer be used for operations that require a connection.
Am I forgetting something?
You created and disposed of the context during your function since it was inside the using statement. Each entity happens to know from which context it was created so that lazy loading is possible.
When you accessed the Ts_Companies property, the entity realized that it had not yet loaded that property since it is probably a navigation property and attempted to ask its ObjectContext (TimeShedulerEntities) to load that property. However, the context had been disposed and so that it what caused that exception.
You need to modify your query as follows to 'pre-load' the Ts_Companies:
var _branches = (from ct in dbContext.Branches.Include("Ts_Companies")
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
It will take possibly quite a bit longer to load depending on the size of the Ts_Companies object and how many you end up bringing back at once, but the entity will stop asking its object context to load the Ts_Companies since you would have already loaded them.
A side note: I have found that creation and disposal of object context on a per-method basis causes problems when the entities are passed outside the function. If you want to create and destroy the object context in every function, you probably want to have the function return something that is not an entity. In other words, have an object that can be constructed from an entity and has the properties you need, but don't have it reference the entity. In java these are often called Data Transfer Objects (DTOs). You lose the read-write ability of entity framework, but you don't have unexpected ObjectDisposedExceptions flying all over the place.
The problem comes when you ask an entity to be associated with another (for example, adding on entity to a ICollection property of another entity) when they come from different objectcontexts. This will cause headaches for you since you would have to manually attach the objects to the same context before performing that operation. Additionally, you lose the ability to save changes to those entities without manually attaching them to a different context.
My opinion on how I would do it:
I've found it easier to either have an object containing all of these database access functions control the lifetime of the context (i.e. have your containing object be IDisposable and during disposal, destroy the context) or simply not return entities and have the datastore be read-old, write-new essentially without any modification ability.
For example, I have my object (I will call it my data access object) with a bunch of methods for getting database objects. These methods return entities. The data access object also has a SaveChanges method which simply calls the context's SaveChanges method. The data access object contains the context in a protected property and keeps it around until the data access object itself is disposed. Nobody but the data access object is allowed to touch its context. At that point, the context is disposed by manually calling 'Dispose'. The data access object could then used inside a using statement if that is your use case.
In any case, it is probably best to avoid passing entities attached to a context outside the scope in which their context exists since entity framework keeps references to that context all over the place in the individual entities
But you didn't load your Ts_Companies, use Eager Loading instead:
var _branches = dbContext.Branches
.Where(b => b.Title.Contains(search) || b.Code.Contains(search))
.Include("Ts_Companies")
.OrderBy(c => c.Title)
.Skip((page - 1) * limit)
.Take(limit);
And I came across the same issue before System.ObjectDisposedException, in my MVC project and I didn't use using blocks,instead I define my context on class level.If I need to return and use an array (in my View) I use that context.If I need to just update some information then I have used using blocks.I hope this helps.
I ran into an annoying issue recently. I'm going to simplify my datamodel here, but the principle is just the same. I have a class "User". In that class I have a property that is a list of objects the user owns. I also have this class "object". Because every "object" has an owner, it has a property of type "User", which links to its owner. Now, what I'm trying to do is basically this
return Json(myUser,JsonRequestBehavior.AllowGet);
When I load the page, it takes like 30 seconds and then I get the error "RecursionLimit exceeded".
I guess this is because the objects are linking to each other. Now my question is, how can I tell "Json" that it shouldn't go deeper then 1 level of objects to avoid this?
myUser is probably a type generated by EntityFramework.
When you return Json, the framework is going to prepare each property in essence firing off SQL command to lazy-load all the data.
Instead, you should prepare a ViewModel class with concrete properties not attached to EntityFramework and prepare that object as deep as you want it to go.
It might happen when your object has some properties of itself. for example.
public object Employee()
{
string ID {get; set;}
string Name {get; set;}
int Age {get; set;}
Employee Boss{get; set;} //<-- here
}
var employee = new Employee();
return Json(employee,JsonRequestBehavior.AllowGet); //The Boss property will cause "RecursionLimit exceeded".
To avoid that. you can do something like that:
var employee = new Employee();
var prepareForJson = new {
ID = employee.ID,
Name = employee.Name,
Age = employee.Age,
Boss = employee.Boss.ID
};
return Json(prepareForJson , JsonRequestBehavior.AllowGet);
You can configure recursion depth through web.config
http://msdn.microsoft.com/en-us/library/bb763183.aspx
but you probably just want to sort out your model not to have recursion in the first place. Think how much data is needed in your current situation and return just that.
I think Edison Chuang is the answer for the vast majority of cases (mapping data models to some service models that do not have those navigation properties that will most certainly cause loops during the serialization).
However, since most of the service models will share many of the properties of the data models (EF models), AutoMapper can be used, greatly simplifying code especially for objects with many properties. E.g.:
// set up map
cfg.CreateMap<Employee, EmployeeSm>();
// use the map
var sm = Mapper.Map<EmployeeSm>(employee);
// this can be also set up during mapping configuration phase
sm.Boss = employee.Boss.ID;
For those rare cases when serialization depth goes beyond the default 100, you can either increase this limit or use Json.NET to serialize your objects.
I have a Reason object:
public class Reason
{
public virtual long Id { get; set; }
public virtual string Name { get; set; }
public virtual Company Company {get;set;}
}
I am using entity framework 4 and Company is navigation property to Company.
I also use webservices in order to return data to the client.
I have web method that returns Reasons:
[WebMethod]
public Reason[] GetCallReasons()
{
IReasonRepository rep =
ObjectFactory.GetInstance<IReasonRepository>();
return rep.GetReasonsList().ToArray();
}
Because of the ef4 I get the following exception for executing the web method:
A circular reference was detected while serializing an object of type 'System.Data.Entity.DynamicProxies.Reason_24A0E4BBE02EE6BC2CF30BB56CFCB670C7D9D96D03D40AF4D174B89C9D3C5537'
The problem accurs because ef4 adds property that can't be serialized:
In order to solve this and eliminate the error, I can disable the navigation property by not making it virtual or by remove the navigation property. But I neet it and want to use the lazy loading feature.
I also can write spesific serializer for Reason but I have many many classes the I used in my web-services and write a serializer for all of them is a lot of work.
How can I solve this exception?..
There are multiple solutions for your problem and they really depend on the type of service you are using and on the type of serialization:
The clean approach is using DTO (data transfer objects) as #Mikael already suggested. DTO is special object which transfers exactly what you need and nothing more. You can simply create DTOs to not contain circular references and use AutoMapper to map between entities and DTOs and vice versa. +1 for #Mikael because he was the first to mentioned this.
All other approaches are based on tweeking serialization as #Haz suggested:
WCF and DataContractSerializer: explicitly mark your entities with DataContract[IsReference=true] and all properties with [DataMember] attributes. This will allow you to use circular references. If you are using T4 template to generate entities you must modify it to add these attributes for you.
WCF and DataContractSerializer: implicit serialization. Mark one of related navigation properties with [IgnoreDataMember] attribute so that property is not serialized.
XmlSerializer: mark one fo related navigation properties with [XmlIgnore] attribute
Other serializations: mark one of related navigation properties with [NonSerialized] (+1 for Haz he was the first to mention this) for common serialization or [ScriptIgnore] for some JSON related serialization.
I usually write specific classes for the webservice. While this is some extra work it has the advantage that the webservice gets more robust as small changes in your entities won't go unoticed and silently fail on the side of the consumer/javascript. For example if I change the name of a property.
There are a few things you can do to reduce the work and one is to use AutoMapper which can automatically map between objects.
You haven't supplied the definition for your company class.... But I'm guessing you have a collection of Reason as a property.
Lazy loading in an SOA Enviroment doesn't really work. You can't have unlimited lazy navigation over a serialized class, once you leave the webmethod you have no way to call back into the original datacontext from the webmethod consumer to lookup the properites... so the serializer will try and visit all properties, including lazy properties at the time of serialization.
You need to disable serialization on one part of the circular reference, either on the Reason collection in Company class, or the Company in Reason class.
You can use the "NotSerialized" attribute to disable serialization of a particular field.