We Keep Coding

Json.Net Class hierarchy with ObservableCollection and INotifyPropertyChange gets serialized but not deserialized

I find myself a bit lost on this one. I honestly can't see the error if it's just a class structure doesn't match JSON error. But I doubt it since it's the very same class structure I'm using to create the JSON.
If anyone can point me in the right direction, I'd be most greateful.
I've created a dotnetfiddle to avoid clutching the question with huge pieces of code. Here's the link: Fiddle
I generate that JSON with a console application that gets info on the DB schema. I use a common project with all the entities defined in it to load the data in memory and then generate the JSON from that structure. Then I use the same project with the same entities on another application to compare another DB schema to the JSON log. That application is unable to deserialize the JSON. Tried to provide a minimal example with a single class and as you can see on the fiddle...that doesn't deserialize either.
It is my understanding that ObservableCollections should in fact serialize and deserialize without issues, and that INotifyPropertyChange should not cause issues (as long as you're not trying to fire an event with a null reference). So...anyone has any idea what's going on here?.
EDIT: Forgot to mention. Notice how only the base type string gets deserialized...so it IS running some deserialization, just not of classes like ObservableCollection or user classes. Maybe that helps somehow to pinpoint the issue.
EDIT2: Added a trace writer and the JSON.Net trace is detecting the right type for the objects, so I'm guessing the issue is on converting types or initializing some of the types

The problem is in how your property getters work combined with the default ObjectCreationHandling setting in Json.Net. Allow me to explain:
By default, if a reference property has an existing (non-null) value during deserialization, Json.Net tries to reuse the existing instance and populate it instead of creating a new instance. To find out whether the property has a value, Json.Net calls the getter. In your case, the getter returns a new instance when the backing field is null, but, critically, it does not set the backing field to the new instance:
get { return _header ?? new StoredProcedureDataHeader(); }
Json.Net then populates the new instance. Because the backing field was never set to the new instance, that instance ultimately gets thrown away. Json.Net never calls your setter because it assumes that your object already has a reference to the new instance, since it got that instance from the getter. Then, when you next call that getter after deserialization, you get a new, empty instance back instead of what you were expecting.
There are two ways to fix the problem:
Change your getters to set the backing field whenever a new instance is created, for example:
get
{
if (_header == null)
{
_header = new StoredProcedureDataHeader();
}
return _header;
}
OR
Change the ObjectCreationHandling setting to Replace to force Json.Net to always create new instances when deserializing. Json.Net will then call the setter and not the getter, which I think is what you want.
var settings = new JsonSerializerSettings
{
ObjectCreationHandling = ObjectCreationHandling.Replace
};
var data = JsonConvert.DeserializeObject<StoredProcedureData>(json, settings);
In your case, I would actually recommend that you apply both fixes. If you don't fix your getters (option 1), you could run into a similar issue elsewhere in your code. For example, you might have something like this:
var data = new StoredProcedureData();
data.Header.SPName = "foo";
if (data.Header.SPName == "foo")
{
Console.WriteLine("foo");
}
else
{
Console.WriteLine("oops");
}
Guess which value will be printed?
And option 2 will protect against possibly unexpected results if you happen to have initialized a collection somewhere to have a default set of values. For example, if you had something like this:
public StoredProcedureData()
{
_funcRef = new ObservableCollection<string>();
_funcRef.Add("Initialize");
}
then when you deserialize, you will get the default values plus the values from the JSON, which is probably not what you want. Setting ObjectCreationHandling to Replace will ensure that you will end up with just the values which were deserialized from the JSON.

Is good practice to throw exceptions in a property with DataMember attribute applied?

DataContract and DataMember attributes may be also used in order to serialize an object to file and for deserialization, by using a DataContractSerializer. Suppose we have a class with the following private field and public property.
public class MyClass
{
private int positiveValue;
public int PositiveValue
{
get { return positiveValue; }
set
{
if (value < 1)
throw new ArgumentOutOfBoundException(...);
positiveValue = value;
}
}
}
Now suppose we have an XML file containing the state of a previously serialized object, and assume that the user has modified this file, specifying an incorrect value (that is a non positive value) for the PositiveValue property. During deserialization an exception would be thrown, since the value in the file is invalid.
Suppose we want to deserialize a list of MyClass objects from a file: if some object is not valid, an exception is thrown. Is it possible to make sure that the DataContractSerializer ignores invalid objects? Moreover, taking into account the problem just explained, is good practice to throw exceptions in a property with DataMember attribute applied?

If you want to validate data when setting properties, but want to bypass these validations during deserialization, you can mark the property's backing field with the DataMemberAttribute instead of the property. This will cause the DataContractSerializer to set the value directly in the field (not the property) and not throw any validation exceptions.
[DataMember]
private int positiveValue;
public int PositiveValue
{
get { return positiveValue; }
set
{
if (value < 1)
throw new ArgumentOutOfBoundException(...);
positiveValue = value;
}
}
Note, that the backing field can be private (otherwise, the property would not make much sense). This works fine with DataContractSerializer.
Moreover, taking into account the problem just explained, is good practice to throw exceptions in a property with DataMember attribute applied?
That depends on whether you want to allow to have objects with invalid data in your application. The advantage of not allowing to set invalid data is that you don't have to check the object each time you use it, but in that case you must check any user input and be prepared for exceptions when loading data. However, I don't think that throwing exceptions in data member properties is wrong. It's just a matter of a design decision.

In terms of a list, no you can't make DCS ignore invalid objects. If an exception occurs, the entire deserialize is aborted. If it doesn't throw, it it going to be added to your list.
I would consider checking validity after-and-separate-from deserialization. In some cases, you might be able to just say "ignore invalid items in lists", but frankly in most cases if there is any problem you just want to reject the entire thing.

Only update some properties on an EF entity that are not set to null

I've got a browser sending up JSON but it only includes the properties of a given model that have been changed. So once the WCF DataContractJsonSerializer does it's work I have an object that will have perhaps only the ID and Description fields populated.
Attaching this to the DbContext as is will result in the description field being updated but all the other fields being set to their types default value in the database. This is because if WCF doesn't see the property specified in the JSON then it'll skip over it, meaning the property in the instance will just use the types default value as per the auto-implemented property.
So this means that I need to decide on which fields have been passed up without having access to the JSON itself. In fact it could be XML on the wire so all I can work from is this partly serialized object.
What seems most logical is to use null as the special value that means this property hasn't been serializd over. So in the constructor of the POCO model I set all the properties to null.
In the Update method I then use this serialized object as a stub. I have to walk each property and if the value isn't set to null then I set it's state to modified. As far as I can tell this is working without any side effects but I'm just not sure that this is the way to do something like this.
One limitation it does add is that the client can no longer intentionally set a property to null as that update would be lost. One way around this is to have a special int value that can be set to represent null in the database and perhaps an empty string to represent null in the database and have code in the update to look for these special values and then set the entity property to null. Far from ideal and likely to be prone to bugs.
Here is the code I currently have to process the update. I would really really appreciate advice as to a better, perhaps more obvious way of doing this.
To Summerise: How can I tell which properties on an model instance have been set by the DataContractSerializer/DataContractJsonSerializer and which are just using default values from it's constructor. Using special values is problematic as the client might want to set something to an empty string, or to 0 or -1 or indeed to null.
public T Update(T obj)
{
var entity = ds.Attach(obj);
// For each property in the model
foreach (var p in typeof(T).GetProperties())
{
// Get the value of the property
var v = p.GetValue(obj, null);
// Assume null means that the property wasn't passed from the client
if (v == null)
continue;
// Set this property on the entity to modified unless it's ID which won't change
if (p.Name != "ID")
dc.Entry(entity).Property(p.Name).IsModified = true;
}
dc.SaveChanges();
return entity;
}
UPDATE: Using Hammerstein's answer below to have self tracked models, I've updated my update function as below. Unfortunately due to my use of the Required attribute on the models for pre-save validation EF throws a wobbly when using a stub instance that contains nulls for the non modified values. You would think that EF would realise as part of it's validation that some fields are set to not modified but alas it doesn't so I'm forced to do a read and then update that. Actually this might be a good candidate for a separate question to post to try and avoid the read.
public virtual T Update(T obj)
{
var entity = ds.Find(obj.ID);
((TrackedModel)obj).Modified.ForEach(p => {
var prop = dc.Entry(entity).Property(p.PropertyName);
prop.CurrentValue = p.NewValue;
prop.IsModified = true;
});
dc.SaveChanges();
return entity;
}

My solution to this problem was a tracked change model, I created an abstract base class that had a list of strings, then for each property on my model, I called a method NotifyChanged( "MyProperty") which added the property to the list.
Because model binding will only set the fields that have been posted back, you should get an accurate list of fields that changed.
Then I loop through the list, and set the values on my entity.
Not clean, but it worked for my purpose.
Update: My solution did require me to get away from auto-properties and to hand write them. In the setter, after setting the value, I call NotifyChanged. I'm using this with MVC regular model binding, I don't think I have a working example of passing an object as JSON and deserializing. You could look at JSON.NET, controlling the serialization/deserialization I believe you can tell it to ignore default property values etc.

Serializing a Sitecore.Data.Items.Item to JSON

I'm currently attempting to build a service to retrieve and serialize a Sitecore data item to JSON, so our Javascript code can access Sitecore content data.
I've tried serializing the object directly with JavascriptSerializer and JSON.Net; both broke due to recursion likely due to the various circular references on the child properties.
I've also attempted to serialize the item to XML (via item.GetOuterXml()), then converting the Xml to JSON. The conversion worked fine; but it only retrieves fields that were set on the item itself, not the fields that were set in the _standardvalues. I tried calling item.Fields.ReadAll() before serializing, as well as a foreach loop with calls to item.Fields.EnsureField(Field.id); however, neither resulted in retrieving the missing fields. However, debugging the code; the Fields array appears to contain all inherited fields from its base template as well as the ones set on the item; so I'm guessing GetOuterXml is just ignoring all fields that weren't set specifically on the item.
The more I look at this, the more it looks like I'm going to need a custom model class to encapsulate the data item and the necessary fields, decorate it with the appropriate JSON.Net serialization attributes, and serialize from there. This feels like a dirty hack though.
So before I go down this road; I wanted to know if anyone here had experience serializing Sitecore content items to JSON for client-side consumption, and is there an easier way that I'm missing. Any constructive input is greatly appreciated.
Cheers,
Frank

I would suggest pursuing your approach of creating a custom model class to encapsulate just the item data you need to pass to the client. Then serialize that class to JSON. This cuts down on the amount of data you're sending over the wire and allows you to be selective about which data are being sent (for security reasons).
The CustomItem pattern and partial classes lend themselves to this approach very well. In the code samples below, the .base class is your base custom item wrapper. You can use this class to access fields and field values in a strongly-typed manner. The .instance class could be used for JSON serialization.
By splitting out the properties you want serialized, you have granular control over the data being sent back to the requesting client and you don't have to worry as much about circular references. If you need to make any changes to field definitions, you could simply change your .base class with minimal impact on your JSON serialization.
Hope this helps!
MyCustomItem.base.cs
public partial class MyCustomItem : Sitecore.Data.Items.CustomItem
{
public const string TitleFieldName = "Title";
public MyCustomItem(Item innerItem) : base(innerItem)
{
}
public static implicit operator MyCustomItem(Item innerItem)
{
return innerItem != null ? new MyCustomItem(innerItem) : null;
}
public static implicit operator Item(MyCustomItem customItem)
{
return customItem != null ? customItem.InnerItem : null;
}
public string Title
{
get { return InnerItem[TitleFieldName]); }
}
}
MyCustomItem.instance.cs
[JsonObject(MemberSerialization.OptIn)]
public partial class MyCustomItem
{
[JsonProperty("Title")]
public string JsonTitle
{
get { return Title; }
}
}

I wonder if you wouldn't be better off using an XSLT to recursively build the JSON?

XML Serialization and empty collections

I have a a property defined as:
[XmlArray("delete", IsNullable = true)]
[XmlArrayItem("contact", typeof(ContactEvent)),
XmlArrayItem("sms", typeof(SmsEvent))]
public List<Event> Delete { get; set; }
If the List<> Delete has no items
<delete />
is emitted. If the List<> Delete is set to null
<delete xsi:nil="true" />
is emitted. Is there a way using attributes to get the delete element not to be emitted if the collection has no items?
Greg - Perfect thanks, I didn't even read the IsNullable documentation just assumed it was signalling it as not required.
Rob Cooper - I was trying to avoid ISerializable, but Gregs suggestion works. I did run into the problem you outlined in (1), I broke a bunch of code by just returning null if the collection was zero length. To get around this I created a EventsBuilder class (the class I am serializing is called Events) that managed all the lifetime/creation of the underlying objects of the Events class that spits our Events classes for serialization.

I've had the same issue where I did not want an element outputted if the field is empty or 0.
The XML outputted could not use xsi:null="true" (by design).
I've read somewhere that if you include a property of type bool with the same name as the field you want to control but appended with 'Specified', the XMLSerializer will check the return value of this property to determine if the corresponding field should be included.
To achieve this without implementing IXMLSerializer:
public List<Event> Delete { get; set; }
[XMLIgnore]
public bool DeleteSpecified
{
get
{
bool isRendered = false;
if (Delete != null)
{
isRendered = (Delete.Count > 0);
}
return isRendered;
}
set
{
}
}

If you set IsNullable=false or just remove it (it is false by default), then the "delete" element will not be emitted. This will work only if the collection equals to null.
My guess is that there is a confusion between "nullability" in terms of .NET, and the one related to nullable elements in XML -- those that are marked by xml:nil attribute. XmlArrayAttribute.IsNullable property controls the latter.

First off, I would say ask yourself "What is Serialization?".
The XmlSerializer is doing exactly what it is supposed to be doing, persisting the current state of the object to XML. Now, I am not sure why the current behaviour is not "right" for you, since if you have initialized the List, then it is initialized.
I think you have three options here:
Add code to the Getter to return null if the collection has 0 items. This may mess up other code you have though.
Implement the IXmlSerializable interface and do all the work yourself.
If this is a common process, then you may want to look at my question "XML Serialization and Inherited Types" - Yes, I know it deals with another issue, but it shows you how to create a generic intermediary serialization class that can then be "bolted on" to allow a serilization process to be encapsulated. You could create a similar class to deal with overriding the default process for null/zero-item collections.
I hope this helps.

You could always implement IXmlSerializer and perform the serialization manually.
See http://www.codeproject.com/KB/cs/IXmlSerializable.aspx for an example.