We Keep Coding

Deserializing JSON with multi-level nesting

Yet another mtgjson.com inspired question; none of the other, similar questions are getting me where I need to be. First, a couple lines of sample JSON (from mtgjson's AllPrices.json):
"00028782-6ec2-54fe-8633-2c906d8f1076": {"prices": {"mtgo": {}, "mtgoFoil": {}, "paper": {"2019-12-01": 0.15}, "paperFoil": {}}},
"00040b50-3b84-5cea-b663-70038b87fa08": {"prices": {"mtgo": {"2019-12-02": 0.02}, "mtgoFoil": {"2019-12-02": 0.02}, "paper": {"2019-12-01": 0.15}, "paperFoil": {"2019-12-01": 0.53}}}
Each parent object is a GUID and the Price Info; the Price Info is the four types of prices offered, and for each of those four types, the price data is Last Updated Date and Price.
The classes I've created (after lots of other approaches, all of which have failed):
public class price_Class
{
public string Updated { get; set; }
public decimal Price { get; set; }
}
public class PriceInfo
{
[JsonProperty("mtgo")] public price_Class mtgo { get; set; }
[JsonProperty("mtgoFoil")] public price_Class mtgof { get; set; }
[JsonProperty("paper")] public price_Class RegPrice { get; set; }
[JsonProperty("paperFoil")] public price_Class FoilPrice { get; set; }
}
And how I'm using it:
dynamic prices = JsonConvert.DeserializeObject(sJSON);
IDictionary<string, JToken> pricelist = prices;
foreach (var priceline in pricelist)
{
sUUID = priceline.Key.ToString();
PriceInfo pi = JsonConvert.DeserializeObject<PriceInfo>(priceline.Value.ToString());
Stepping through in debug mode, I see that prices seems fine; pricelist, also. The foreach defines priceline as I'd expect, and sUUID is correctly defined - but pi shows up with all four sets of price data as null - not just those that are null, but those that should have data.
priceline.value looks fine, to me:
{{
"mtgo": {},
"mtgoFoil": {},
"paper": {
"2019-12-01": 0.53
},
"paperFoil": {
"2019-12-01": 4.53
}
}}
When I expand pi in the Locals window, it shows the four classes (FoilPrice, RegPrice, mtgo, mtgof), but the contents are null.
What I need, in case it's not obvious, is to have pi.RegPrice and pi.FoilPrice defined, with a Date and Price, when that data actually exists in the JSON.
I'll admit, nested classes and JSON in general is still outside my comfort zone; I appreciate all help!

The price_Class is not adequate for deserialize your JSON object.
try with this:
public class PriceInfo
{
[JsonProperty("mtgo")] public Dictionary<string, decimal> mtgo { get; set; }
[JsonProperty("mtgoFoil")] public Dictionary<string, decimal> mtgof { get; set; }
[JsonProperty("paper")] public Dictionary<string, decimal> RegPrice { get; set; }
[JsonProperty("paperFoil")] public Dictionary<string, decimal> FoilPrice { get; set; }
}

using this tool you can find exactly what are your DTO
the problem are on your json data as i believe date are not send like this in json it come as array of integers and with specific order like day month year .
as well as you need to create your DTO similar to json even in types you can't map it till you set the same type of json in your DTO
public class Prices
{
public Mtgo mtgo { get; set; }
public MtgoFoil mtgoFoil { get; set; }
public Paper paper { get; set; }
public PaperFoil paperFoil { get; set; }
}
public class RootObject
{
public Prices prices { get; set; }
}
don't forget to tag all of them with [JsonProperty("json prop name ")]

Parsing and formatting utilities for JSON.
A central concept in lift-json library is Json AST which models the structure of a JSON document as a syntax tree.
sealed abstract class JValue
case object JNothing extends JValue // 'zero' for JValue
case object JNull extends JValue
case class JString(s: String) extends JValue
case class JDouble(num: Double) extends JValue
case class JInt(num: BigInt) extends JValue
case class JBool(value: Boolean) extends JValue
case class JField(name: String, value: JValue) extends JValue
case class JObject(obj: List[JField]) extends JValue
case class JArray(arr: List[JValue]) extends JValue
It comes with Lift, but non-Lift users can add lift-json as a dependency in following ways. Note, replace XXX with correct Lift version.
SBT users
Add dependency to your project description:
val lift_json = "net.liftweb" %% "lift-json" % "XXX"
Maven users
Add dependency to your pom:
<dependency>
<groupId>net.liftweb</groupId>
<artifactId>lift-json</artifactId>
<version>XXX</version>
</dependency>
Summary of the features:
Fast JSON parser
LINQ style queries
Case classes can be used to extract values from parsed JSON
Diff & merge
DSL to produce valid JSON
XPath like expressions and HOFs to manipulate JSON
Pretty and compact printing
XML conversions
Serialization
Low level pull parser API
Try using this for deeply nested JSONs.

It seems that the odd sub-structure of {Prices:{label:{date:amount}}} just doesn't work well with Newtonsoft's (otherwise excellent) JSON tools.
I tried the various tools (some suggested here) to generate classes; they were getting confused by the dates, creating classes for each date. I even tried generating classes for just the substring of data (priceline.value, in the example) - nope, still wouldn't work.
I ended going with a brute-force, string manipulation approach; it's ugly, I'm not exactly proud of it - but I now have what I needed. Here's the relevant snippets, just in case anyone else stumbles on the same things as I did:
private static string RemoveNoise(string input)
{
input = Regex.Replace(input, #"\r\n?|\n", string.Empty); // no more NewLine stuff
return input.Replace(" ", string.Empty)
.Replace(#"""",string.Empty);
}
...
public class PriceData
{
public string UUID { get; set; }
public string Updated { get; set; }
public string Price { get; set; }
public string FoilUpd { get; set; }
public string FoilPrc { get; set; }
}
...
string sPaperTag = #"PAPER:{";
string sPprFlTag = #"PAPERFOIL:{";
...
dynamic prices = JsonConvert.DeserializeObject(sJSON);
IDictionary pricelist = prices;
foreach (var priceline in pricelist)
{
PriceData pData = new PriceData();
pData.UUID = priceline.Key.ToString();
bool bWeHavePrice = false;
string pi = RemoveNoise(priceline.Value.ToString().ToUpper());
// parse out paper, paperFoil dates & prices manually (unusual JSON format...)
iBeg = pi.IndexOf(sPaperTag);
if (iBeg >= 0)
{
sTemp = pi.Substring(iBeg, pi.Length - iBeg);
iBeg = sTemp.IndexOf(":") + 2;
iEnd = sTemp.IndexOf("}");
sTemp = sTemp.Substring(iBeg, iEnd - iBeg); // either YYYY-MM-DD:n.nn, or an empty string
iBeg = sTemp.IndexOf(":");
if (iBeg > 0)
{
if (DateTime.TryParse(sTemp.Substring(0, iBeg), out dtTemp)) { pData.Updated = dtTemp.ToString(); bWeHavePrice = true; }
if (Decimal.TryParse(sTemp.Substring(++iBeg, sTemp.Length - iBeg), out decTemp)) { pData.Price = decTemp.ToString(); bWeHavePrice = true; }
}
}
I do that string manipulation dance again for the foil prices; I'm not currently interested in the 'mtgo' or 'mtgoFoil' data.
I'm doing all that TryParse stuff to make sure I have a valid date or amount, but I'm using the results to populate parameters in a SQLCommand, so I have to have strings; seems like extra work, going from string to Date or Decimal, then back to string - but this way I don't get exceptions when executing the SQL Insert command.
My thanks to all who helped, or tried to help. And if someone figures out how to handle it via JSON.Net, I'd love to see it!

Passing c# object as query string

I want to pass C# object as query string & i used following code to get the desired result.
class Program
{
public static string GetQueryString(object obj)
{
var properties = from p in obj.GetType().GetProperties()
where p.GetValue(obj, null) != null
select p.Name + "=" + HttpUtility.UrlEncode(p.GetValue(obj, null).ToString());
return String.Join("&", properties.ToArray());
}
static void Main(string[] args)
{
Filters fil = new Filters();
fil.Age = 10;
fil.Id = "some id";
fil.Divisions = new List<string>();
fil.Divisions.Add("div 1");
fil.Divisions.Add("div 2");
fil.Divisions.Add("div 3");
fil.Names = new List<string>();
fil.Names.Add("name 1");
fil.Names.Add("name 2");
fil.Names.Add("name 3");
var queryStr = GetQueryString(fil);
Console.ReadKey();
}
}
public class Filters
{
public List<string> Names { get; set; }
public List<string> Divisions { get; set; }
public int Age { get; set; }
public string Id { get; set; }
}
using the above code give me following result:
Names=System.Collections.Generic.List%601%5bSystem.String%5d&Divisions=System.Collections.Generic.List%601%5bSystem.String%5d&Age=10&Id=some+id
The output is not a valid query string. I need help to convert any POCO class into query string format.
I have a similar JavaScript object and i am able to convert it into correct query string.
{
"id":"some id",
"age":10,
"division":["div 1","div 2","div 3"],
"names":["name 1","name 2","name 3"]
}
using Jquery I can say $.param(obj) and this will result in:
"id=some+id&age=10&division%5B%5D=div+1&division%5B%5D=div+2&division%5B%5D=div+3&names%5B%5D=name+1&names%5B%5D=name+2&names%5B%5D=name+3"
I want a similar output using c#.

It looks like The problem is that you are calling ToString() on your objects. List<String>.ToString() will return "List<System.String>", which is what you're seeing, except URL encoded.
You will need to either:
Provide an iterface with a ToQueryString method:
public interface IQueryStringable
{
string ToQueryString();
}
and have all classes you might want to use as query strings implement it, or
Rewrite your reflection so that it iterates sequences. Something like (pseudocode):
Get property.
See if it is an instance of IEnumerable. If not, proceed as before
Otherwise:
for each item, construct a string consisting of the property name, "[]=" and the value of that item.
Concatenate the produced strings and urlencode it.
For sanity's sake, I would recommend option 1, and I enjoy playing with reflection. It gets more complex if you want to allow arbitrary nesting of classes.

Xml Empty Tag Deserialization

Could you please help me to find the solution to deserialize xml file which contains an empty tag?
Example is here:
<Report>
<ItemsCount></ItemsCount>
</Report>
And I want to deserialize it into object of class like:
public class Report{
public int? ItemsCount { get;set;}
}
my xml schema which i'm using in deserialization is:
[XmlRoot]
public partial class Report
{
private int? itemsCount;
[XmlElement(IsNullable = true)]
public int? ItemsCount {
get
{
return itemsCount;
}
set
{
itemsCount = value;
}
}
It works well if the ItemsCount tag is missing at all, but if it is exist and is empty at the same moment, in that case it throwing the exception regarding lines there this tag is located in xml.
I saw a lot of links here while trying to find the solution, but without success.
And also, i don't want to just ignore the tag for all the cases, i want to get a null value instead then it is empty.

XmlSerializer is trying to convert string.Empty value of tag to integer and failing. Change your property as below to convert data type to string:
[XmlElement]
public string ItemsCount {
get
{
return itemsCount;
}
set
{
itemsCount = value;
}
This will set property Itemscount to empty in the above case.
For null value for the above property the xml should be as below:
<ItemsCount xs:Nil='true'/>

How about this approach?
Define the class as follows:
public class Report
{
[XmlIgnore]
public int? ItemsCount { get; set; }
}
Due to the XmlIgnore attribute, this tag will be treated as unknown.
When creating the serializer add the event handler:
var xs = new XmlSerializer(typeof(Report));
xs.UnknownElement += Xs_UnknownElement;
In the event handler interpret an empty string as null:
private void Xs_UnknownElement(object sender, XmlElementEventArgs e)
{
var report = (Report)e.ObjectBeingDeserialized;
if (e.Element.InnerText == string.Empty)
report.ItemsCount = null;
else
report.ItemsCount = int.Parse(e.Element.InnerText);
}
Use the serializer as usual:
Report report;
using (var fs = new FileStream("test.xml", FileMode.Open))
{
report = (Report)xs.Deserialize(fs);
}

To my understanding, the described behaviour is correct; if the tag ItemsCount is missing, its value is null; if it is empty, its value cannot be converted from "" to a value of int?. That being said, it would be possible to implement some custom parsing into the accessors of ItemsCount, which would have to be of type string. However, this seems more like a workaround to me. If possible, the document should be changed to begin with.

c# is there a method to serialize to UrlEncoded?

I want to use facebook's API and i find it hard to convert objects to urlEncoded.
so, for now i have something like:
string postData = JsonConvert.SerializeObject(req);
postData = postData.Replace(#"\", "");
postData = HttpUtility.UrlEncode(postData);
byte[] data = Encoding.UTF8.GetBytes(postData);
string facebookUrl = "https://graph.facebook.com/v2.5/";
problem is that facebook doesn't accept jsons but UrlEncoded data, as it seems, correct me if im wrong.
So, Im pretty sure converting objects to UrlEncoded string is impossbile in .Net 4.5.1 because I've tried to use some of the answers for this questions that are while ago they are not working for me.
for example:
var result = new List<string>();
foreach (var property in TypeDescriptor.GetProperties(req))
{
result.Add(property.Name + "=" + property.GetValue(req));
}
postData = string.Join("&", result);
but .Name and .GetValue aren't defined at all.
Would like to get some help with that, TIA.
Objects i use:
internal sealed class FacebookValidationRequest
{
public string access_token;
public fbReq[] batch;
public string method;
public string format;
public int pretty;
public int suppress_http_code;
public string debug;
public FacebookValidationRequest(string appId, string userToken)
{
access_token = userToken;
batch = new[]
{
//test code
new fbReq("GET", "me"),
new fbReq("GET", "me/friends?limit=50") //,
//new fbReq("GET", "app?access_token=" + userToken)
};
method = "post";
format = "json";
pretty = 0;
suppress_http_code = 1;
debug = "all";
}
}
internal sealed class fbReq
{
public string method;
public string relative_url;
public fbReq(string m, string url)
{
method = m;
relative_url = url;
}
}
FacebookValidationRequest req = new FacebookValidationRequest(appToken, userToken);
Also, took the token for the facebook debugger site
how facebook wants to object to look like after encoding:
access_token=mytoken&batch=%5B%7B%22method%22%3A%22GET%22%2C%20%22relative_url%22%3A%22me%22%7D%2C%7B%22method%22%3A%22GET%22%2C%20%22relative_url%22%3A%22me%2Ffriends%3Flimit%3D50%22%7D%5D&debug=all&fields=id%2Cname&format=json&method=post&pretty=0&suppress_http_code=1

Seems to me that the easiest way to do this is with Attributes to describe your properties, just like how the .Net Json's DataContract system does it. Basically, you assign an attribute to each property you want serialized, and make that attribute contain the name to serialize it as. I don't think you want to get into the mess of actually writing your own DataContractSerializer, though, so it might be easier to simply create your own Property class and a simple serializer using reflection.
The attribute class:
[AttributeUsage(AttributeTargets.Property)]
public sealed class UrlEncodeAttribute : System.Attribute
{
public String Name { get; private set; }
public UrlEncodeAttribute(String name)
{
this.Name = name;
}
}
Then, to apply to your data class... put the attributes on all properties:
internal sealed class FacebookValidationRequest
{
[UrlEncodeAttribute("access_token")]
public String AccessToken { get; set; }
[UrlEncodeAttribute("method")]
public String Method { get; set; }
[UrlEncodeAttribute("format")]
public String Format { get; set; }
[UrlEncodeAttribute("pretty")]
public Int32 Pretty { get; set; }
[UrlEncodeAttribute("suppress_http_code")]
public Int32 SuppressHttpCode { get; set; }
[UrlEncodeAttribute("debug")]
public string Debug { get; set; }
public fbReq[] Batch { get; set; }
[UrlEncodeAttribute("batch")]
public String BatchString
{
get
{
// put your json serialization code here to return
// the contents of Batch as json string.
}
}
}
As you see, Batch does not have the UrlEncodeAttribute, while its string representation BatchString does. Its get is what will be called by the serializer, so you can put the conversion code in there.
Also note that thanks to the text names you give in the attributes, your properties don't need to have the names you actually get in the serialization, which looks much cleaner in my opinion. C#'s own serialization to xml and json works in the same way.
Now, let's take a look at the actual serialization, using reflection to get those properties:
public static String Serialize(Object obj, Boolean includeEmpty)
{
// go over the properties, see which ones have a UrlEncodeAttribute, and process them.
StringBuilder sb = new StringBuilder();
PropertyInfo[] properties = obj.GetType().GetProperties();
foreach (PropertyInfo p in properties)
{
object[] attrs = p.GetCustomAttributes(true);
foreach (Object attr in attrs)
{
UrlEncodeAttribute fldAttr = attr as UrlEncodeAttribute;
if (attr == null)
continue;
String objectName = fldAttr.Name;
Object objectDataObj = p.GetValue(obj, null);
String objectData = objectDataObj == null ? String.Empty : objectDataObj.ToString();
if (objectData.Length > 0 || includeEmpty)
{
objectData = HttpUtility.UrlEncode(objectData);
objectName= HttpUtility.UrlEncode(objectName);
if (sb.Length > 0)
sb.Append("&");
sb.Append(objectName).Append("=").Append(objectData);
}
break; // Only handle one UrlEncodeAttribute per property.
}
}
return sb.ToString();
}
A more advanced version of this could be made by including a serialization method property in the UrlEncodeAttribute class (probably best done with an enum), so you can simply specify to serialize the array on the fly using json. You'll obviously need to put the actual json converter into the Serialize function then. I thought using the getter on a dummy property as preparation method was simpler, here.
Obviously, calling it is simply this: (assuming here the Serialize() function is in a class called UrlEncodeSerializer)
FacebookValidationRequest fbreq = new FacebookValidationRequest();
// fill your data into fbreq here
// ...
// includeEmpty is set to true for testing here, but normally in
// UrlEncoded any missing property is just seen as empty anyway, so
// there should be no real difference.
String serialized = UrlEncodeSerializer.Serialize(fbreq, true);

XML deserialization - throwing custom errors

So I have the following method:
private int? myIntField
[System.Xml.Serialization.XmlElementAttribute(Form = System.Xml.Schema.XmlSchemaForm.Unqualified)]
public int? IntField{
get {
return this.myIntField;
}
set {
this.myIntField= value;
}
}
Now, I am deserializing xml from a post, if for whatever reason I am getting a string, such as "here is the int field: 55444" instead of 55444, the error I get in response is: Input string was not in a correct format. which isn't very specific, especially considering I will have more than one int field I need to verify.
Originally, I was planning something like this:
private string myIntField
[System.Xml.Serialization.XmlElementAttribute(Form = System.Xml.Schema.XmlSchemaForm.Unqualified)]
public int? IntField{
get {
return this.myIntField.CheckValue();
}
set {
this.myIntField= value;
}
}
Where CheckValue performs a try-parse to an Int32, and if it fails it returns a null and adds an error to a list. However, I can't seem to nail this set-up for the generated classes.
Is there I way I can throw a specific error if I am getting strings in place of ints, DateTimes, etc?

It's easy if you have schema(s) for you XML and validate it against schema before deserializing. Suppose you have schema(s) for your XML, you can initialize a XmlSchemaSet, add your schema(s) in it and the:
var document = new XmlDocument();
document.LoadXml(xml); // this a string holding the XML
document.Schemas.XmlResolver = null; //if you don't need to resolve every references
document.Schemas.Add(SchemaSet); // System.Xml.Schema.XmlSchemaSet instance filled with schemas
document.Validate((sender, args) => { ... }); //args are of type ValidationEventArgs and hold problem if there is one...
Personally I think this is a better approach, because you can validate your XML before deserializing and be sure the XML is correct otherwise the deserializer will most probably throw an exception if something is wrong and you will almost never be able to show a meaningful feedback to the user...
P.S. I recommend creating schema(s) describing the XML

The "Input string was not in a correct format" messages comes from a standard System.FormatException raised by a call to int.Parse, added to the automatically generated assembly that does the deserialization. I don't think you can add some custom logic to that.
One solution is to do something like this:
[XmlElement("IntField")]
[Browsable(false)] // not displayed in grids
[EditorBrowsable(EditorBrowsableState.Never)] // not displayed by intellisense
public string IntFieldString
{
get
{
return DoSomeConvert(IntField);
}
set
{
IntField = DoSomeOtherConvert(value);
}
}
[XmlIgnore]
public int? IntField { get; set; }
It's not perfect, because you can still get access to the public IntFieldString, but at least, the "real" IntField property is used only programmatically, but not by the XmlSerializer (XmlIgnore), while the field that's holding the value back & forth is hidden from programmers (EditorBrowsable), grids (Browsable), etc... but not from the XmlSerializer.

I have three approaches for you.
Assuming your data is being entered by a user in a user interface, use input validation to ensure the data is valid. It seems odd to allow random strings to be entered when it should be an integer.
Use exactly the approach you suggest above. Here's an example using LINQ Pad
void Main()
{
using(var stream = new StringReader(
"<Items><Item><IntValue>1</IntValue></Item></Items>"))
{
var serializer = new XmlSerializer(typeof(Container));
var items = (Container)serializer.Deserialize(stream);
items.Dump();
}
}
[XmlRoot("Items")]
public class Container
{
[XmlElement("Item")]
public List<Item> Items { get; set; }
}
public class Item
{
[XmlElement("IntValue")]
public string _IntValue{get;set;}
[XmlIgnore]
public int IntValue
{
get
{
// TODO: check and throw appropriate exception
return Int32.Parse(_IntValue);
}
}
}
Take control of serialization using IXmlSerializable, here's another example
void Main()
{
using(var stream = new StringReader(
"<Items><Item><IntValue>1</IntValue></Item></Items>"))
{
var serializer = new XmlSerializer(typeof(Container));
var items = (Container)serializer.Deserialize(stream);
items.Dump();
}
}
[XmlRoot("Items")]
public class Container
{
[XmlElement("Item")]
public List<Item> Items { get; set; }
}
public class Item : IXmlSerializable
{
public int IntValue{get;set;}
public void WriteXml (XmlWriter writer)
{
writer.WriteElementString("IntValue", IntValue.ToString());
}
public void ReadXml (XmlReader reader)
{
var v = reader.ReadElementString();
// TODO: check and throw appropriate exception
IntValue = int.Parse(v);
}
public XmlSchema GetSchema()
{
return(null);
}
}