Data Serialization on Custom DataSourceControl - c#

I am writing a custom DataSourceControl that will basically have the Select capability working just as a normal ObjectDataSource, with a TypeName and SelectMethod properties.
The Data coming from the TypeName class will be saved in Files indexed by the Hash of the parameter values and a ContextName. Meaning that, everytime a GridView requests the DataSource and the same parameter values are given, the control will find the corresponding File and load the data from there. In fact, every different combination of parameter values will generate a new File with the data.
This functionality could be very helpful in some cases we have when the data takes too long to be processed and retrieved from the database and doesn't need to be live to the user (its always from the last day).
The main difficulty I'm having is to serialize the Data coming from the SelectMethod. As the only thing i know is that the return type will be an instance of IEnumerable. I'm using XMLSerializer for saving and retrieving the data content from a file, but when trying to serialize it gives me an error Cannot serialize interface.
This is the basic code that executes the SelectMethod and does the serialization part:
//Gets the select type
Type selectType = Type.GetType(TypeName);
//Gets the select method
MethodInfo selectMethod = selectType.GetMethod(SelectMethod, BindingFlags.Instance | BindingFlags.Public);
//Creates a new instance of the TypeName class
object selectInstance = Activator.CreateInstance(selectType);
//Executes the select method
object selectResult = selectMethod.Invoke(selectInstance, parameters);
IEnumerable list = (IEnumerable)selectResult;
//Create a serializer
XmlSerializer serializer = new XmlSerializer(typeof(IEnumerable));
//Writes to the XML file
using (XmlWriter writer = XmlWriter.Create(filePath))
{
serializer.Serialize(writer, list);
}
I would use this code to deserialize:
//Creates the XML File
XmlSerializer deserializer = new XmlSerializer(typeof(IEnumerable));
IEnumerable list = null;
//Reads from the XML file
using (XmlReader writer = XmlReader.Create(filePath))
{
list = (IEnumerable) deserializer.Deserialize(writer);
}
How can i generically Serialize/Deserialize the Select Method result to XML?
Update:
I tried using System.Web.UI.LosFormatter to Serialize/Deserialize the data. It did both actions with an IEnumerable instance but i had to put the Serializable attribute on the entities. However i noticed a significant difference in performance compared to XMLSerializer when retrieving the data from the file. System.Web.UI.LosFormatter was 4 times slower at deserializing on my specific test (4MB file). The data file would be half the size compared to the XMLSerializer tho. So, for me, XMLSerializer still the best option.
Update2:
Tried to make a simple test using ServiceStack JsonSerializer with the following code:
List<Dummy> dummies = new List<Dummy>();
dummies.Add(new Dummy() { Name = "name" });
dummies.Add(new Dummy() { Name = "name1" });
dummies.Add(new Dummy() { Name = "name2" });
IEnumerable enumerableThing = dummies;
string filePath = Path.Combine(Server.MapPath("~/App_Data"), "data2.json");
using (StreamWriter writer = new StreamWriter(filePath))
{
JsonSerializer.SerializeToWriter<IEnumerable>(enumerableThing, writer);
}
using (StreamReader reader = new StreamReader(filePath))
{
enumerableThing = JsonSerializer.DeserializeFromReader<IEnumerable>(reader);
}
It does work serializing, but when trying to DeserializeFromReader, it gives me the error "Type System.Collections.IEnumerable is not of type IDictionary<,>".
Is there any way i could make this work?

Well, as you've already discovered, you can't serialize an interface...that said, you might be able to work around this and still be "in the dark" about the actual object types:
First, any IEnumerable can be transmogrified into an array via a Cast/ToArray combo
var enumerableThing = Foo.GetEnumerable();
var asArray = enumerableThing.Cast<object>().ToArray();
Second, Arrays of a "known type" are Serializable
var allContainedTypes = asArray.Select(x => x.GetType()).Distinct().ToArray();
var ser = new XmlSerializer(asArray.GetType(), allContainedTypes);
var ser = new XmlSerializer(asArray.GetType());
var sb = new StringBuilder();
using(var sw = new StringWriter(sb))
using(var xw = XmlWriter.Create(sw))
ser.Serialize(xw, asArray);
sb.ToString().Dump();
Or, all together:
void Main()
{
var enumerableThing = Foo.GetEnumerable();
var asArray = enumerableThing.Cast<object>().ToArray();
var allContainedTypes = asArray.Select(x => x.GetType()).Distinct().ToArray();
var ser = new XmlSerializer(asArray.GetType(), allContainedTypes);
var sb = new StringBuilder();
using(var sw = new StringWriter(sb))
using(var xw = XmlWriter.Create(sw))
ser.Serialize(xw, asArray);
sb.ToString().Dump();
}
public class Foo
{
public static IEnumerable GetEnumerable()
{
return new[] { 1, 2, 3, 4, 5, 6, 7 };
}
public static IEnumerable GetEnumerable2()
{
return new object[] { "1", 2, "bob", 4, null, 6, 7 };
}
}
Oh, this produces the format:
<?xml version="1.0" encoding="utf-16"?>
<ArrayOfAnyType
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<anyType xsi:type="xsd:int">1</anyType>
<anyType xsi:type="xsd:int">2</anyType>
<anyType xsi:type="xsd:int">3</anyType>
<anyType xsi:type="xsd:int">4</anyType>
<anyType xsi:type="xsd:int">5</anyType>
<anyType xsi:type="xsd:int">6</anyType>
<anyType xsi:type="xsd:int">7</anyType>
</ArrayOfAnyType>
Edit: Deserialization does pose some problems - however, you can do something akin to:
// Least common denominator...
object[] tempResult;
var assemblyStuffIsFrom = Assembly.GetExecutingAssembly();
var allSerializableTypes = assemblyStuffIsFrom
.GetTypes()
.Where(t => t.GetCustomAttributes(typeof(SerializableAttribute), true).Any())
// TODO: add as much filtering as you can here to help trim down the set
.ToArray();
var hope = new XmlSerializer(typeof(object[]), allSerializableTypes);
using(var sr = new StringReader(sb.ToString()))
using(var xr = XmlReader.Create(sr))
tempResult = ((object[])hope.Deserialize(xr));

Related

How to update/modify values in Json File

I want to update values in the following type of json file in c#
{"ItemName":"ABC","Id":1}
{"ItemName":"DEF","Id":2}
{"ItemName":"GHI","Id":3}
{"ItemName":"ABC","Id":1}
{"ItemName":"JKL","Id":2}
{"ItemName":"MNO","Id":3}
How can i remove duplicate values based on 'ItemName' and than update Id. Like expected output will be:
{"ItemName":"ABC","Id":1}
{"ItemName":"DEF","Id":2}
{"ItemName":"GHI","Id":3}
{"ItemName":"JKL","Id":4}
{"ItemName":"MNO","Id":5}
I searched alot about json.writer function but couldn't found a solution for a file without using [] and , seperation.
However i can read the file by using below code
var jsonReader = new JsonTextReader(new StringReader(File.ReadAllText("path")))
{
SupportMultipleContent = true
};
var jsonSerializer = new JsonSerializer();
dynamic data = jsonSerializer.Deserialize(jsonReader);
while (jsonReader.Read())
{
String Items_name = data.ItemName;
}
You have a class to deser to:
public record X(string ItemName, int Id);
You read your file deser'ing each line:
var xs = File.ReadAllLines("path").Select(l => JsonConvert.DeserializeObject<X>(l)).ToList();
You remove your duplicates:
var deduped = xs.DistinctBy(x => x.ItemName);
and perform your other manipulations etc (maybe you will want to call ToList on the above). Then you reserialize:
File.WriteAllLines("path", deduped.Select(x => JsonConvert.SerializeObject(x)).ToArray());

Return object of Type T from XML data

I am parsing an XML and able to extract data into List of object using below code. But the front end uses an type of T binding data object which is currently working. Once I return ienumerable it is not working. How can I convert this enumerable to type of T?
var serializer = new XmlSerializer(typeof(Feed));
var xmlfeed = new XMLfeed();
using (var reader = new StringReader(doc.OuterXml))
{
xmlfeed = (XMLfeed)(serializer.Deserialize(reader));
}
foreach (var properties in XMLfeed?.values)
{
var feedObj= new FeedObj();
feedObj.Name = properties?.Name;
feedObj.Value = properties?.Value;
lstObjfeed.Add(feed);
}
return lstObjfeed.AsEnumerable();
Can you use Enumerable.Cast<TResult>(IEnumerable) from LINQ?
So something like lstObjfeed.AsEnumerable().Cast<T>().

Find differences between two json objects

Are there any libraries in .Net to help compare and find differences between two json objects? I've found some solutions available for JavaScript, but nothing interesting for C#. The point of my question is to create json with changes marked in some way, based on the comparison. So that the user could see where the changes are.
using Microsoft.XmlDiffPatch;
using Newtonsoft.Json;
Convert each json to xml and use MS XmlDiff libary. Available on nuget. Differences are given in another xml doc which here I write to the console. This is suitable for unit testing for example.
public bool CompareJson(string expected, string actual)
{
var expectedDoc = JsonConvert.DeserializeXmlNode(expected, "root");
var actualDoc = JsonConvert.DeserializeXmlNode(actual, "root");
var diff = new XmlDiff(XmlDiffOptions.IgnoreWhitespace |
XmlDiffOptions.IgnoreChildOrder);
using (var ms = new MemoryStream())
using (var writer = new XmlTextWriter(ms, Encoding.UTF8))
{
var result = diff.Compare(expectedDoc, actualDoc, writer);
if (!result)
{
ms.Seek(0, SeekOrigin.Begin);
Console.WriteLine(new StreamReader(ms).ReadToEnd());
}
return result;
}
}
I have used different JSON objects than those in your example but it will apply to your case correctly.
private static string GetJsonDiff(string action, string existing, string modified, string objectType)
{
// convert JSON to object
JObject xptJson = JObject.Parse(modified);
JObject actualJson = JObject.Parse(existing);
// read properties
var xptProps = xptJson.Properties().ToList();
var actProps = actualJson.Properties().ToList();
// find differing properties
var auditLog = (from existingProp in actProps
from modifiedProp in xptProps
where modifiedProp.Path.Equals(existingProp.Path)
where !modifiedProp.Value.ToString().Equals(existingProp.Value.ToString())
select new AuditLog
{
Field = existingProp.Path,
OldValue = existingProp.Value.ToString(),
NewValue = modifiedProp.Value.ToString(),
Action = action, ActionBy = GetUserName(),
ActionDate = DateTime.UtcNow.ToLongDateString(),
ObjectType = objectType
}).ToList();
return JsonConvert.SerializeObject(auditLog);
}
I think your best bet is to use JSON.NET to create two JSON objects, then recursively loop through the tree, comparing each node to see if it exists and is equal while you go.
I think the best way to go here is to create objects using newtonsoft json.http://www.nuget.org/packages/newtonsoft.json/
So, you will have two objects of the same type, which you can easily compare and mark the differences.
private IEnumerable<JProperty> JSONCompare(string expectedJSON, string actualJSON)
{
// convert JSON to object
JObject xptJson = JObject.Parse(expectedJSON);
JObject actualJson = JObject.Parse(actualJSON);
// read properties
var xptProps = xptJson.Properties().ToList();
var actProps = actualJson.Properties().ToList();
// find missing properties
var missingProps = xptProps.Where(expected => actProps.Where(actual => actual.Name == expected.Name).Count() == 0);
return missingProps;
}

JavaScriptSerializer : Unable to deserialize object containing HashSet field

I am trying to deserialize an instance of the following class from a JSON string using JavaScriptSerializer:
public class Filter
{
public HashSet<int> DataSources { get; set; }
}
Here is the code I am trying out:
Filter f = new Filter();
f.DataSources = new HashSet<int>(){1,2};
string json = (new JavaScriptSerializer()).Serialize(f);
var g= (new JavaScriptSerializer()).Deserialize<Filter>(json);
It errors out with the following message:
Object of type 'System.Collections.Generic.List1[System.Int32]'
cannot be converted to type
'System.Collections.Generic.HashSet1[System.Int32]'.
Apparently, the serializer is unable to distinguish between a list and set from JSON representation. What is the solution to this?
Note : I would prefer avoiding the use of external libraries due to constraints at work.
What is the solution to this?
Use Json.Net. This code works...
Filter f = new Filter();
f.DataSources = new HashSet<int>() { 1, 2 };
string json = JsonConvert.SerializeObject(f);
var g = JsonConvert.DeserializeObject<Filter>(json);
EDIT
DataContractJsonSerializer seems to work too...
DataContractJsonSerializer dcjs = new DataContractJsonSerializer(typeof(Filter));
var g2 = dcjs.ReadObject(new MemoryStream(Encoding.UTF8.GetBytes(json))) as Filter;
Here's my simple NOT so great solution, but it works.
var dataList = new JavaScriptSerializer().Deserialize<List<int>>(returnData);
var data = new HashSet<int>(dataList);

How do I convert a dictionary to a JSON String in C#?

I want to convert my Dictionary<int,List<int>> to JSON string. Does anyone know how to achieve this in C#?
This answer mentions Json.NET but stops short of telling you how you can use Json.NET to serialize a dictionary:
return JsonConvert.SerializeObject( myDictionary );
As opposed to JavaScriptSerializer, myDictionary does not have to be a dictionary of type <string, string> for JsonConvert to work.
Serializing data structures containing only numeric or boolean values is fairly straightforward. If you don't have much to serialize, you can write a method for your specific type.
For a Dictionary<int, List<int>> as you have specified, you can use Linq:
string MyDictionaryToJson(Dictionary<int, List<int>> dict)
{
var entries = dict.Select(d =>
string.Format("\"{0}\": [{1}]", d.Key, string.Join(",", d.Value)));
return "{" + string.Join(",", entries) + "}";
}
But, if you are serializing several different classes, or more complex data structures, or especially if your data contains string values, you would be better off using a reputable JSON library that already knows how to handle things like escape characters and line breaks. Json.NET is a popular option.
Json.NET probably serializes C# dictionaries adequately now, but when the OP originally posted this question, many MVC developers may have been using the JavaScriptSerializer class because that was the default option out of the box.
If you're working on a legacy project (MVC 1 or MVC 2), and you can't use Json.NET, I recommend that you use a List<KeyValuePair<K,V>> instead of a Dictionary<K,V>>. The legacy JavaScriptSerializer class will serialize this type just fine, but it will have problems with a dictionary.
Documentation: Serializing Collections with Json.NET
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.Serialization.Json;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
Dictionary<int, List<int>> foo = new Dictionary<int, List<int>>();
foo.Add(1, new List<int>( new int[] { 1, 2, 3, 4 }));
foo.Add(2, new List<int>(new int[] { 2, 3, 4, 1 }));
foo.Add(3, new List<int>(new int[] { 3, 4, 1, 2 }));
foo.Add(4, new List<int>(new int[] { 4, 1, 2, 3 }));
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(Dictionary<int, List<int>>));
using (MemoryStream ms = new MemoryStream())
{
serializer.WriteObject(ms, foo);
Console.WriteLine(Encoding.Default.GetString(ms.ToArray()));
}
}
}
}
This will write to the console:
[{\"Key\":1,\"Value\":[1,2,3,4]},{\"Key\":2,\"Value\":[2,3,4,1]},{\"Key\":3,\"Value\":[3,4,1,2]},{\"Key\":4,\"Value\":[4,1,2,3]}]
Simple One-Line Answer
(using System.Web.Script.Serialization )
This code will convert any Dictionary<Key,Value> to Dictionary<string,string> and then serialize it as a JSON string:
var json = new JavaScriptSerializer().Serialize(yourDictionary.ToDictionary(item => item.Key.ToString(), item => item.Value.ToString()));
It is worthwhile to note that something like Dictionary<int, MyClass> can also be serialized in this way while preserving the complex type/object.
Explanation (breakdown)
var yourDictionary = new Dictionary<Key,Value>(); //This is just to represent your current Dictionary.
You can replace the variable yourDictionary with your actual variable.
var convertedDictionary = yourDictionary.ToDictionary(item => item.Key.ToString(), item => item.Value.ToString()); //This converts your dictionary to have the Key and Value of type string.
We do this, because both the Key and Value has to be of type string, as a requirement for serialization of a Dictionary.
var json = new JavaScriptSerializer().Serialize(convertedDictionary); //You can then serialize the Dictionary, as both the Key and Value is of type string, which is required for serialization.
Sorry if the syntax is the tiniest bit off, but the code I'm getting this from was originally in VB :)
using System.Web.Script.Serialization;
...
Dictionary<int,List<int>> MyObj = new Dictionary<int,List<int>>();
//Populate it here...
string myJsonString = (new JavaScriptSerializer()).Serialize(MyObj);
If your context allows it (technical constraints, etc.), use the JsonConvert.SerializeObject method from Newtonsoft.Json : it will make your life easier.
Dictionary<string, string> localizedWelcomeLabels = new Dictionary<string, string>();
localizedWelcomeLabels.Add("en", "Welcome");
localizedWelcomeLabels.Add("fr", "Bienvenue");
localizedWelcomeLabels.Add("de", "Willkommen");
Console.WriteLine(JsonConvert.SerializeObject(localizedWelcomeLabels));
// Outputs : {"en":"Welcome","fr":"Bienvenue","de":"Willkommen"}
In Asp.net Core use:
using Newtonsoft.Json
var obj = new { MyValue = 1 };
var json = JsonConvert.SerializeObject(obj);
var obj2 = JsonConvert.DeserializeObject(json);
You can use System.Web.Script.Serialization.JavaScriptSerializer:
Dictionary<string, object> dictss = new Dictionary<string, object>(){
{"User", "Mr.Joshua"},
{"Pass", "4324"},
};
string jsonString = (new JavaScriptSerializer()).Serialize((object)dictss);
net core :
System.Text.Json.JsonSerializer.Serialize(dict)
You could use JavaScriptSerializer.
It seems a lot of different libraries and what not have seem to come and go over the previous years. However as of April 2016, this solution worked well for me. Strings easily replaced by ints.
TL/DR; Copy this if that's what you came here for:
//outputfilename will be something like: "C:/MyFolder/MyFile.txt"
void WriteDictionaryAsJson(Dictionary<string, List<string>> myDict, string outputfilename)
{
DataContractJsonSerializer js = new DataContractJsonSerializer(typeof(Dictionary<string, List<string>>));
MemoryStream ms = new MemoryStream();
js.WriteObject(ms, myDict); //Does the serialization.
StreamWriter streamwriter = new StreamWriter(outputfilename);
streamwriter.AutoFlush = true; // Without this, I've run into issues with the stream being "full"...this solves that problem.
ms.Position = 0; //ms contains our data in json format, so let's start from the beginning
StreamReader sr = new StreamReader(ms); //Read all of our memory
streamwriter.WriteLine(sr.ReadToEnd()); // and write it out.
ms.Close(); //Shutdown everything since we're done.
streamwriter.Close();
sr.Close();
}
Two import points. First, be sure to add System.Runtime.Serliazation as a reference in your project inside Visual Studio's Solution Explorer. Second, add this line,
using System.Runtime.Serialization.Json;
at the top of the file with the rest of your usings, so the DataContractJsonSerializer class can be found. This blog post has more information on this method of serialization.
Data Format (Input / Output)
My data is a dictionary with 3 strings, each pointing to a list of strings. The lists of strings have lengths 3, 4, and 1.
The data looks like this:
StringKeyofDictionary1 => ["abc","def","ghi"]
StringKeyofDictionary2 => ["String01","String02","String03","String04"]
Stringkey3 => ["someString"]
The output written to file will be on one line, here is the formatted output:
[{
"Key": "StringKeyofDictionary1",
"Value": ["abc",
"def",
"ghi"]
},
{
"Key": "StringKeyofDictionary2",
"Value": ["String01",
"String02",
"String03",
"String04",
]
},
{
"Key": "Stringkey3",
"Value": ["SomeString"]
}]
Here's how to do it using only standard .Net libraries from Microsoft …
using System.IO;
using System.Runtime.Serialization.Json;
private static string DataToJson<T>(T data)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer serialiser = new DataContractJsonSerializer(
data.GetType(),
new DataContractJsonSerializerSettings()
{
UseSimpleDictionaryFormat = true
});
serialiser.WriteObject(stream, data);
return Encoding.UTF8.GetString(stream.ToArray());
}
This is Similar to what Meritt has posted earlier. just posting the complete code
string sJSON;
Dictionary<string, string> aa1 = new Dictionary<string, string>();
aa1.Add("one", "1"); aa1.Add("two", "2"); aa1.Add("three", "3");
Console.Write("JSON form of Person object: ");
sJSON = WriteFromObject(aa1);
Console.WriteLine(sJSON);
Dictionary<string, string> aaret = new Dictionary<string, string>();
aaret = ReadToObject<Dictionary<string, string>>(sJSON);
public static string WriteFromObject(object obj)
{
byte[] json;
//Create a stream to serialize the object to.
using (MemoryStream ms = new MemoryStream())
{
// Serializer the object to the stream.
DataContractJsonSerializer ser = new DataContractJsonSerializer(obj.GetType());
ser.WriteObject(ms, obj);
json = ms.ToArray();
ms.Close();
}
return Encoding.UTF8.GetString(json, 0, json.Length);
}
// Deserialize a JSON stream to object.
public static T ReadToObject<T>(string json) where T : class, new()
{
T deserializedObject = new T();
using (MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(json)))
{
DataContractJsonSerializer ser = new DataContractJsonSerializer(deserializedObject.GetType());
deserializedObject = ser.ReadObject(ms) as T;
ms.Close();
}
return deserializedObject;
}
Just for reference, among all the older solutions: UWP has its own built-in JSON library, Windows.Data.Json.
JsonObject is a map that you can use directly to store your data:
var options = new JsonObject();
options["foo"] = JsonValue.CreateStringValue("bar");
string json = options.ToString();
improved mwjohnson's version:
string WriteDictionaryAsJson_v2(Dictionary<string, List<string>> myDict)
{
string str_json = "";
DataContractJsonSerializerSettings setting =
new DataContractJsonSerializerSettings()
{
UseSimpleDictionaryFormat = true
};
DataContractJsonSerializer js =
new DataContractJsonSerializer(typeof(Dictionary<string, List<string>>), setting);
using (MemoryStream ms = new MemoryStream())
{
// Serializer the object to the stream.
js.WriteObject(ms, myDict);
str_json = Encoding.Default.GetString(ms.ToArray());
}
return str_json;
}

Categories