I'm trying to deserialize json to an object model where the collections are represented as IList<T> types.
The actual deserializing is here:
JavaScriptSerializer serializer = new JavaScriptSerializer();
return serializer.Deserialize<IList<Contact>>(
(new StreamReader(General.GetEmbeddedFile("Contacts.json")).ReadToEnd()));
Before i post the exception i'm getting you should know what the implicit conversions are. This is the Contact type:
public class Contact
{
public int ID { get; set; }
public string Name { get; set; }
public LazyList<ContactDetail> Details { get; set; }
//public List<ContactDetail> Details { get; set; }
}
And this is the ContactDetail type:
public class ContactDetail
{
public int ID { get; set; }
public int OrderIndex { get; set; }
public string Name { get; set; }
public string Value { get; set; }
}
The important thing to know with the LazyList<T> is that it implements IList<T>:
public class LazyList<T> : IList<T>
{
private IQueryable<T> _query = null;
private IList<T> _inner = null;
private int? _iqueryableCountCache = null;
public LazyList()
{
this._inner = new List<T>();
}
public LazyList(IList<T> inner)
{
this._inner = inner;
}
public LazyList(IQueryable<T> query)
{
if (query == null)
throw new ArgumentNullException();
this._query = query;
}
Now this LazyList<T> class definition was fine until i tried deserializing Json into it. The System.Web.Script.Serialization.JavaScriptSerializer seems to want to serialize lists to List<T> which makes sense coz of it's age but i need them in the type IList<T> so they will cast into my LazyList<T> (at least that's where i think i am going wrong).
I get this exception:
System.ArgumentException: Object of type 'System.Collections.Generic.List`1[ContactDetail]' cannot be converted to type 'LazyList`1[ContactDetail]'..
When i try using List<ContactDetail> in my Contact type (as you can see commented above) it seems to work. But i dont want to use List<T>'s. I even tried having my LazyList<T> inheriting from List<T> which seemed to execute but passing the List<T>'s internal T[] to my implementation was a nightmare and i simply don't want the bloat of List<T> anywhere in my model.
I also tried some other json libraries to no avail (it's possible i may not be using these to their full potential. I more or less replaced the references and attempted to repeat the code quoted at the top of this question. Maybe passing settings params will help??).
I dont know what to try now. Do i go with another deserializer? Do i tweak the deserializing itself? Do i need to change my types to please the deserializer? Do i need to worry more about implicit casting or just implement another interface?
It is not possible to deserialize directly to an interface, as interfaces are simply a contract. The JavaScriptSerializer has to deserialize to some concrete type that implements IList<T>, and the most logical choice is List<T>. You will have to convert the List to a LazyList, which given the code you posted, should be easy enough:
var list = serializer.Deserialize<IList<Contact>>(...);
var lazyList = new LazyList(list);
Unfortunately you will probably need to fix your class, as there is no way for a deserializer to know that it should be of type IList, since List is an implementation of IList.
Since the deserializers at http://json.org have source available you could just modify one to do what you want.
I ended up using the Json.NET lib which has good linq support for custom mapping. This is what my deserializing ended up looking like:
JArray json = JArray.Parse(
(new StreamReader(General.GetEmbeddedFile("Contacts.json")).ReadToEnd()));
IList<Contact> tempContacts = (from c in json
select new Contact
{
ID = (int)c["ID"],
Name = (string)c["Name"],
Details = new LazyList<ContactDetail>(
(
from cd in c["Details"]
select new ContactDetail
{
ID = (int)cd["ID"],
OrderIndex = (int)cd["OrderIndex"],
Name = (string)cd["Name"],
Value = (string)cd["Value"]
}
).AsQueryable()),
Updated = (DateTime)c["Updated"]
}).ToList<Contact>();
return tempContacts;
Related
I'm trying to persist the following class to DynamoDB using the .NET SDK:
public class MyClass
{
public string Id { get; set; }
public string Name { get; set; }
public object Settings { get; set; }
}
The problem is with the Settings property. It can be any type of object, and I do not know in advance what might be assigned to it. When I try to persist it to DynamoDB, I get the following exception:
System.InvalidOperationException: 'Type System.Object is unsupported, it has no supported members'
Both the Document Model and Object Persistence Model methods result in the same exception.
Is there a way to persist these objects in DynamoDB? Other databases like MongoDB and Azure DocumentDB will do this without any issue, and they can be deserialized to either the proper type with a discriminator, or as a dynamic JSON object.
You can use the general approach documented here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBContext.ArbitraryDataMapping.html
Here's my implementation for any arbitrary object:
public class DataConverter : IPropertyConverter
{
public object FromEntry(DynamoDBEntry entry)
{
var primitive = entry as Primitive;
if (primitive == null || !(primitive.Value is String) || string.IsNullOrEmpty((string)primitive.Value))
throw new ArgumentOutOfRangeException();
object ret = JsonConvert.DeserializeObject(primitive.Value as string);
return ret;
}
public DynamoDBEntry ToEntry(object value)
{
var jsonString = JsonConvert.SerializeObject(value);
DynamoDBEntry ret = new Primitive(jsonString);
return ret;
}
}
Then annotate your property like this:
[DynamoDBProperty(typeof(DataConverter))]
public object data { get; set; }
Little improvement to the previous answer: make converter generic so that you can deserialize to the correct type, like this:
public class SerializeConverter<T> : IPropertyConverter
{
public object FromEntry(DynamoDBEntry entry)
{
var primitive = entry as Primitive;
if (primitive is not { Value: string value } || string.IsNullOrEmpty(value))
throw new ArgumentException("Data has no value", nameof(entry));
return JsonConvert.DeserializeObject<T>(value);
}
public DynamoDBEntry ToEntry(object value) =>
new Primitive(JsonConvert.SerializeObject(value));
}
Usage:
[DynamoDBProperty(typeof(SerializeConverter<YourType>))]
public YourType data{ get; set; }
I struggled to find a good solution for interacting with thoroughly unstructured data, then eventually realized that the DynamoDBContext really isn't designed for that.
For anyone else who gets to this point, my advice is to drop to a lower abstraction level and use the AmazonDynamoDBClient directly with Dictionary<string, AttributeValue> objects.
How would I deserialize YAML to a immutable data structure?
e.g. I have this YAML:
Value: SomeString
Number: 99
And this data structure:
public class MyData
{
public MyData(string value, int number)
{
Value = value;
Number = number;
}
public string Value { get; }
public int Number { get; }
}
For this I'd to use the constructor. So somehow I'd need to first retrieve a Dictionary<string, object> parsed from the YAML respecting my class (so 99 would be int, not string), then scan my type for an appropriate constructor,
Although the question doesn't mention it, I'm assuming you are using YamlDotNet (or SharpYaml which is a fork of YamlDotNet)
YamlDotNet doesnt support deserializing into classes that do not have a default constructor - but one option to achieve what you want is to deserialize into an intermediate Builder type that is mutable which can produce the final type.
e.g.
public class MyDataBuilder
{
public string Value { get; set; }
public int Number { get; set; }
public MyData Build() => new MyData(Value, Number);
}
And then use something like:
deserializer.Deserialize<MyDataBuilder>(yaml).Build();
You would end up having to create a parallel set of builders for your whole model however, e.g. if MyData had a third parameter of type MyOtherData (I've changed the example to use records instead of classes to make it concise):
public record MyOtherData(string OtherValue);
public record MyData(string Value, int Number, MyOtherData otherData);
In which case we would need another Builder:
public class MyOtherDataBuilder
{
public string OtherValue { get; set; }
}
And MyDataBuilder would look like:
public class MyDataBuilder
{
public string Value { get; set; }
public int Number { get; set; }
public MyOtherDataBuilder MyOtherData { get; set; }
public MyData Build() => new MyData(Value, Number, MyOtherData.Build());
}
It's an old but surprisingly relevant question. Now, with records in C#, immutable collections in .net, lack of ability to deserialize immutable data is a blocker - there is no way we need to change all our data types just to be able to deserialize. One practical workaround that I found - is to convert yaml to json first, then deal with json your preferred way - System.Text.Json, Newtonsoft, etc.
Here is how to do is easiest way:
static string ConvertToJson(string yaml) {
object DeserializeYaml() =>
new DeserializerBuilder()
.Build()
.Deserialize(new StringReader(yaml))
?? throw new InvalidOperationException("Cannot deserialize yaml string:" + Environment.NewLine + yaml);
string SerializeYamlObjectToJson(object yamlObject) =>
new SerializerBuilder()
.JsonCompatible()
.Build()
.Serialize(yamlObject);
return SerializeYamlObjectToJson(DeserializeYaml());
}
The only disadvantage, potentially big, is performance. I feel, however, that it's rarely an important requirement for yaml.
use the FormatterServices.GetUninitializedObject API (this will NOT invoke any constructors at all) and then use reflection to set fields.
Code example:
var instance = FormatterServices.GetUninitializedObject(typeof(MyData));
var flags = BindingFlags.NonPublic | BindingFlags.Instance;
var type = typeof(MyData);
var stringField = type.GetField("_value", flags);
stringField.SetValue(instance, "SomeString");
var numberField = type.GetField("_number", flags);
numberField.SetValue(instance, 99);
MyData data = (MyData)instance;
I want to create a key value table in my database along the lines of
public class KeyValue {
public string Id { get; set; }
public dynamic Value {get; set; }
}
Using a slightly modified SqlProvider I have no problems getting CreateTable<KeyValue>() to generate varchar(1024) Id, varchar(max) Value.
I have no issues saving objects to it. The problem is when I load the objects
var content = dbConn.GetById<KeyValue>("about");
content.Value at this point is a string.
Looking at the database record, the text for value does not appear to store any type information.
Is there really anything I can do better other than manually invoking ServiceStack.Text and call deserialize with the appropriate type information?
I do not need absolute dynamic, my actual use case is for polymorphism with a base class instead of dynamic. So I don't really care what type Value is whether it's the base class, dynamic, object, etc. Regardless other than using the class
public class KeyValue {
public string Id { get; set; }
public MySpecificChildType Value {get; set; }
}
I haven't been able to get anything other than a string back for Value. Can I tell OrmLite to serialize the type information to be able to correctly deserialize my objects or do I just have to do it manually?
Edit: some further information. OrmLite is using the Jsv serializer defined by ServiceStack.Text.TypeSerializer and is in no way pluggable in the BSD version. If I add a Type property to my KeyValue class with the dynamic Value I can do
var value = content.Value as string;
MySpecificChildType strongType =
TypeSerializer.DeserializeFromString(content, content.Type);
I just really want a better way to do this, I really don't like an object of 1 type going into the db coming back out with a different type (string).
I haven't worked much with the JsvSerializer but with the JsonSerializer you can achieve this (in a few different ways) and as of ServiceStack 4.0.11 you can opt to use the JsonSerializer instead, see https://github.com/ServiceStack/ServiceStack/blob/master/release-notes.md#v4011-release-notes.
Example
public abstract class BaseClass {
//Used for second example of custom type lookup
public abstract string Type { get; set; }
}
public class ChildA : BaseClass {
//Used for second example of custom type lookup
public override string Type { get; set; }
public string PropA { get; set; }
}
And then in your init/bootstrap class you can configure the serializer to emit the type information needed for proper deserialization:
public class Bootstrapper {
public void Init() {
ServiceStack.Text.JsConfig.ExcludeTypeInfo = false;
ServiceStack.Text.JsConfig.IncludeTypeInfo = true;
}
}
If you wish to use something other that the default "__type" attribute that ServiceStack uses (if you for example want to have a friendly name identifying the type rather then namespace/assembly) you can also configure your own custom type lookup as such
public class Bootstrapper {
public void Init() {
ServiceStack.Text.JsConfig.ExcludeTypeInfo = false;
ServiceStack.Text.JsConfig.IncludeTypeInfo = true;
ServiceStack.Text.JsConfig.TypeAttr = "type";
ServiceStack.Text.JsConfig.TypeFinder = type =>
{
if ("CustomTypeName".Equals(type, StringComparison.OrdinalIgnoreCase))
{
return typeof(ChildA);
}
return typeof(BaseClass);
}
}
}
I am currently using a list to handle a JSON string which works fine for one instance of this, as can be seen below. What I want to do is make these methods that handle the conversion completely generic so I can use them for multiple JSON strings.
This is a snippet of my current code as it stands.
public class GetPerson
{
public string fooName { get; set; }
public string fooAddress { get; set; }
public string fooPosition { get; set; }
}
public class GetPosition
{
public string fooTitle { get; set; }
public string fooDepartment { get; set; }
public string fooSalary { get; set; }
}
private static List<GetPerson> ConvertToList(string jsonString)
{
List< listJson = new List<JsonObject>();
listJson = (List<GetPerson>)JsonConvert.DeserializeObject<List<GetPerson>>(jsonString);
return listJson;
}
This is just a quick sample but the List<GetPerson> is what I need to be generic so it can be reused, because as it stands the GetPosition will obviously not work with this, as I would want to be able to iterate through my code changing the type accordingly.
Is there a way I can assign a variable as a type? I saw another question about this but it didn't go into detail. Or is there another way that this could be achieved?
Thanks in advance.
Very Simple. You just have to make ConvertToList() generic and pass the desired class as Type Paramter in ConvertToList()
private static List<T> ConvertToList<T>(string jsonString)
{
var listJson = new List<JsonObject>();
listJson = (List<T>)JsonConvert.DeserializeObject<List<T>>(jsonString);
return listJson;
}
var personList = ConvertToList<GetPerson>(jsonString);
var positionList = ConvertToList<GetPosition>(jsonString);
You can use Generics to help make the ConvertToList function reusable for different types
private static List<T> ConvertToList<T>(string jsonString)
{
return (List<T>)JsonConverty.DeserializeObject<List<T>>(jsonString();
}
You can now call it using both GetPerson and GetPosition as the generic type.
var listOfPeople = ConvertToList<GetPerson>(personJson);
var listOfPositions = ConvertToList<GetPosition>(positionJson);
You can read more about Generics on MSDN.
Also, if all that you want to do is to [de]serialize JSON, you might want to consider a third-party library for that like JSON.net, Jil or ServiceStack.Text, all of which have built in functions to do what you are trying to do.
Question
Is there a way to define a method only once in C# (in a helper class or something) not knowing which type is given to be returned?
Long explanation
I get the following error:
Unable to cast object of type
System.Data.Objects.ObjectQuery1[WerkStageNu.Vacancies]'
to type
'System.Linq.IQueryable1[WerkStageNu.Models.IFilteredEntities]'.
I have a ListingsController which does a Search through my current Vacancies in the database:
public ActionResult Search(int? page, string branchid, string hoursago, string jobtypeid, string educationlevelid, string careerlevelid)
{
string searchResult = string.Empty;
const int pageSize = 10;
IQueryable<IFilteredEntities> selectedListings = (IQueryable<IFilteredEntities>)Repository.Instance._entities.Vacancies.AsQueryable();
Dictionary<string, string> filterParams = new Dictionary<string, string>() {
{"branchid", branchid}, {"hoursago", hoursago}, {"jobtypeid", jobtypeid}, {"educationlevelid", educationlevelid}, {"careerlevelid", careerlevelid}};
selectedListings = FilterByIDHelper.Filter(selectedListings, filterParams);
var paginatedDinners = new PaginatedList<Vacancies>(((IQueryable<Vacancies>)selectedListings).ToList(), page ?? 0, pageSize);
return View("Index", paginatedDinners);
}
Now, this search is just for Vacancies. But one can imagine we have searches all over the place all in general the same routine so I want to call the same method getting back different types. For this case I have made an Interface , IFilteredEntities. In my partial class Vacancies (partial class, class Vacancies is generated by my DB entity framework) I just do:
public partial class Vacancies : IFilteredEntities
And of course implement the methods in the Interface which are not implemented by Default. In my Interface I have:
interface IFilteredEntities
{
string EducationLevelID { get; set; }
string BrancheID { get; set; }
string CareerLevelID { get; set; }
string JobTypeID { get; set; }
Branches Branches { get; set; }
DateTime? DateOfCreation { get; set; }
CareerLevels CareerLevels { get; set; }
JobTypes JobTypes { get; set; }
EducationLevels EducationLevels { get; set; }
}
For convenience I have uploaded the two helper classes PaginatedList and FilterCriteriaHelper here and here.
Now, the method which would do the actual filtering is placed inside another helper class: FilterByIDHelper.cs.
public static IQueryable<IFilteredEntities> Filter(IQueryable<IFilteredEntities> collection, Dictionary<string, string> filterParams)
{
if (filterParams.ContainsKey("branchid")) collection = FilterByBranchId(collection, filterParams["branchid"]);
if (filterParams.ContainsKey("hoursago")) collection = FilterByHoursAgo(collection, filterParams["hoursago"]);
if (filterParams.ContainsKey("jobtypeid")) collection = FilterByJobTypeId(collection, filterParams["jobtypeid"]);
if (filterParams.ContainsKey("educationlevelid")) collection = FilterByEducationLevelId(collection, filterParams["educationlevelid"]);
if (filterParams.ContainsKey("careerlevelid")) collection = FilterByCareerLevelId(collection, filterParams["careerlevelid"]);
return collection;
}
public static IQueryable<IFilteredEntities> Filter(IQueryable<IFilteredEntities> collection, Dictionary<string, string> filterParams)
{
if (filterParams.ContainsKey("branchid")) collection = FilterByBranchId(collection, filterParams["branchid"]);
if (filterParams.ContainsKey("hoursago")) collection = FilterByHoursAgo(collection, filterParams["hoursago"]);
if (filterParams.ContainsKey("jobtypeid")) collection = FilterByJobTypeId(collection, filterParams["jobtypeid"]);
if (filterParams.ContainsKey("educationlevelid")) collection = FilterByEducationLevelId(collection, filterParams["educationlevelid"]);
if (filterParams.ContainsKey("careerlevelid")) collection = FilterByCareerLevelId(collection, filterParams["careerlevelid"]);
return collection;
}
For convenience here is a picture of a part of my solution explorer:
Solution Explorer http://www.bastijn.nl/zooi/solutionexplorer.png
In short:
What I try to do is instead of calling like:
selectedListings = Repository.Instance._entities.Vacancies.AsQueryable();
Dictionary<string, string> filterParams = new Dictionary<string, string>() {
{"branchid", branchid}, {"hoursago", hoursago}, {"jobtypeid", jobtypeid}, {"educationlevelid", educationlevelid}, {"careerlevelid", careerlevelid}};
selectedListings = FilterByIDHelper.Filter(selectedListings, filterParams);
var paginatedDinners = new PaginatedList<Vacancies>(selectedListings.ToList(), page ?? 0, pageSize);
return View("Index", paginatedDinners);
Call the variant shown up, using an Interface so I only have to define te "Filter" method once instead of for all classes / models. Now Notice that all of this DOES compile! The problem is that I get the following error:
Unable to cast object of type 'System.Data.Objects.ObjectQuery`1[WerkStageNu.Vacancies]' to type 'System.Linq.IQueryable`1[WerkStageNu.Models.IFilteredEntities]'.
I hope I have not forgotten any information but I'm already staring at this code for some while. Might forget a relation or something, just ask for it if I did :).
-----------------------------------------------------
EDIT AFTER COMMENTS
-----------------------------------------------------
O crap, nevermind this part, I forgot to as AsEnumerable, was still using AsQueryable.
It looks to me like this is a covariance vs. contravariance issue. Basically, an IQueryable<Vacancies> is not a sub-type of IQueryable<IFilteredEntities>, even though Vacancies implements IFilteredEntities. Thus, the line with the cast is causing a runtime error. So rather than doing the cast try this instead:
IEnumerable<IFilteredEntities> selectedListings =
Repository.Instance._entities.Vacancies.AsQueryable()
.OfType<IFilteredEntities>();
What this will do is project each element of the collection to an IFilteredEntities type.
Another option is to rewrite your filter methods so they use generics, like this:
public static IEnumerable<T> Filter<T>(
IEnumerable<T> collection, IDictionary<string, string> filterParams)
where T : IFilteredEntities
{
...
}
This would then allow you to pass in a collection containing any type that derives from IFilteredEntities and get back a collection of the same type. And if you're using C# 3, you don't even have to specify the type parameter if it can be implicitly determined by the compiler.