We Keep Coding

How to add a known type to a List<T>?

The overall goal here is like this: We have a lot of CSV files of various names and format stored in Azure blob storage. We need to convert them to lists.
I have an interface:
public interface IGpasData
{
List<T> ConvertToList<T>(StreamReader reader);
}
And then here's an example of a class that Implements it:
public class GpasTableOfContent : IGpasData
{
public string TocProp0 { get; set; }
public string TocProp1 { get; set; }
public string TocProp2 { get; set; }
public List<T> ConvertToList<T>(StreamReader reader)
{
List<T> dataList = new List<T>();
while (!reader.EndOfStream)
{
var lineItem = reader.ReadLine();
GpasTableOfContent dataItem = new GpasTableOfContent
{
TocProp0 = lineItem.Split(',')[0],
TocProp1 = lineItem.Split(',')[1],
Type = lineItem.Split(',')[2]
};
dataList.Add(dataItem);
}
return dataList;
}
}
To keep going with the example of the class above, there is a file called ToC.csv. In a class that is designed to convert THAT file into a list, I make this call:
List<GpasTableOfContent> gpasToCList = ConvertCloudFileToList<GpasTableOfContent>("ToC.csv", "MyModel");
Some other possible examples:
List<GpasFoo> gpasFooList = ConvertCloudFileToList<GpasFoo>("foo.csv", "MyModel");
List<GpasBar> gpasBarList = ConvertCloudFileToList<GpasBar>("bar.csv", "MyModel");
Here's ConvertCloudFileToList:
private List<T> ConvertCloudFileToList<T>(string fileName, string modelName)
{
// Get the .csv file from the InProgress Directory
string filePath = $"{modelName}/{fileName}";
CloudFile cloudFile = _inProgressDir.GetFileReference(filePath);
List<T> dataList = new List<T>();
// Does the file exist?
if (!cloudFile.Exists())
return dataList;
using (StreamReader reader = new StreamReader(cloudFile.OpenRead()))
{
IGpasData gpasData = (IGpasData)Activator.CreateInstance<T>();
dataList = gpasData.ConvertToList<T>(reader);
}
return dataList;
}
And that brings us back to ConvertToList. The problem is here:
dataList.Add(dataItem);
Can not convert 'GpasFoo' to 'T'
Not sure how to work around this.

Any object that is an IGpasData is expected to be able to produce a List of any given type when provided with a StreamReader. GpasTableOfContent does not fulfill this requirement, it can only produce a list of its own type.
However it doesn't seem reasonable to have one type of GpasData be responsible for converting everything so I'd suggest moving the Type argument from the ConvertToList method into the interface. This way subclasses will only be responsible for converting lists of a particular type.
public interface IGpasData<T>
{
List<T> ConvertToList(StreamReader reader);
}
public class GpasTableOfContent : IGpasData<GpasTableOfContent>
{
//...
public List<GpasTableOfContent> ConvertToList(StreamReader reader)
{
//...
}
}
On a side note, creating an empty table of contents and then using it to read from a stream and produce a list of the real table of contents seems very clunky to me. In my opinion, the behaviour of creating these content objects should be moved into its own class.

You can't do what you want here without providing some additional logic. The problem is that you have a string from reading the CSV file, and you want to convert it to a T, but there is no rule for converting a string into any arbitrary type.
One approach would be to change the method to also take a delegate Func that is used to convert each line into a T. Then if, for example, your data is guaranteed to consist of doubles, you could pass t => Double.Parse(t) for that argument. Of course, this approach requires that you change the signature of the interface method you are implementing.
If you are not able to change the signature of the interface method, then all I can suggest is trying to handle a pre-defined set of types and throwing an exception for other types.

As other have pointed out, this design is flawed:
public interface IGpasData
{
List<T> ConvertToList<T>(StreamReader reader);
}
This contract says that an IGpasData should only know how deserialize anything. It doesn't make sense.
An IGpasData should know how to deserialize itself, and for this we would need a self-referencing interface:
public interface IGpasData<T> where T : IGpasData<T>
{
List<T> ConvertToList(StreamReader reader);
}
public class GpasBar: IGpasData<GpasBar>
{
public string MyPropertyA { get; set; }
public int MyPropertyB { get; set; }
public List<GpasBar> ConvertToList(StreamReader reader)
{
var results = new List<GpasBar>();
while (!reader.EndOfStream)
{
var values = reader.ReadLine().Split(',');
results.Add(new GpasBar()
{
PropertyA = values[0],
PropertyB = int.Parse(values[1]),
});
}
return results;
}
}
Or, an IGpasData should know how to populate itself from an array of values:
public interface IGpasData
{
void Populate(string[] values);
}
public class GpasBar
{
public string MyPropertyA { get; set; }
public int MyPropertyB { get; set; }
public void Populate(string[] values)
{
MyPropertyA = values[0];
MyPropertyB = int.Parse(values[1]);
}
}
public static List<T> ConvertCloudFileToList<T>(string fileName, string modelName)
where T : IGpasData, new()
{
// ...
using (StreamReader reader = new StreamReader(cloudFile.OpenRead()))
{
var results = new List<T>();
while (!reader.EndOfStream)
{
var item = new T();
item.Populate(reader.ReadLine().Split(','));
results.Add(item);
}
return results;
}
}
Using this 2nd approach, you can avoid duplicating the part about StreamReader and read lines.

Method with generic return type but not generic input. Is this possible?

Suppose we have a NodeData class:
public class NodeData<T>
{
public string Name;
public T Value;
public NodeData(string name, T value)
{
this.Name = name;
this.Value = value;
}
}
And a base Node class and child classes that have several properties with type NodaData:
public class Node
{
public List<NodeData<T>> listOutputs<T>()
{
var fieldInfos = GetType().GetFields();
var list = new List<NodeData<T>>();
foreach (var item in fieldInfos)
{
Type t = item.FieldType;
string name = item.Name;
if (t == typeof(NodeData<T>))
{
var output = new NodeData<T>(name, default(T));
list.Add(output);
}
}
return list;
}
}
public class TestNode : Node {
public NodeData<int> data;
public NodeData<double> data2;
public NodeData<double> data3;
public TestNode ()
{
data = new NodeData<int>("test", 111);
data2 = new NodeData<double>("test", 113);
}
}
As you can see there is a method which lists all outputs with type T in the Node class So I can find what are the output fields of the child class in runtime:
TestNode node = new TestNode ();
var list = node.listOutputs<int>(); // this returns data
But I need to know how to use this method to list all NodeOutputs of any type T. In this example int and double. Do I need to add a method with this signature public List<NodeData<T>> listOutputs() // should return all properties data, data2, data3. Is it possible to have method like this? return type is generic but there is no type argument for method.

Even after your edit(s) it is not entirely clear what you are trying to achieve but here are my assumptions:
-You want to have some kind of Node object that acts as a container for different types of NodeData elements.
-You want to be able to return one list from this Node object that contains all NodeData elements stored in the Node container, regardless of the NodeData objects' type.
Instead of returning a List> object from the listOutputs methods, just return the non-generic version of the List object. Then you don't have to deal with T in the method call.
The logic that loops through the objects in the non-generic list can then examine the type to process the contained NodeData objects correctly.
Important note: My proposed solution is by no means pretty but I think it answers the question. In my opinion something is already seriously flawed from an OO point of view in the presented code (e.g. use of reflection) and a better solution would have to start by changing the underlying data structures. But that can only be done if we have more information how this is to be used, e.g. what kind of logic consumes the returned list.

You can create a base interface that will be used to return the generic data.
public interface INodeData
{
string Name { get; }
}
public class NodeData<T> : INodeData
{
public string Name { get; private set; }
public T Value { get; private set; }
public NodeData(string name, T value)
{
this.Name = name;
this.Value = value;
}
}
I modified the function to return a list of the interface. Doing this you won't depend on T.
public class Node
{
public List<INodeData> listOutputs()
{
var fieldInfos = GetType().GetFields();
var list = new List<INodeData>();
foreach (var item in fieldInfos)
{
INodeData data = GetType().GetField(item.Name).GetValue(this) as INodeData;
list.Add(data);
}
return list;
}
}
If you test the method, it should return the fields in a list. To work with a specific type, you can make use of is before using the type you search for.
public class TestNode : Node
{
public NodeData<int> data;
public NodeData<double> data2;
public NodeData<double> data3;
public TestNode()
{
data = new NodeData<int>("test", 111);
data2 = new NodeData<double>("test", 113);
}
}
private static void Main(string[] args)
{
TestNode node = new TestNode();
var list = node.listOutputs(); // this returns data
}

This may well be an XY problem, in that you probably want to rethink how you are designing your classes because using reflection in this way doesn't seem right. But give the problem you've presented, I'd tackle it like this:
public abstract class NodeDataBase
{
public string Name { get; set; }
public NodeData(string name)
{
this.Name = name;
}
// this isn't actually needed, but might be helpful
public abstract object GetValue();
}
public class NodeData<T> : NodeDataBase
{
public T Value { get; set; }
public NodeData(string name, T value) : base(name)
{
this.Value = value;
}
public override object GetValue()
{
return Value;
}
}
And now your method signature would be:
public List<NodeDataBase> listOutputs()
And with the list returned, you can use the GetValue method to get the actual values without needing to cast to the right generic type to be able to get at the Value property.
You could also just have a return type of List<object>, but then you'll have to cast each member of that list to the right generic type before you can access it's properties.
You can also avoid that nasty reflection code, instead of having data, data1, and data2, you could simply do this in your Node class:
public class Node
{
public List<NodeDataBase> Data { get; protected set; }
public Node()
{
Data = new List<NodeDataBase>();
}
}
And now you don't even need your listOutputs method because you can just get the list from the node (unless you actually wanted a copy, but that's fairly trivial to implement).
And you TestNode would be just:
public class TestNode : Node {
public TestNode ()
{
Data.Add(new NodeData<int>("test", 111));
Data.Add(new NodeData<double>("test", 113));
}
}

Serialize list of interface types with ServiceStack.Text

I'm looking at ways to introduce something other than BinaryFormatter serialization into my app to eventually work with Redis. ServiceStack JSON is what I would like to use, but can it do what I need with interfaces?
It can serialize (by inserting custom __type attribute)
public IAsset Content;
but not
public List<IAsset> Contents;
- the list comes up empty in serialized data. Is there any way to do this - serialize a list of interface types?
The app is big and old and the shape of objects it uses is probably not going to be allowed to change.
Thanks

Quoting from http://www.servicestack.net/docs/framework/release-notes
You probably don't have to do much :)
The JSON and JSV Text serializers now support serializing and
deserializing DTOs with Interface / Abstract or object types. Amongst
other things, this allows you to have an IInterface property which
when serialized will include its concrete type information in a __type
property field (similar to other JSON serializers) which when
serialized populates an instance of that concrete type (provided it
exists).
[...]
Note: This feature is automatically added to all
Abstract/Interface/Object types, i.e. you don't need to include any
[KnownType] attributes to take advantage of it.
By not much:
public interface IAsset
{
string Bling { get; set; }
}
public class AAsset : IAsset
{
public string Bling { get; set; }
public override string ToString()
{
return "A" + Bling;
}
}
public class BAsset : IAsset
{
public string Bling { get; set; }
public override string ToString()
{
return "B" + Bling;
}
}
public class AssetBag
{
[JsonProperty(TypeNameHandling = TypeNameHandling.None)]
public List<IAsset> Assets { get; set; }
}
class Program
{
static void Main(string[] args)
{
try
{
var bag = new AssetBag
{
Assets = new List<IAsset> {new AAsset {Bling = "Oho"}, new BAsset() {Bling = "Aha"}}
};
string json = JsonConvert.SerializeObject(bag, new JsonSerializerSettings()
{
TypeNameHandling = TypeNameHandling.Auto
});
var anotherBag = JsonConvert.DeserializeObject<AssetBag>(json, new JsonSerializerSettings()
{
TypeNameHandling = TypeNameHandling.Auto
});

How to Serialize ICollection<T> as List<T> to Xml?

I am trying to dynamically serialize a List to Xml.
I am able to do so, as long I do not have a ICollection as a property of T.
I would like to dynamically overwrite the ICollection type into List before I write it to Xml.
This is what I have so far.
List<-XmlElementAttribute-> attrToConvertList = new List<-XmlElementAttribute->();
foreach (var propertyInfo in typeof(T).GetProperties())
{
if (propertyInfo.PropertyType.Name == "ICollection`1")
{
XmlElementAttribute attrToConvert = new XmlElementAttribute();
attrToConvert.ElementName = propertyInfo.Name;
attrToConvert.Type = typeof(List<>);
attrToConvert.Type = attrToConvert.Type.MakeGenericType(propertyInfo.PropertyType.GetGenericArguments()[0]);
attrToConvertList.Add(attrToConvert);
}
}
XmlAttributeOverrides overrides = new XmlAttributeOverrides();
XmlAttributes attributesToConvert = new XmlAttributes();
foreach (var xmlElementAttribute in attrToConvertList)
attributesToConvert.XmlElements.Add(xmlElementAttribute);
overrides.Add(typeof(T), attributesToConvert);
XmlSerializer serializer = new XmlSerializer(typeof(List<T>), overrides);
I get the error that I cannot serialize the type ICollection because it is an interface.
I was under the impression that what I was doing with the XmlAttributeOverrides was supposed to overwrite the ICollection to the type List.

XML serialization doesn't handle interfaces, and apparently XmlAttributeOverride doesn't allow you to bypass that behavior. You can change the type of your property, or make a type, for serialization only, where the property is a List<T>.
Example:
class RealClass
{
ICollection<int> SomeInts { get; set; }
}
class MySerializationClass
{
private readonly RealClass _wrappedObject;
public SerializationClass() : this(new RealClass()) { }
public SerializationClass(RealClass wrappedObject)
{
_wrappedObject = wrappedObject;
}
public List<T> SomeInts
{
get { return new List<T>(_wrappedObject.SomeInts); }
set { _wrappedObject.SomeInts = value; }
}
}
You could also do this with explicit interface member implementation, and use the interface in most of your code:
interface IHaveSomeInts
{
ICollection<int> SomeInts { get; set; }
}
class TheClass : IHaveSomeInts
{
public List<T> SomeInts { get; set; }
ICollection<T> IHaveSomeInts.SomeInts
{
get { return SomeInts; }
set { SomeInts = new List<T>(value); }
}
}
When assigning an ICollection<T> to an IList<T>, I would probably use as to see if I can just cast the object rather than creating a new one, to avoid creating lists needlessly.

I solved my original issue by using Newton.Json to serialize the object.

How to revert back to 'default' XML serialization when implementing IXmlSerializable in a base class?

I'm trying to serialize a class that inherits from a base class that implements IXmlSerializable.
The base class, called PropertyBag is a class that allows dynamic properties (credits to Marc Gravell).
I implemented IXmlSerializable so that the dynamic properties (stored in a Dictionary) are written as normal xml elements.
e.g.
When serializing a class Person with a public property (non dynamic) Name and a dynamic property Age, I would like for it to generate the following XML:
<Person>
<Name>Tim</Name>
<DynamicProperties>
<Country>
<string>USA</string>
</Country>
</DynamicProperties>
<Person>
I can get the part to work with the following implementation of WriteXml in the base PropertyBag class:
public void WriteXml(System.Xml.XmlWriter writer)
{
writer.WriteStartElement("DynamicProperties");
// serialize every dynamic property and add it to the parent writer
foreach (KeyValuePair<string, object> kvp in properties)
{
writer.WriteStartElement(kvp.Key);
StringBuilder itemXml = new StringBuilder();
using (XmlWriter itemWriter = XmlWriter.Create(itemXml))
{
// serialize the item
XmlSerializer xmlSer = new XmlSerializer(kvp.Value.GetType());
xmlSer.Serialize(itemWriter, kvp.Value);
// read in the serialized xml
XmlDocument doc = new XmlDocument();
doc.LoadXml(itemXml.ToString());
// write to modified content to the parent writer
writer.WriteRaw(doc.DocumentElement.OuterXml);
}
writer.WriteEndElement();
}
writer.WriteEndElement();
}
However, when serializing the Person class, it no longer serializes the normal (non dynamic) properties unless I overwrite the WriteXml method in Person (which I do not want to do). Is there any way that in the base class I can automatically add the static properties? I know I can do this manually using reflection, but I was wondering if there is some built-in functionality in the .Net Framework?

I've spent quite a bit of time with XmlSerializer (and various other serialization APIs), and I'm pretty sure that simply: you can't. Implementing IXmlSerializable is all or nothing.
The closest I can think of is to cheat and move all the fixed properties to a sub-object; this would give you slightly different xml - something like:
<FixedProperties>
<Name>Tim</Name>
</FixedProperties>
<DynamicProperties>
<Country>
<string>USA</string>
</Country>
</DynamicProperties>
but I expect it would work. You would have pass-thru properties on your base object:
[Browsable(false), EditorBrowsable(EditorBrowsableState.Never)]
public FixedProperties FixedProps {get;set;}
public string Name {
get {return FixedProps.Name;}
set {FixedProps.Name = value;}
}
Make sense? You could also mark Name as [XmlIgnore], but it seems pretty redundant. In your bespoke serialize method you'd use new XmlSerializer(typeof(FixedProperties))
Edit: Here's a working "serialize" example:
using System;
using System.ComponentModel;
using System.Xml.Serialization;
static class Program
{
static void Main()
{
MyType obj = new MyType { Name = "Fred" };
var ser = new XmlSerializer(obj.GetType());
ser.Serialize(Console.Out, obj);
}
}
public class MyType : IXmlSerializable
{
public MyType()
{
FixedProperties = new MyTypeFixedProperties();
}
[Browsable(false), EditorBrowsable(EditorBrowsableState.Never)]
public MyTypeFixedProperties FixedProperties { get; set; }
[XmlIgnore]
public string Name
{
get { return FixedProperties.Name; }
set { FixedProperties.Name = value; }
}
System.Xml.Schema.XmlSchema IXmlSerializable.GetSchema()
{
return null;
}
void IXmlSerializable.ReadXml(System.Xml.XmlReader reader)
{
throw new System.NotImplementedException();
}
void IXmlSerializable.WriteXml(System.Xml.XmlWriter writer)
{
writer.WriteStartElement("DynamicProperties");
writer.WriteElementString("Foo", "Bar");
writer.WriteEndElement();
fixedPropsSerializer.Serialize(writer, FixedProperties);
}
static readonly XmlSerializer fixedPropsSerializer
= new XmlSerializer(typeof(MyTypeFixedProperties));
}
[XmlRoot("FixedProperties")]
public class MyTypeFixedProperties
{
public string Name { get; set; }
}

Marc, your answer on putting the FixedProperties in a seperate collection got me thinking that instead of inheriting from PropertyBag, I should create a property of that type.
So I created a PropertyBagWrapper class that my Person class inherits from and it works.
[Serializable]
[TypeDescriptionProvider(typeof(PropertyBagDescriptionProvider))]
public abstract class PropertyBagWrapper
{
[Browsable(false), EditorBrowsable(EditorBrowsableState.Never)]
public PropertyBag DynamicProperties { get; set; }
public object this[string name]
{
get { return DynamicProperties[name]; }
set { DynamicProperties[name] = value; }
}
protected PropertyBagWrapper()
{
DynamicProperties = new PropertyBag(this.GetType());
}
}
[Serializable]
public class Person : PropertyBagWrapper
{
[Browsable(true)]
public string Name { get; set; }
}
I won't repeat all the code for the PropertyBag and the custom classes needed for ICustomTypeDescriptor implementation, you can find that here.
I did move the TypeDescriptionProvider attribute from the PropertyBag class to the PropertyBagWrapper class.
The PropertyBag class still has the same implementation for WriteXml() method as posted in the question.