We Keep Coding

How to add a known type to a List<T>?

The overall goal here is like this: We have a lot of CSV files of various names and format stored in Azure blob storage. We need to convert them to lists.
I have an interface:
public interface IGpasData
{
List<T> ConvertToList<T>(StreamReader reader);
}
And then here's an example of a class that Implements it:
public class GpasTableOfContent : IGpasData
{
public string TocProp0 { get; set; }
public string TocProp1 { get; set; }
public string TocProp2 { get; set; }
public List<T> ConvertToList<T>(StreamReader reader)
{
List<T> dataList = new List<T>();
while (!reader.EndOfStream)
{
var lineItem = reader.ReadLine();
GpasTableOfContent dataItem = new GpasTableOfContent
{
TocProp0 = lineItem.Split(',')[0],
TocProp1 = lineItem.Split(',')[1],
Type = lineItem.Split(',')[2]
};
dataList.Add(dataItem);
}
return dataList;
}
}
To keep going with the example of the class above, there is a file called ToC.csv. In a class that is designed to convert THAT file into a list, I make this call:
List<GpasTableOfContent> gpasToCList = ConvertCloudFileToList<GpasTableOfContent>("ToC.csv", "MyModel");
Some other possible examples:
List<GpasFoo> gpasFooList = ConvertCloudFileToList<GpasFoo>("foo.csv", "MyModel");
List<GpasBar> gpasBarList = ConvertCloudFileToList<GpasBar>("bar.csv", "MyModel");
Here's ConvertCloudFileToList:
private List<T> ConvertCloudFileToList<T>(string fileName, string modelName)
{
// Get the .csv file from the InProgress Directory
string filePath = $"{modelName}/{fileName}";
CloudFile cloudFile = _inProgressDir.GetFileReference(filePath);
List<T> dataList = new List<T>();
// Does the file exist?
if (!cloudFile.Exists())
return dataList;
using (StreamReader reader = new StreamReader(cloudFile.OpenRead()))
{
IGpasData gpasData = (IGpasData)Activator.CreateInstance<T>();
dataList = gpasData.ConvertToList<T>(reader);
}
return dataList;
}
And that brings us back to ConvertToList. The problem is here:
dataList.Add(dataItem);
Can not convert 'GpasFoo' to 'T'
Not sure how to work around this.

Any object that is an IGpasData is expected to be able to produce a List of any given type when provided with a StreamReader. GpasTableOfContent does not fulfill this requirement, it can only produce a list of its own type.
However it doesn't seem reasonable to have one type of GpasData be responsible for converting everything so I'd suggest moving the Type argument from the ConvertToList method into the interface. This way subclasses will only be responsible for converting lists of a particular type.
public interface IGpasData<T>
{
List<T> ConvertToList(StreamReader reader);
}
public class GpasTableOfContent : IGpasData<GpasTableOfContent>
{
//...
public List<GpasTableOfContent> ConvertToList(StreamReader reader)
{
//...
}
}
On a side note, creating an empty table of contents and then using it to read from a stream and produce a list of the real table of contents seems very clunky to me. In my opinion, the behaviour of creating these content objects should be moved into its own class.

You can't do what you want here without providing some additional logic. The problem is that you have a string from reading the CSV file, and you want to convert it to a T, but there is no rule for converting a string into any arbitrary type.
One approach would be to change the method to also take a delegate Func that is used to convert each line into a T. Then if, for example, your data is guaranteed to consist of doubles, you could pass t => Double.Parse(t) for that argument. Of course, this approach requires that you change the signature of the interface method you are implementing.
If you are not able to change the signature of the interface method, then all I can suggest is trying to handle a pre-defined set of types and throwing an exception for other types.

As other have pointed out, this design is flawed:
public interface IGpasData
{
List<T> ConvertToList<T>(StreamReader reader);
}
This contract says that an IGpasData should only know how deserialize anything. It doesn't make sense.
An IGpasData should know how to deserialize itself, and for this we would need a self-referencing interface:
public interface IGpasData<T> where T : IGpasData<T>
{
List<T> ConvertToList(StreamReader reader);
}
public class GpasBar: IGpasData<GpasBar>
{
public string MyPropertyA { get; set; }
public int MyPropertyB { get; set; }
public List<GpasBar> ConvertToList(StreamReader reader)
{
var results = new List<GpasBar>();
while (!reader.EndOfStream)
{
var values = reader.ReadLine().Split(',');
results.Add(new GpasBar()
{
PropertyA = values[0],
PropertyB = int.Parse(values[1]),
});
}
return results;
}
}
Or, an IGpasData should know how to populate itself from an array of values:
public interface IGpasData
{
void Populate(string[] values);
}
public class GpasBar
{
public string MyPropertyA { get; set; }
public int MyPropertyB { get; set; }
public void Populate(string[] values)
{
MyPropertyA = values[0];
MyPropertyB = int.Parse(values[1]);
}
}
public static List<T> ConvertCloudFileToList<T>(string fileName, string modelName)
where T : IGpasData, new()
{
// ...
using (StreamReader reader = new StreamReader(cloudFile.OpenRead()))
{
var results = new List<T>();
while (!reader.EndOfStream)
{
var item = new T();
item.Populate(reader.ReadLine().Split(','));
results.Add(item);
}
return results;
}
}
Using this 2nd approach, you can avoid duplicating the part about StreamReader and read lines.

Automapper: List to members

I'd like to map an arbitrary list of abstract types to an arbitrary set of properties, that share the same base type.
Here is some UnitTest code, which currently fails and which I want to success. Could you help me, get a generic solution?
Here are the classes:
public class Source
{
public string Name { get; set; } = "SomeName";
public Dictionary<string, ValueType> SourceList { get; set; } = new Dictionary<string, ValueType>();
}
public interface IDestination
{
string Name { get; set; }
}
public class Destination : IDestination //And many other classes like this, with other properties inherited from ValueType
{
public string Name { get; set; }
public double DoubleValue { get; set; }
public int IntValue { get; set; }
public string SomeOtherProperty { get; set; }
}
And here is the unit test, I'd like to succeed:
[TestMethod]
public void TestMethod1()
{
var source = new Source();
source.SourceList.Add("IntValue", (int) 3);
source.SourceList.Add("DoubleValue", (double) 3.14);
Mapper.Initialize(config =>
{
//Put in some magic code here!!!
});
var destinationAbstract = Mapper.Map<Source, IDestination>(source); //the type of destination is known only at runtime. Therefore Mapping to Interface
var destination = (Destination) destinationAbstract;
Assert.AreEqual(source.Name, destination.Name);
Assert.AreEqual((int)source.SourceList["IntValue"], destination.IntValue);
Assert.AreEqual((double)source.SourceList["DoubleValue"], destination.DoubleValue);
}
Please be aware, that
the number of classes, that inherit from IDestination is only known at runtime
the content of the SourceList may be different for each Source-instance and therefore the properties of the destination class could also change for each class definition
I hope you can help me, because I wasn't able to determine a generic solution with the help of the documentation.
Thanks in advance.

You can map from Dictionary<string, object>(property names to property values) to some class by default, without any extra configuration. The docs and the tests.

After considering Lucians hint and after trying different things with Automapper, I finally found a solution for my initial unit-test:
[TestMethod]
public void TestMethod1()
{
var source = new Source();
source.SourceList.Add("IntValue", (int) 3);
source.SourceList.Add("DoubleValue", (double) 3.14);
Mapper.Initialize(config =>
{
//"Magic code"
config.CreateMap<Source, IDestination>();
config.CreateMap(typeof(Source), typeof(Destination)).IncludeBase(typeof(Source), typeof(IDestination));
});
//standard map-call
var destination = Mapper.Map<Destination>(source);
//Additional "Trick":
Dictionary<string, object> mappingDict =
source.SourceList.ToDictionary(pair => pair.Key, pair => (object) pair.Value);
Mapper.Map(mappingDict, destination, typeof(Dictionary<string, object>), typeof(Destination));
Assert.AreEqual(source.Name, destination.Name);
Assert.AreEqual(source.SourceList["IntValue"], destination.IntValue);
Assert.AreEqual(source.SourceList["DoubleValue"], destination.DoubleValue);
}
The "trick" is to cast my Dictionary<string, ValueType> to Dictionary<string,object> and to map this dictionary-member to the destination object in addition(!) to the standard map-call.
This works, but has some drawbacks:
Mapping validation is not possible (Validation says: either source member "SourceList" is not mapped or the destination members "DoubleValue" or "IntValue" are not mapped)
Casting the dictionary is kind of ugly (and seems unnecessary to me...)
I need 2 calls to Mapper.Map instead of only one.
It seems to me, that there is no other way to solve my initial problem. But I am open for any suggestions or improvements to my solution.
The initial problem could be solved also easily through using reflections, so all information for a proper mapping setup should be existent, but I was not able to find this proper mapping setup.

Best way to pass a get/set property as an argument without reflection

Sometimes, when handling data transfer objects (for instance retrieved from the database or a csv file), it's nice to write some helper functions to move the data.
For instance:
class MyDto
{
public string Name { get; set; }
}
class MyBusinessObject
{
public string Name { get; set;}
}
I'd like to write something like:
MyDto source;
MyBusinessObject target;
var hasChanged = target.Set(source, source => source.Name, target => target.Name); // lamdba expressions, or whatever it takes to make it work
with the extension method:
public static bool Set<TS, TT, TValue>(this TS source, IGetProperty<TS, TValue> sourceGetProperty, IGetOrSetProperty<TT, TValue> targetGetOrSetProperty)
{
var sourceValue = sourceGetProperty.Invoke(source);
var actualValue = targetGetOrSetProperty.Invoke(target);
if(sourceValue != actualValue)
{
targetGetOrSetPropery.Invoke(target, sourceValue);
return true;
}
return false;
}
I made up the IGetProperty and IGetOrSetProperty. Is it possible to implement them some way without using reflection (so that it's compile-time checked)?
Or is there an elegant way to handle this kind of situation?
EDIT: the example was misleading because the goal wasn't to use an Automapper, but to represent somehow properties as objects. I realize that it's pretty close in fact to the idea of using properties as "ref" for instance, so it's more a language-related question that has always been answered here: Passing properties by reference in C#

It's not exactly possible without reflection, but the expression lambda gives you compile time checking:
public static bool Set<TTarget, TValue>(
this TTarget target,
Expression<Func<TTarget, TValue>> targetProperty,
TValue sourceValue)
{
var actualValue = targetProperty.Compile().Invoke(target);
if (actualValue.Equals(sourceValue))
{
return false;
}
var property = (PropertyInfo)((MemberExpression)targetProperty.Body).Member;
property.SetValue(target, sourceValue);
return true;
}
Usage looks like so:
var hasChanged = target.Set(t => t.Name, source.Name);
Working example: https://dotnetfiddle.net/CJVxIS
Why you should not do this:
targetProperty.Compile() is slow,
Automapper does such mappings for you.

You can consider serialising/deserialising. It may be less efficient (but then again, rudimentary implementation of reflection is expensive too) but it will be syntactically more readable and elegant. The other benefit is, Json.Net is very flexible so you can customise the copy behaviours (e.g map Name property to a property of another name)
class MyDto
{
public string Name { get; set; }
public string Other { get; set; }
public string Remap { get; set; }
}
class MyBusinessObject
{
[JsonIgnore]
public string Other { get; set; }
public string Name { get; set; }
[JsonProperty(PropertyName = "Remap")]
public string RemmapedField { get; set; }
}
public T DeepCopy<T>(object o)
{
string json=JsonConvert.SerializeObject(o);
T newO=JsonConvert.DeserializeObject<T>(json);
return newO;
}
Usage
MyDto source = new MyDto() { Name = "JP", Other = "Something",Remap="R" };
var target = DeepCopy<MyBusinessObject>(source);
Result:
Name: "JP"
Other: null
RemmapedField: "R"

Which Json deserializer renders IList<T> collections?

I'm trying to deserialize json to an object model where the collections are represented as IList<T> types.
The actual deserializing is here:
JavaScriptSerializer serializer = new JavaScriptSerializer();
return serializer.Deserialize<IList<Contact>>(
(new StreamReader(General.GetEmbeddedFile("Contacts.json")).ReadToEnd()));
Before i post the exception i'm getting you should know what the implicit conversions are. This is the Contact type:
public class Contact
{
public int ID { get; set; }
public string Name { get; set; }
public LazyList<ContactDetail> Details { get; set; }
//public List<ContactDetail> Details { get; set; }
}
And this is the ContactDetail type:
public class ContactDetail
{
public int ID { get; set; }
public int OrderIndex { get; set; }
public string Name { get; set; }
public string Value { get; set; }
}
The important thing to know with the LazyList<T> is that it implements IList<T>:
public class LazyList<T> : IList<T>
{
private IQueryable<T> _query = null;
private IList<T> _inner = null;
private int? _iqueryableCountCache = null;
public LazyList()
{
this._inner = new List<T>();
}
public LazyList(IList<T> inner)
{
this._inner = inner;
}
public LazyList(IQueryable<T> query)
{
if (query == null)
throw new ArgumentNullException();
this._query = query;
}
Now this LazyList<T> class definition was fine until i tried deserializing Json into it. The System.Web.Script.Serialization.JavaScriptSerializer seems to want to serialize lists to List<T> which makes sense coz of it's age but i need them in the type IList<T> so they will cast into my LazyList<T> (at least that's where i think i am going wrong).
I get this exception:
System.ArgumentException: Object of type 'System.Collections.Generic.List`1[ContactDetail]' cannot be converted to type 'LazyList`1[ContactDetail]'..
When i try using List<ContactDetail> in my Contact type (as you can see commented above) it seems to work. But i dont want to use List<T>'s. I even tried having my LazyList<T> inheriting from List<T> which seemed to execute but passing the List<T>'s internal T[] to my implementation was a nightmare and i simply don't want the bloat of List<T> anywhere in my model.
I also tried some other json libraries to no avail (it's possible i may not be using these to their full potential. I more or less replaced the references and attempted to repeat the code quoted at the top of this question. Maybe passing settings params will help??).
I dont know what to try now. Do i go with another deserializer? Do i tweak the deserializing itself? Do i need to change my types to please the deserializer? Do i need to worry more about implicit casting or just implement another interface?

It is not possible to deserialize directly to an interface, as interfaces are simply a contract. The JavaScriptSerializer has to deserialize to some concrete type that implements IList<T>, and the most logical choice is List<T>. You will have to convert the List to a LazyList, which given the code you posted, should be easy enough:
var list = serializer.Deserialize<IList<Contact>>(...);
var lazyList = new LazyList(list);

Unfortunately you will probably need to fix your class, as there is no way for a deserializer to know that it should be of type IList, since List is an implementation of IList.
Since the deserializers at http://json.org have source available you could just modify one to do what you want.

I ended up using the Json.NET lib which has good linq support for custom mapping. This is what my deserializing ended up looking like:
JArray json = JArray.Parse(
(new StreamReader(General.GetEmbeddedFile("Contacts.json")).ReadToEnd()));
IList<Contact> tempContacts = (from c in json
select new Contact
{
ID = (int)c["ID"],
Name = (string)c["Name"],
Details = new LazyList<ContactDetail>(
(
from cd in c["Details"]
select new ContactDetail
{
ID = (int)cd["ID"],
OrderIndex = (int)cd["OrderIndex"],
Name = (string)cd["Name"],
Value = (string)cd["Value"]
}
).AsQueryable()),
Updated = (DateTime)c["Updated"]
}).ToList<Contact>();
return tempContacts;

Casting problems when trying to be generic / 'type free' | ASP MVC

Question
Is there a way to define a method only once in C# (in a helper class or something) not knowing which type is given to be returned?
Long explanation
I get the following error:
Unable to cast object of type
System.Data.Objects.ObjectQuery1[WerkStageNu.Vacancies]'
to type
'System.Linq.IQueryable1[WerkStageNu.Models.IFilteredEntities]'.
I have a ListingsController which does a Search through my current Vacancies in the database:
public ActionResult Search(int? page, string branchid, string hoursago, string jobtypeid, string educationlevelid, string careerlevelid)
{
string searchResult = string.Empty;
const int pageSize = 10;
IQueryable<IFilteredEntities> selectedListings = (IQueryable<IFilteredEntities>)Repository.Instance._entities.Vacancies.AsQueryable();
Dictionary<string, string> filterParams = new Dictionary<string, string>() {
{"branchid", branchid}, {"hoursago", hoursago}, {"jobtypeid", jobtypeid}, {"educationlevelid", educationlevelid}, {"careerlevelid", careerlevelid}};
selectedListings = FilterByIDHelper.Filter(selectedListings, filterParams);
var paginatedDinners = new PaginatedList<Vacancies>(((IQueryable<Vacancies>)selectedListings).ToList(), page ?? 0, pageSize);
return View("Index", paginatedDinners);
}
Now, this search is just for Vacancies. But one can imagine we have searches all over the place all in general the same routine so I want to call the same method getting back different types. For this case I have made an Interface , IFilteredEntities. In my partial class Vacancies (partial class, class Vacancies is generated by my DB entity framework) I just do:
public partial class Vacancies : IFilteredEntities
And of course implement the methods in the Interface which are not implemented by Default. In my Interface I have:
interface IFilteredEntities
{
string EducationLevelID { get; set; }
string BrancheID { get; set; }
string CareerLevelID { get; set; }
string JobTypeID { get; set; }
Branches Branches { get; set; }
DateTime? DateOfCreation { get; set; }
CareerLevels CareerLevels { get; set; }
JobTypes JobTypes { get; set; }
EducationLevels EducationLevels { get; set; }
}
For convenience I have uploaded the two helper classes PaginatedList and FilterCriteriaHelper here and here.
Now, the method which would do the actual filtering is placed inside another helper class: FilterByIDHelper.cs.
public static IQueryable<IFilteredEntities> Filter(IQueryable<IFilteredEntities> collection, Dictionary<string, string> filterParams)
{
if (filterParams.ContainsKey("branchid")) collection = FilterByBranchId(collection, filterParams["branchid"]);
if (filterParams.ContainsKey("hoursago")) collection = FilterByHoursAgo(collection, filterParams["hoursago"]);
if (filterParams.ContainsKey("jobtypeid")) collection = FilterByJobTypeId(collection, filterParams["jobtypeid"]);
if (filterParams.ContainsKey("educationlevelid")) collection = FilterByEducationLevelId(collection, filterParams["educationlevelid"]);
if (filterParams.ContainsKey("careerlevelid")) collection = FilterByCareerLevelId(collection, filterParams["careerlevelid"]);
return collection;
}
public static IQueryable<IFilteredEntities> Filter(IQueryable<IFilteredEntities> collection, Dictionary<string, string> filterParams)
{
if (filterParams.ContainsKey("branchid")) collection = FilterByBranchId(collection, filterParams["branchid"]);
if (filterParams.ContainsKey("hoursago")) collection = FilterByHoursAgo(collection, filterParams["hoursago"]);
if (filterParams.ContainsKey("jobtypeid")) collection = FilterByJobTypeId(collection, filterParams["jobtypeid"]);
if (filterParams.ContainsKey("educationlevelid")) collection = FilterByEducationLevelId(collection, filterParams["educationlevelid"]);
if (filterParams.ContainsKey("careerlevelid")) collection = FilterByCareerLevelId(collection, filterParams["careerlevelid"]);
return collection;
}
For convenience here is a picture of a part of my solution explorer:
Solution Explorer http://www.bastijn.nl/zooi/solutionexplorer.png
In short:
What I try to do is instead of calling like:
selectedListings = Repository.Instance._entities.Vacancies.AsQueryable();
Dictionary<string, string> filterParams = new Dictionary<string, string>() {
{"branchid", branchid}, {"hoursago", hoursago}, {"jobtypeid", jobtypeid}, {"educationlevelid", educationlevelid}, {"careerlevelid", careerlevelid}};
selectedListings = FilterByIDHelper.Filter(selectedListings, filterParams);
var paginatedDinners = new PaginatedList<Vacancies>(selectedListings.ToList(), page ?? 0, pageSize);
return View("Index", paginatedDinners);
Call the variant shown up, using an Interface so I only have to define te "Filter" method once instead of for all classes / models. Now Notice that all of this DOES compile! The problem is that I get the following error:
Unable to cast object of type 'System.Data.Objects.ObjectQuery`1[WerkStageNu.Vacancies]' to type 'System.Linq.IQueryable`1[WerkStageNu.Models.IFilteredEntities]'.
I hope I have not forgotten any information but I'm already staring at this code for some while. Might forget a relation or something, just ask for it if I did :).
-----------------------------------------------------
EDIT AFTER COMMENTS
-----------------------------------------------------
O crap, nevermind this part, I forgot to as AsEnumerable, was still using AsQueryable.

It looks to me like this is a covariance vs. contravariance issue. Basically, an IQueryable<Vacancies> is not a sub-type of IQueryable<IFilteredEntities>, even though Vacancies implements IFilteredEntities. Thus, the line with the cast is causing a runtime error. So rather than doing the cast try this instead:
IEnumerable<IFilteredEntities> selectedListings =
Repository.Instance._entities.Vacancies.AsQueryable()
.OfType<IFilteredEntities>();
What this will do is project each element of the collection to an IFilteredEntities type.
Another option is to rewrite your filter methods so they use generics, like this:
public static IEnumerable<T> Filter<T>(
IEnumerable<T> collection, IDictionary<string, string> filterParams)
where T : IFilteredEntities
{
...
}
This would then allow you to pass in a collection containing any type that derives from IFilteredEntities and get back a collection of the same type. And if you're using C# 3, you don't even have to specify the type parameter if it can be implicitly determined by the compiler.