find value in List of object by string - c#

I need to find the longest value from a list of objects like this...
var longestValue = list.Max(x => x.Name);
The problem is that I can not access it directly like this but this need to be made in a loop. Here is what I have so far..
public static void SetPropertyValue(this object obj, string propName, object value)
{
obj.GetType().GetProperty(propName).SetValue(obj, value, null);
}
var user = new User();
var list = new List<User>
{
new Svedea {Name = "Steve", Car = "Volkswagen"},
new Svedea {Name = "Denice Longhorn", Car = "Tesla"},
new Svedea {Name = "Rebecca", Car = "Ford"},
new Svedea {Name = "Mike O", Car = "Mercedes-Benz"}
};
var properties = user.GetType().GetProperties();
var propList = properties.Select(pi => pi.Name).ToList();
var newUser = new User();
foreach (var row in propList)
{
// Here I need to find the longest value from the list above like this...
// var longestValue = list.Max(x => x.row); // This is obviously not correct but I need to find this value needs to be found dynamically
var longestValue = list.Max(x => x.row);
newUser.SetPropertyValue(row, longestValue);
}


Use LINQ OrderByDescending to sort your list by the longest value of Name and the longest value of Car. Then you can create a new User longestUser that has the longest values of Name and Car:
var longestName = list.OrderByDescending(x => x.Name.Length).First().Name;
var longestCar = list.OrderByDescending(x => x.Car.Length).First().Car;
var longestUser = new User() { Name = longestName, Car = longestCar };
longestUser:
Name = "Denice Longhorn"
Car = "Mercedes-Benz"
EDIT:
I have modified my solution above so that you can access the property names with string values :
var longestNameUser = list.OrderByDescending(x => x.GetType().GetProperty("Name").GetValue(x, null).ToString().Length).First();
var longestName = longestNameUser.GetType().GetProperty("Name").GetValue(longestNameUser, null);
var longestCarUser = list.OrderByDescending(x => x.GetType().GetProperty("Car").GetValue(x, null).ToString().Length).First();
var longestCar = longestCarUser.GetType().GetProperty("Car").GetValue(longestCarUser, null);
var newUser = new User();
PropertyInfo propertyInfo;
propertyInfo = newUser.GetType().GetProperty("Name");
propertyInfo.SetValue(newUser, Convert.ChangeType(longestName, propertyInfo.PropertyType), null);
propertyInfo = newUser.GetType().GetProperty("Car");
propertyInfo.SetValue(newUser, Convert.ChangeType(longestCar, propertyInfo.PropertyType), null);
Your modified for loop :
var newUser = new User();
PropertyInfo propertyInfo;
foreach (var row in propList)
{
var longestValueUser = list.OrderByDescending(x => x.GetType().GetProperty(row).GetValue(x, null).ToString().Length).First();
var longestValue = longestValueUser.GetType().GetProperty(row).GetValue(longestValueUser, null);
propertyInfo = newUser.GetType().GetProperty(row);
propertyInfo.SetValue(newUser, Convert.ChangeType(longestValue, propertyInfo.PropertyType), null);
}


You can do something like this with a mixture of generics and reflection
Exmaple
public static T CreateType<T>(List<T> list) where T : new ()
{
var obj = new T();
var type = typeof(T);
var properties = type.GetProperties();
foreach (var prop in properties)
{
var value = list.Select(x => x.GetType()
.GetProperties()
.First(y => y == prop)
.GetValue(x) as string)
.OrderByDescending(x => x.Length)
.First();
var propInstance = obj.GetType()
.GetProperties()
.First(x => x == prop);
propInstance.SetValue(obj, value);
}
return obj;
}
Usage
var someList = new List<SomeClass>()
{
new SomeClass()
{
Name = "asd",
Car = "sdfsdf"
},
new SomeClass()
{
Name = "dfgfdg",
Car = "dfgdf"
}
};
var SomeClass = CreateType(someList);
Output
some value 1 longer, Some value 2 longer
Full demo here

Related

Dynamic LINQ Performance

I have a routine (written with the generous help of others here) that allows me to take a List objects, and using any number of properties in any order it dynamically builds a TreeView structure with a Count at each node. This dynamic ability is a firm user requirement.
So a source List of:
{Prop1 = "A", Prop2 = "I", Prop3 = "X"},
{Prop1 = "A", Prop2 = "J", Prop3 = "X"},
{Prop1 = "B", Prop2 = "I", Prop3 = "X"},
{Prop1 = "B", Prop2 = "I", Prop3 = "Y"},
{Prop1 = "C", Prop2 = "K", Prop3 = "Z"}
Gives the following when the Selection is by Prop1 by Prop3:
Total (5)
- A(2)
- - X(2)
- B(2)
- - X(1)
- - Y(1)
- C(1)
- - Z(1)
Functionally this works fine. However, the performance leaves a lot to be desired when the number of distinct values increases. For example - one particular run in a dataset with 5K objects and 1K distinct values in Prop1 will take several seconds.
Here is the routine:
public static class TreeBuilder
{
public static Dictionary<string, TreeItem> BuildTree<TSource>(List<TSource> source, List<string> columns)
{
return new Dictionary<string, TreeItem>()
{
{ "Total",
new TreeItem()
{
Key = "Total",
RawKey = "Total",
Count = source.Count,
Items = GroupBySelector<TSource>(source, columns, 0, "Total")
}
}
};
}
public static MethodInfo GetGenericMethod(this Type type, string name, Type[] genericTypeArgs, Type[] paramTypes)
{
foreach (MethodInfo method in type.GetMethods())
if (method.Name == name)
{
var pa = method.GetParameters();
if (pa.Length == paramTypes.Length)
{
var genericMethod = method.MakeGenericMethod(genericTypeArgs);
if (genericMethod.GetParameters().Select(p => p.ParameterType).SequenceEqual(paramTypes))
return genericMethod;
}
}
return null;
}
private static MethodInfo GetGroupByMethodStatically<TElement, TKey>()
=> typeof(Enumerable).GetGenericMethod("GroupBy", new[] { typeof(TElement), typeof(TKey) }, new[] { typeof(IEnumerable<TElement>), typeof(Func<TElement, TKey>) });
private static MethodInfo GetEnumerableMethod(string methodName, Type tElement, Type tTKey)
{
var tIElement = typeof(IEnumerable<>).MakeGenericType(tElement);
var tFunction = typeof(Func<,>).MakeGenericType(tElement, tTKey);
return typeof(Enumerable).GetGenericMethod(methodName, new[] { tElement, tTKey }, new[] { tIElement, tFunction });
}
private static MethodInfo GetEnumerableMethod(string methodName, Type tElement)
{
var tIELEMENT = typeof(IEnumerable<>).MakeGenericType(tElement);
return typeof(Enumerable).GetGenericMethod(methodName, new[] { tElement }, new[] { tIELEMENT });
}
public static Dictionary<string, TreeItem> GroupBySelector<TElement>(IEnumerable<TElement> source, IList<string> columnNames, int entry = 0, string key = "")
{
if (source == null) throw new ArgumentNullException(nameof(source));
List<string> columnParameters = columnNames[entry].Split('|').ToList();
string columnName = columnParameters[0];
if (columnName == null) throw new ArgumentNullException(nameof(columnName));
if (columnName.Length == 0) throw new ArgumentException(nameof(columnName));
int nextEntry = entry + 1;
var tElement = typeof(TElement);
var tIElement = typeof(IEnumerable<TElement>);
var keyParm = Expression.Parameter(tElement);
var prop = Expression.Property(keyParm, columnName);
var param = Expression.Parameter(tIElement, "p");
var groupByMethod = GetEnumerableMethod("GroupBy", tElement, prop.Type);
var groupByExpr = Expression.Lambda(prop, keyParm);
var bodyExprCall = Expression.Call(groupByMethod, param, groupByExpr);
var tSelectInput = typeof(IGrouping<,>).MakeGenericType(prop.Type, tElement);
var selectParam = Expression.Parameter(tSelectInput, "p");
var tKey = typeof(TreeItem).GetMember("Key").Single();
var tRawKey = typeof(TreeItem).GetMember("RawKey").Single();
var tCount = typeof(TreeItem).GetMember("Count").Single();
var tParentKey = typeof(TreeItem).GetMember("ParentKey").Single();
var tItems = typeof(TreeItem).GetMember("Items").Single();
Expression selectParamKey = Expression.Property(selectParam, "Key");
Expression selectParamRawKey = selectParamKey;
if (selectParamKey.Type != typeof(string))
{
var toStringMethod = selectParamKey.Type.GetMethod("ToString", Type.EmptyTypes);
selectParamKey = Expression.Call(selectParamKey, toStringMethod);
}
if (selectParamRawKey.Type != typeof(string))
{
var toStringMethod = selectParamRawKey.Type.GetMethod("ToString", Type.EmptyTypes);
selectParamRawKey = Expression.Call(selectParamRawKey, toStringMethod);
}
var countMethod = GetEnumerableMethod("Count", tElement);
var countMethodExpr = Expression.Call(countMethod, selectParam);
var concatFullKeyExpr = Expression.Call(typeof(string).GetMethod("Concat", new[] { typeof(string), typeof(string), typeof(string) }),
Expression.Constant(key),
Expression.Constant("|"),
selectParamRawKey);
var groupBySelectorMethod = GetGenericMethod(MethodBase.GetCurrentMethod().DeclaringType, "GroupBySelector", new[] { tElement }, new[] { tIElement, typeof(IList<string>), typeof(int), typeof(string) });
var groupBySelectorMethodExpr = Expression.Call(groupBySelectorMethod, selectParam, Expression.Constant(columnNames), Expression.Constant(nextEntry), concatFullKeyExpr);
var newMenuItemExpr = Expression.New(typeof(TreeItem));
var selectBodyExpr = Expression.MemberInit(newMenuItemExpr, new[] {
Expression.Bind(tKey, selectParamKey),
Expression.Bind(tRawKey, selectParamRawKey),
Expression.Bind(tParentKey, Expression.Constant(key) ),
Expression.Bind(tCount, countMethodExpr),
Expression.Bind(tItems, groupBySelectorMethodExpr)
});
var selectBodyExprLamba = Expression.Lambda(selectBodyExpr, selectParam);
var selectBodyLastExpr = Expression.MemberInit(newMenuItemExpr, new[] {
Expression.Bind(tKey, selectParamKey),
Expression.Bind(tRawKey, selectParamRawKey),
Expression.Bind(tParentKey, Expression.Constant(key) ),
Expression.Bind(tCount, countMethodExpr)
});
var selectBodyLastExprLambda = Expression.Lambda(selectBodyLastExpr, selectParam);
var selectMethod = GetEnumerableMethod("Select", tSelectInput, typeof(TreeItem));
bodyExprCall = Expression.Call(selectMethod, bodyExprCall, (nextEntry < columnNames.Count) ? selectBodyExprLamba : selectBodyLastExprLambda);
var selectParamout = Expression.Parameter(typeof(TreeItem), "o");
Expression selectParamKeyout = Expression.Property(selectParamout, "FullKey");
var selectParamKeyLambda = Expression.Lambda(selectParamKeyout, selectParamout);
var lmi = GetEnumerableMethod("ToDictionary", typeof(TreeItem), typeof(string));
bodyExprCall = Expression.Call(lmi, bodyExprCall, selectParamKeyLambda);
var returnFunc = Expression.Lambda<Func<IEnumerable<TElement>, Dictionary<string, TreeItem>>>(bodyExprCall, param).Compile();
return returnFunc(source);
}
}
The routine is used to take data from a DB table and convert it into a hierarchical structure for use in a WPF TreeView.
Dictionary<string, TreeItem> treeItems = new Dictionary<string, TreeItem>();
treeItems = TreeBuilder.BuildTree<IDBRecord>(DBService.GetDBRecordList(), PropertySortList);
Can anyone offer any advice on how to improve the performance of this routine? Or suggest any alternative way of achieving the same result in a more performant way?
Thanks

A couple of optimizations are possible. A lot of time is spent in the call to Compile and you are calling Compile for each key at each level in the tree, which is adding up to a lot of overhead, about 7 seconds on my tests of 5k items. I first changed the code to pull out all static Reflection that had fixed types, so it is only done once per program run. That only made a small difference, since building the Expression tree is not the main issue.
I then changed the method to separate building the Expression from compiling the Expression and calling the resultant lambda. This allowed me to modify the recursive call to the Expression builder to instead be an inline Invoke of a new lambda for the new level. Then I called compile once on the resulting Expression and executed it. The new version no longer takes the entry parameter, but it could be put back in.
This reduced the overall time from about 7.6 seconds to 0.14 seconds for around a 50x speedup. A test with all three properties resulted in a 280x speedup.
If it is still possible for repeated calls to the method, adding a cache would be of even more benefit, though a quick test only shows about 14% time savings, and in the hundredths of a second of real time.
static MemberInfo tKey = typeof(TreeItem).GetMember("Key").Single();
static MemberInfo tRawKey = typeof(TreeItem).GetMember("RawKey").Single();
static MemberInfo tCount = typeof(TreeItem).GetMember("Count").Single();
static MemberInfo tParentKey = typeof(TreeItem).GetMember("ParentKey").Single();
static MemberInfo tItems = typeof(TreeItem).GetMember("Items").Single();
// Concat(string, string, string)
static MethodInfo Concat3MI = ((Func<string, string, string, string>)String.Concat).Method;
// new TreeItem() { ... }
static NewExpression newMenuItemExpr = Expression.New(typeof(TreeItem));
// Enumerable.ToDictionary<TreeItem>(IEnumerable<TreeItem>, Func<TreeItem,string>)
static MethodInfo ToDictionaryMI = GetEnumerableMethod("ToDictionary", typeof(TreeItem), typeof(string));
static Expression<Func<IEnumerable<TElement>, Dictionary<string, TreeItem>>> BuildGroupBySelector<TElement>(IList<string> columnNames, int entry, Expression key) {
List<string> columnParameters = columnNames[entry].Split('|').ToList();
string columnName = columnParameters[0];
if (columnName == null) throw new ArgumentNullException(nameof(columnName));
if (columnName.Length == 0) throw new ArgumentException(nameof(columnName));
int nextEntry = entry + 1;
var tElement = typeof(TElement);
var tIElement = typeof(IEnumerable<TElement>);
// (TElement kp)
var keyParm = Expression.Parameter(tElement, "kp");
// kp.columnName
var prop = Expression.Property(keyParm, columnName);
// (IEnumerable<TElement> p)
var IEParam = Expression.Parameter(tIElement, "p");
// GroupBy<TElement>(IEnumerable<TElement>, Func<TElement, typeof(kp.columnName)>)
var groupByMethod = GetEnumerableMethod("GroupBy", tElement, prop.Type);
// kp => kp.columnName
var groupByExpr = Expression.Lambda(prop, keyParm);
// GroupBy(p, kp => kp.columnName)
var bodyExprCall = Expression.Call(groupByMethod, IEParam, groupByExpr);
// typeof(IGrouping<typeof(kp.columnName), TElement>)
var tSelectInput = typeof(IGrouping<,>).MakeGenericType(prop.Type, tElement);
// (IGrouping<typeof(kp.columnName), TElement> sp)
var selectParam = Expression.Parameter(tSelectInput, "sp");
// sp.Key
Expression selectParamKey = Expression.Property(selectParam, "Key");
Expression selectParamRawKey = selectParamKey;
if (selectParamKey.Type != typeof(string)) {
var toStringMethod = selectParamKey.Type.GetMethod("ToString", Type.EmptyTypes);
// sp.Key.ToString()
selectParamKey = Expression.Call(selectParamKey, toStringMethod);
selectParamRawKey = selectParamKey;
}
// Count<TElement>()
var countMethod = GetEnumerableMethod("Count", tElement);
// sp.Count()
var countMethodExpr = Expression.Call(countMethod, selectParam);
LambdaExpression selectBodyExprLamba;
if (nextEntry < columnNames.Count) {
// Concat(key, "|", sp.Key.ToString())
var concatFullKeyExpr = Expression.Call(Concat3MI, key, Expression.Constant("|"), selectParamRawKey);
// p# => p#.GroupBy().Select().ToDictionary()
var groupBySelectorLambdaExpr = BuildGroupBySelector<TElement>(columnNames, nextEntry, (Expression)concatFullKeyExpr);
// Invoke(p# => p#..., sp#)
var groupBySelectorInvokeExpr = Expression.Invoke(groupBySelectorLambdaExpr, selectParam);
var selectBodyExpr = Expression.MemberInit(newMenuItemExpr, new[] {
Expression.Bind(tKey, selectParamKey),
Expression.Bind(tRawKey, selectParamRawKey),
Expression.Bind(tParentKey, key ),
Expression.Bind(tCount, countMethodExpr),
Expression.Bind(tItems, groupBySelectorInvokeExpr)
});
// sp => new TreeItem { Key = sp.Key.ToString(), RawKey = sp.Key.ToString(), ParentKey = key, Count = sp.Count(), Items = Invoke(p# => p#..., sp)) }
selectBodyExprLamba = Expression.Lambda(selectBodyExpr, selectParam);
}
else { // Last Level
var selectBodyExpr = Expression.MemberInit(newMenuItemExpr, new[] {
Expression.Bind(tKey, selectParamKey),
Expression.Bind(tRawKey, selectParamRawKey),
Expression.Bind(tParentKey, key ),
Expression.Bind(tCount, countMethodExpr)
});
// sp => new TreeItem { Key = sp.Key.ToString(), RawKey = sp.Key.ToString(), ParentKey = key, Count = sp.Count() }
selectBodyExprLamba = Expression.Lambda(selectBodyExpr, selectParam);
}
// Enumerable.Select<IGrouping<typeof<kp.columnName>, TElement>>(IEnumerable<IGrouping<>>, Func<IGrouping<>, TreeItem>)
var selectMethod = GetEnumerableMethod("Select", tSelectInput, typeof(TreeItem));
// p.GroupBy(kp => kp => kp.columnName).Select(sp => ...)
bodyExprCall = Expression.Call(selectMethod, bodyExprCall, selectBodyExprLamba);
// (TreeItem o)
var selectParamout = Expression.Parameter(typeof(TreeItem), "o");
// o.FullKey
Expression selectParamKeyout = Expression.Property(selectParamout, "FullKey");
// o => o.FullKey
var selectParamKeyLambda = Expression.Lambda(selectParamKeyout, selectParamout);
// p.GroupBy(...).Select(...).ToDictionary(o => o.FullKey)
bodyExprCall = Expression.Call(ToDictionaryMI, bodyExprCall, selectParamKeyLambda);
// p => p.GroupBy(kp => kp => kp.columnName).Select(sp => ...).ToDictionary(o => o.FullKey)
return Expression.Lambda<Func<IEnumerable<TElement>, Dictionary<string, TreeItem>>>(bodyExprCall, IEParam);
}
public static Dictionary<string, TreeItem> GroupBySelector<TElement>(IEnumerable<TElement> source, IList<string> columnNames, string key = "") {
if (source == null) throw new ArgumentNullException(nameof(source));
// p => p.GroupBy(kp => kp => kp.columnName).Select(sp => ...).ToDictionary(o => o.FullKey)
var returnFunc = BuildGroupBySelector<TElement>(columnNames, 0, Expression.Constant(key)).Compile();
return returnFunc(source);
}

Want to make Func<UnknownType, bool> where UnknownType is only known at runtime

So, I'm experimenting with expressions atm. Got code below:
Code works fine except for 1 thing: I need to replace ViewModel type with ForeignKeyProperty.PropertyType, which is only known at runtime, at the line of var condition = Expression.Lambda < Func < ViewModel, bool> >
Expected end result:
ForeignKeyProperty.SetValue(model, repository.GetList <ForeignKeyProperty.PropertyType >().Single(x => x.Id == model.Id));
protected List < Action < IVenturaRepository, ViewModel>> SetForeignKeyProperties<ViewModel>() where ViewModel : BaseViewModel
{
var viewModelType = typeof(ViewModel);
var foreignKeyProperties = viewModelType.GetProperties().Where(x => x.PropertyType.IsSubclassOf(typeof(BaseViewModel)));
var actions = new List < Action < IVenturaRepository, ViewModel>>();
var repositoryType = typeof(IVenturaRepository);
foreach(var ForeignKeyProperty in foreignKeyProperties)
{
var foreignKeyIdProperty = viewModelType.GetProperties().SingleOrDefault(x => x.Name == ForeignKeyProperty.Name + "Id");
//ForeignKeyProperty.SetValue(model, repository.GetList<ViewModel>().Single(x => x.Id == model.Id));
var listMethod = repositoryType.GetMethods().SingleOrDefault(x => x.Name == "GetList").MakeGenericMethod(ForeignKeyProperty.PropertyType);
//Expression.Call(singleMethod,);
var repositoryVariable = Expression.Parameter(repositoryType, "repository");
var paramViewModelType = Expression.Parameter(viewModelType, "model");
var paramForeignEntityId = Expression.Property(paramViewModelType, "Id");
var listMethodCall = Expression.Call(repositoryVariable, listMethod);
var modelParameter = Expression.Parameter(ForeignKeyProperty.PropertyType, "x");
var foreignKeyTypeConstant = Expression.Constant(ForeignKeyProperty.PropertyType);
var condition =
Expression.Lambda < Func < ViewModel, bool>>(
Expression.Equal(
Expression.Property(paramViewModelType, foreignKeyIdProperty.Name),
Expression.Convert(Expression.Property(modelParameter, "Id"),foreignKeyIdProperty.PropertyType)
),
modelParameter
);
//var singleMethod = typeof(Enumerable).GetMethods().SingleOrDefault(x => x.Name.Equals("SingleOrDefault") && x.GetParameters().Count() ==2).MakeGenericMethod(viewModelType);
//var singleMethod = typeof(IEnumerable<ViewModel>).GetMethods().SingleOrDefault(x => x.GetParameters().Count() > 0).MakeGenericMethod(viewModelType);
//var singleLambda = Expression.Lambda(Expression.Property(modelParameter, "Id"), modelParameter);
var singleMethodCall = Expression.Call(typeof(Enumerable), "SingleOrDefault", new[] { ForeignKeyProperty.PropertyType },listMethodCall, condition);
//var singleMethodCall = Expression.Call(listMethodCall, singleMethod, condition);
var setMethod = ForeignKeyProperty.GetSetMethod();
var oParameter = Expression.Parameter(viewModelType, "obj");
var vParameter = Expression.Parameter(typeof(ViewModel),"value");
var method = Expression.Call(oParameter,setMethod, singleMethodCall);
var expression = Expression.Lambda<Action<IVenturaRepository, ViewModel>>(method);
actions.Add(expression.Compile());
}
return actions;
}
Could someone point me in the right direction please?

use Object as type. then you can check type by getType() at runtime and after check cast to correct type. or use dynamic to avoid casting.

So, I did as you suggested and used object instead. It works now.
Code below works in a specific controller that inherits from BaseController.
public override ActionResult Edit(long id, DeliveryEditViewModel model)
{
model.SourceModel.DeliveryRound = Repository.DeliveryRounds.Single(x => x.Id == model.DeliveryRoundId);
model.SourceModel.Sale = Repository.Sales.Single(x => x.Id == model.SaleId);
return base.Edit(id, model);
}
Idea is that code below works in BaseController:
public virtual ActionResult Edit(long id, EditModel model)
{
try
{
if (setForeignKeyActionList == null)
setForeignKeyActionList = SetForeignKeyProperties();
setForeignKeyActionList.ForEach(action => action(Repository, model.SourceModel));
Repository.SaveItem(model);
return RedirectToAction("Index");
}
catch(Exception ex)
{
ModelState.AddModelError("Error", ex);
return View(Repository.GetEditableItem(id));
}
}
This is the revised code based on your suggestion to use object type
protected List< Action< IVenturaRepository, ViewModel>> SetForeignKeyProperties()
{
var viewModelType = typeof(ViewModel);
var foreignKeyProperties = viewModelType.GetProperties().Where(x => x.PropertyType.IsSubclassOf(typeof(BaseViewModel)));
var actions = new List< Action< IVenturaRepository, ViewModel>>();
var repositoryType = typeof(IVenturaRepository);
foreach (var ForeignKeyProperty in foreignKeyProperties)
{
var foreignKeyIdProperty = viewModelType.GetProperties().SingleOrDefault(x => x.Name == ForeignKeyProperty.Name + "Id");
//ForeignKeyProperty.SetValue(model, repository.GetList< OtherViewModel>().Single(x => x.Id == model.Id));
var listMethod = repositoryType.GetMethods().SingleOrDefault(x => x.Name == "GetList").MakeGenericMethod(ForeignKeyProperty.PropertyType);
var repositoryVariable = Expression.Parameter(repositoryType, "repository");
var paramViewModelType = Expression.Parameter(viewModelType, "model");
var paramForeignEntityId = Expression.Property(paramViewModelType, "Id");
var listMethodCall = Expression.Call(repositoryVariable, listMethod);
var foreignKeyTypeConstant = Expression.Constant(ForeignKeyProperty.PropertyType);
var objectType = Expression.Parameter(typeof(object), "model");
var modelParameter = Expression.Parameter(typeof(object), "x");
var expressionForeignKeyId = Expression.Property(paramViewModelType, foreignKeyIdProperty.Name);
var expressionForeignEntityId = Expression.Convert(Expression.Property(Expression.Convert(modelParameter, ForeignKeyProperty.PropertyType), "Id"), foreignKeyIdProperty.PropertyType);
var condition =
Expression.Lambda<Func<object, bool>>(
Expression.Equal(
expressionForeignKeyId,
expressionForeignEntityId
),
modelParameter
);
var singleMethodCall = Expression.Call(typeof(Enumerable), "SingleOrDefault", new[] { ForeignKeyProperty.PropertyType }, listMethodCall, condition);
//var singleMethodCall = Expression.Call(listMethodCall, singleMethod, condition);
var setMethod = ForeignKeyProperty.GetSetMethod();
//var oParameter = Expression.Parameter(viewModelType, "obj");
var vParameter = Expression.Parameter(typeof(ViewModel), "value");
var method = Expression.Call(paramViewModelType, setMethod, singleMethodCall);
var lamdaParameterExpressions = new[]
{
repositoryVariable,
paramViewModelType
};
var expression = Expression.Lambda<Action<IVenturaRepository, ViewModel>>(method, lamdaParameterExpressions);
actions.Add(expression.Compile());
}
return actions;
}

Best algorithm to determine added and removed items when comparing to collections

I am looking for the best algorithm to compare 2 collections and determine which element got added and which element got removed.
private string GetInvolvementLogging(ICollection<UserInvolvement> newInvolvement, ICollection<UserInvolvement> oldInvolvement)
{
//I defined the new and old dictionary's for you to know what useful data is inside UserInvolvement.
//Both are Dictionary<int, int>, because The Involvement is just a enum flag. Integer. UserId is also Integer.
var newDict = newInvolvement.ToDictionary(x => x.UserID, x => x.Involvement);
var oldDict = oldInvolvement.ToDictionary(x => x.UserID, x => x.Involvement);
//I Want to compare new to old -> and get 2 dictionaries: added and removed.
var usersAdded = new Dictionary<int, Involvement>();
var usersRemoved = new Dictionary<int, Involvement>();
//What is the best algoritm to accomplish this?
return GetInvolvementLogging(usersAdded, usersRemoved);
}
private string GetInvolvementLogging(Dictionary<int, Involvement> usersAdded, Dictionary<int, Involvement> usersRemoved)
{
//TODO: generate a string based on those dictionaries.
return "Change in userinvolvement: ";
}

Added elements are only in newDict removed only in oldDict
var intersection = newDict.Keys.Intersect(oldDict.Keys);
var added = newDict.Keys.Except(intersection);
var removed = oldDict.Keys.Except(intersection);
EDIT
I modify your base function, dictionaries is no neded.
Example UserInvolvement implementation
class UserInvolvement
{
public int UserId;
public string Name;
public string OtherInfo;
public override bool Equals(object obj)
{
if (obj == null)
{
return false;
}
UserInvolvement p = obj as UserInvolvement;
if ((System.Object)p == null)
{
return false;
}
return (UserId == p.UserId) && (Name == p.Name) && (OtherInfo == p.OtherInfo);
}
public override string ToString()
{
return $"{UserId} - {Name} - {OtherInfo}";
}
}
And example function:
private static string GetInvolvementLogging(ICollection<UserInvolvement> newInvolvement,
ICollection<UserInvolvement> oldInvolvement)
{
var intersection = newInvolvement.Select(x => x.UserId).Intersect(oldInvolvement.Select(x => x.UserId));
var addedIds = newInvolvement.Select(x => x.UserId).Except(intersection);
var removedIds = oldInvolvement.Select(x => x.UserId).Except(intersection);
List<UserInvolvement> modifiedUI = new List<UserInvolvement>();
foreach (var i in intersection)
{
var ni = newInvolvement.First(a => a.UserId == i);
var oi = oldInvolvement.First(a => a.UserId == i);
if (!ni.Equals(oi))
{
modifiedUI.Add(ni);
}
}
List<UserInvolvement> addedUI = newInvolvement.Where(x => addedIds.Contains(x.UserId)).Select(w => w).ToList();
List<UserInvolvement> removedUI = oldInvolvement.Where(x => removedIds.Contains(x.UserId)).Select(w => w).ToList();
StringBuilder sb = new StringBuilder();
sb.AppendLine("Added");
foreach (var added in addedUI)
{
sb.AppendLine(added.ToString());
}
sb.AppendLine("Removed");
foreach (var removed in removedUI)
{
sb.AppendLine(removed.ToString());
}
sb.AppendLine("Modified");
foreach (var modified in modifiedUI)
{
sb.AppendLine(modified.ToString());
}
return sb.ToString();
}
And my test function:
static void Main(string[] args)
{
List<UserInvolvement> newUI = new List<UserInvolvement>()
{
new UserInvolvement()
{
UserId = 1,
Name = "AAA",
OtherInfo = "QQQ"
},
new UserInvolvement()
{
UserId = 2,
Name = "BBB",
OtherInfo = "123"
},
new UserInvolvement()
{
UserId = 4,
Name = "DDD",
OtherInfo = "123ert"
}
};
List<UserInvolvement> oldUI = new List<UserInvolvement>()
{
new UserInvolvement()
{
UserId = 2,
Name = "BBBC",
OtherInfo = "123"
},
new UserInvolvement()
{
UserId = 3,
Name = "CCC",
OtherInfo = "QQ44"
},
new UserInvolvement()
{
UserId = 4,
Name = "DDD",
OtherInfo = "123ert"
}
};
string resp = GetInvolvementLogging(newUI, oldUI);
WriteLine(resp);
ReadKey();
WriteLine("CU");
}
Result is:
Added
1 - AAA - QQQ
Removed
3 - CCC - QQ44
Modified
2 - BBB - 123

You could try with Linq:
var usersAdded = newDict.Except(oldDict);
var usersRemoved = oldDict.Except(newDict);
If you need dictionaries as a result you can cast:
var usersAdded = newDict.Except(oldDict).ToDictionary(x => x.Key, x => x.Value);
var usersRemoved = oldDict.Except(newDict).ToDictionary(x => x.Key, x => x.Value);

Think best algorithm will be
foreach (var newItem in newDict)
if (!oldDict.ContainsKey(newItem.Key) || oldDict[newItem.Key]!=newItem.Value)
usersAdded.Add(newItem.Key, newItem.Value);
foreach (var oldItem in oldDict)
if (!newDict.ContainsKey(oldItem.Key) || newDict[oldItem.Key]!=oldItem.Value)
usersRemoved.Add(oldItem.Key, oldItem.Value);

Finally this is my implementation of GetInvolvementLogging:
(the implementation of the string builder method is irrelevant for my question here)
private string GetInvolvementLogging(ICollection<UserInvolvement> newInvolvement, ICollection<UserInvolvement> oldInvolvement)
{
//I defined the new and old dictionary's to focus on the relevant data inside UserInvolvement.
var newDict = newInvolvement.ToDictionary(x => x.UserID, x => (Involvement)x.Involvement);
var oldDict = oldInvolvement.ToDictionary(x => x.UserID, x => (Involvement)x.Involvement);
var intersection = newDict.Keys.Intersect(oldDict.Keys); //These are the id's of the users that were and remain involved.
var usersAdded = newDict.Keys.Except(intersection);
var usersRemoved = oldDict.Keys.Except(intersection);
var addedInvolvement = newDict.Where(x => usersAdded.Contains(x.Key)).ToDictionary(x => x.Key, x => x.Value);
var removedInvolvement = oldDict.Where(x => usersRemoved.Contains(x.Key)).ToDictionary(x => x.Key, x => x.Value);
//Check if the already involved users have a changed involvement.
foreach(var userId in intersection)
{
var newInvolvementFlags = newDict[userId];
var oldInvolvementFlags = oldDict[userId];
if ((int)newInvolvementFlags != (int)oldInvolvementFlags)
{
var xor = newInvolvementFlags ^ oldInvolvementFlags;
var added = newInvolvementFlags & xor;
var removed = oldInvolvementFlags & xor;
if (added != 0)
{
addedInvolvement.Add(userId, added);
}
if (removed != 0)
{
removedInvolvement.Add(userId, removed);
}
}
}
return GetInvolvementLogging(addedInvolvement, removedInvolvement);
}

Convert nested JSON to CSV

I am converting a nested JSON object with more than 10 levels to CSV file in C# .NET.
I have been using JavaScriptSerializer().Deserialize<ObjectA>(json) or XmlNode xml = (XmlDocument)JsonConvert.DeserializeXmlNode(json) to break down the object. With the objects I can further write into CSV file. However now the JSON object further expand. Most data that is not really in use so I would prefer a raw data dump.
Is that easier way I can just dump the data into csv format without declaring the structure?
Sample JSON
{
"F1":1,
"F2":2,
"F3":[
{
"E1":3,
"E2":4
},
{
"E1":5,
"E2":6
},
{
"E1":7,
"E2":8,
"E3":[
{
"D1":9,
"D2":10
}
]
},
]
}
And my expected CSV output is
F1,F2,E1,E2,D1,D2
1,2
1,2,3,4
1,2,5,6
1,2,7,8,9,10

There's an inconsistency in your request: you want a row to be generated for the root object, which has children, but you don't want a row to be generated for the "F3[2]" object, which also has children. So it sounds like your rule is, "print a row for an object with at least one primitive-value property, as long as that object is either the root object or does not have descendant objects with at a least one primitive-value property". That's a little tricky, but can be done with LINQ to JSON
var obj = JObject.Parse(json);
// Collect column titles: all property names whose values are of type JValue, distinct, in order of encountering them.
var values = obj.DescendantsAndSelf()
.OfType<JProperty>()
.Where(p => p.Value is JValue)
.GroupBy(p => p.Name)
.ToList();
var columns = values.Select(g => g.Key).ToArray();
// Filter JObjects that have child objects that have values.
var parentsWithChildren = values.SelectMany(g => g).SelectMany(v => v.AncestorsAndSelf().OfType<JObject>().Skip(1)).ToHashSet();
// Collect all data rows: for every object, go through the column titles and get the value of that property in the closest ancestor or self that has a value of that name.
var rows = obj
.DescendantsAndSelf()
.OfType<JObject>()
.Where(o => o.PropertyValues().OfType<JValue>().Any())
.Where(o => o == obj || !parentsWithChildren.Contains(o)) // Show a row for the root object + objects that have no children.
.Select(o => columns.Select(c => o.AncestorsAndSelf()
.OfType<JObject>()
.Select(parent => parent[c])
.Where(v => v is JValue)
.Select(v => (string)v)
.FirstOrDefault())
.Reverse() // Trim trailing nulls
.SkipWhile(s => s == null)
.Reverse());
// Convert to CSV
var csvRows = new[] { columns }.Concat(rows).Select(r => string.Join(",", r));
var csv = string.Join("\n", csvRows);
Console.WriteLine(csv);
Using
public static class EnumerableExtensions
{
// http://stackoverflow.com/questions/3471899/how-to-convert-linq-results-to-hashset-or-hashedset
public static HashSet<T> ToHashSet<T>(this IEnumerable<T> source)
{
return new HashSet<T>(source);
}
}
Which outputs:
F1,F2,E1,E2,D1,D2
1,2
1,2,3,4
1,2,5,6
1,2,7,8,9,10

I wrote this and it is working for me
Here we save all breadcrumps of object tree in headers with format prop_prop
And save jarray property objects in headers in format prop1
public Dictionary<string, string> ComplexJsonToDictionary(JObject jObject, Dictionary<string, string> result, string field)
{
foreach (var property in jObject.Properties())
{
var endField = field + (string.IsNullOrEmpty(field) ? "" : "_") + property.Name;
var innerDictionary = new Dictionary<string, string>();
try
{
var innerValue = JObject.Parse(Convert.ToString(property.Value));
result.AddOrOverride(ComplexJsonToDictionary(innerValue, innerDictionary, endField));
}
catch (Exception)
{
try
{
var innerValues = JArray.Parse(Convert.ToString(property.Value));
try
{
var i = 0;
foreach (var token in innerValues)
{
var innerValue = JObject.Parse(Convert.ToString(token));
result.AddOrOverride(ComplexJsonToDictionary(innerValue, innerDictionary, endField+i++));
}
}
catch (Exception)
{
result.Add(endField, string.Join(",", innerValues.Values<string>()));
}
}
catch (Exception)
{
result.Add(endField, property.Value.ToString());
}
}
}
return result;
}
Thanks for atantion and please write review if appropriate.

pass the json parsed jobject to static extension, then it will return the array of jobjects with flatten, then convert to csv. some of the source code is taken from the other stack overflow and other resources, I don't have reference.
public static IEnumerable<string> JsonToCsvRowsWithHierarchyHeaders(string jsonData)
{
if (jsonData.Trim().StartsWith("[") && jsonData.Trim().EndsWith("]"))
{
var startString = "{\"appendRoot\":";
var endString = "}";
jsonData = $"{startString}{jsonData}{endString}";
}
var jObject = JsonConvert.DeserializeObject<JObject>(jsonData);
var flattenJObjects = JsonParserExtensions.FlattenJsonGetJObjects(jObject).ToList();
var csvRows = new List<string>();
if (flattenJObjects.Any())
{
var firstRow = flattenJObjects.First().Children<JProperty>().Select(x => x.Name).ToList();
var header = string.Join(delimeter, firstRow).Replace("appendRoot_", "").ToLower();
csvRows.Add(header);
foreach (var flattenJObject in flattenJObjects)
{
var tokens = flattenJObject.Children<JProperty>();
if (firstRow.Count() != tokens.Count())
{
var missMatchPropertyValues = firstRow.Select(cell => tokens.FirstOrDefault(x => x.Name == cell))
.Select(value => value == null
? string.Empty
: CheckAndUpdateRowCellValueTextQualifier(JsonConvert.DeserializeObject<string>(value.Value.ToString(Newtonsoft.Json.Formatting.None))))
.ToList();
var rowString = string.Join(delimeter, missMatchPropertyValues);
csvRows.Add(rowString);
}
else
{
var rowValue = tokens.Select(token =>
CheckAndUpdateRowCellValueTextQualifier(
JsonConvert.DeserializeObject<string>(
token.Value.ToString(Newtonsoft.Json.Formatting.None))));
var rowString = string.Join(delimeter, rowValue);
csvRows.Add(rowString);
}
}
}
return csvRows;
}
private static string CheckAndUpdateRowCellValueTextQualifier(string value)
{
const string q = #"""";
if (!string.IsNullOrEmpty(value) && value.Contains('\"'))
{
return value;
}
if (!string.IsNullOrEmpty(value) && (value.Contains(',') ||
value.Contains('"') ||
value.Contains('\n') || value.Contains('\r')))
{
return $"{q}{value}{q}";
}
return value;
}
}
public static class JsonParserExtensions
{
public static IEnumerable<JObject> FlattenJsonGetJObjects(JObject jObject, string parentName = null)
{
if (!(parentName is null))
jObject = RenamePropertiesByHierarchyName(jObject, parentName);
var fields = jObject.Properties().Where(p => p.Value.GetType().Name == "JValue").ToList();
var objects = jObject.Properties().Where(p => p.Value.GetType().Name == "JObject").ToList();
var arrays = jObject.Properties().Where(p => p.Value.GetType().Name == "JArray").ToList();
var objectsArray = arrays.Where(array => array.Value.All(elements => elements.GetType().Name == "JObject")).ToList();
var valuesArray = arrays.Where(array => array.Value.All(elements => elements.GetType().Name == "JValue")).ToList();
var nestedObjects = ProcessNestedObjects(objects);
var joinedInnerObjects = nestedObjects.Any()
? nestedObjects.Select(innerObject => JoinJObject(new JObject(fields), innerObject))
: new List<JObject> { new JObject(fields) };
var arraysObjectList = GetJObjectsFromArrayOfJProperties(objectsArray);
var arraysValueList = GetJObjectsFromArrayOfValues(valuesArray);
var joinedAll = joinedInnerObjects.SelectMany(inner => JoinMultipleJObjects(arraysObjectList, arraysValueList, inner));
return joinedAll;
}
public static List<JObject> ProcessNestedObjects(List<JProperty> jObjects)
{
var processNestedObjects = new List<JObject>();
var renamedJObjects = jObjects?.Select(obj => RenamePropertiesByHierarchyName(obj.Value.ToObject<JObject>(), obj.Name)).ToList();
if (!(renamedJObjects?.Count > 0)) return processNestedObjects;
var renamed = renamedJObjects.Aggregate((acc, next) => JoinJObject(acc, next));
var nestedObjects = renamed.Properties().Where(p => p.Value.GetType().Name == "JObject").ToList();
var nestedArrays = renamed.Properties().Where(p => p.Value.GetType().Name == "JArray").ToList();
var nestedObjectsArray = nestedArrays.Where(array => array.Value.All(elements => elements.GetType().Name == "JObject")).ToList();
var nestedValuesArray = nestedArrays.Where(array => array.Value.All(elements => elements.GetType().Name == "JValue")).ToList();
nestedArrays.ForEach(p => renamed.Remove(p.Name));
nestedObjects.ForEach(p => renamed.Remove(p.Name));
var nestedObjectList = new List<JObject>();
var nestedMultipleObjectList = new List<JObject>();
foreach (var listJObjects in nestedObjects.Select(innerObject => FlattenJsonGetJObjects(innerObject.Value.ToObject<JObject>(), innerObject.Name)).ToList())
{
if (listJObjects.Count() > 1)
nestedMultipleObjectList.AddRange(listJObjects);
else
nestedObjectList.Add(listJObjects.First());
}
var jObjectsFromArrayOfJProperties = GetJObjectsFromArrayOfJProperties(nestedObjectsArray);
var jObjectsFromArrayOfValues = GetJObjectsFromArrayOfValues(nestedValuesArray);
var aggregate = nestedObjectList.Aggregate(renamed, (acc, next) => JoinJObject(acc, next));
var groupedNestedObjects = (nestedMultipleObjectList.Any()) ? nestedMultipleObjectList.Select(nested => JoinJObject(aggregate, nested))
: new List<JObject> { aggregate };
var groupedNestedObjectsList = groupedNestedObjects.Select(groupedNested => JoinMultipleJObjects(jObjectsFromArrayOfJProperties, jObjectsFromArrayOfValues, groupedNested));
processNestedObjects.AddRange(groupedNestedObjectsList.SelectMany(e => e));
return processNestedObjects;
}
public static List<JObject> JoinMultipleJObjects(List<JObject> nestedArraysObjectList, List<JObject> nestedArraysValueList, JObject groupedNestedObjects)
{
var result = new List<JObject>();
var joined = new List<JObject>();
if (!nestedArraysObjectList.Any())
joined.Add(groupedNestedObjects);
else
nestedArraysObjectList.ForEach(e => joined.Add(JoinJObject(groupedNestedObjects, e)));
result.AddRange(nestedArraysValueList.Any()
? nestedArraysValueList
.SelectMany(value => joined, (value, joinedItem) => JoinJObject(joinedItem, value)).ToList()
: joined);
return result;
}
public static List<JObject> GetJObjectsFromArrayOfJProperties(List<JProperty> nestedJProperties)
{
var fromArrayOfJProperties = new List<JObject>();
foreach (var jProperty in nestedJProperties)
{
var nestedArraysObjectList = new List<JObject>();
var name = jProperty.Name;
var jPropertyValue = jProperty.Value;
var renamedObjects = jPropertyValue?.Select(obj => RenamePropertiesByHierarchyName(obj.ToObject<JObject>(), name)).ToList();
foreach (var jObjects in renamedObjects.Select(innerObject => FlattenJsonGetJObjects(innerObject.ToObject<JObject>())))
{
nestedArraysObjectList.AddRange(jObjects);
}
if (fromArrayOfJProperties.Any() && nestedArraysObjectList.Any())
fromArrayOfJProperties = nestedArraysObjectList
.SelectMany(nested => fromArrayOfJProperties, (current, joined) => JoinJObject(joined, current)).ToList();
if (!fromArrayOfJProperties.Any())
fromArrayOfJProperties.AddRange(nestedArraysObjectList);
}
return fromArrayOfJProperties;
}
public static List<JObject> GetJObjectsFromArrayOfValues(List<JProperty> nestedValuesArray)
{
return (from innerArray in nestedValuesArray let name = innerArray.Name let values = innerArray.Value from innerValue in values select new JObject(new JProperty(name, innerValue.ToObject<JValue>()))).ToList();
}
public static JObject RenamePropertiesByHierarchyName(JObject jObject, string hierarchyName)
{
var properties = jObject.Properties().ToList().Select(p => new JProperty($"{hierarchyName}_{p.Name}", p.Value));
return new JObject(properties);
}
public static JObject JoinJObject(JObject parentJObject, JObject innerObject)
{
var joinJObject = new JObject
{
parentJObject.Properties(),
innerObject.Properties()
};
return joinJObject;
}
}

Dynamic Build lambda with generic type

How to build dynamic lambda function like below?
item => item.AnalyzeData.Any(subitem =>
subitem.DepartmentIDs.Any(subitem2 => subitem2 == "abc"))
AnalyzeData is custom class array type, DepartmentIDs is string array type
The class of AnalyzeData has DepartmentIDs Property
So how to dynamic generator the above lambda expression via expression tree.

After some research, the problem has been resolved. now share the solution code to others who has the some issue.
[TestMethod]
[Description("Dynamic generate lambda expression")]
public void DynamicGenerateMethod1()
{
Item item1 = new Item()
{
ItemID = "test1",
AnalyzeData = new ItemAnalyzeData[]
{
new ItemAnalyzeData()
{
DepartmentIDs = new []{"Department1","Department2"}
},
}
};
Item item2 = new Item()
{
ItemID = "test2",
AnalyzeData = new ItemAnalyzeData[]
{
new ItemAnalyzeData()
{
DepartmentIDs = new []{"Department3","Department4"}
},
}
};
Expression<Func<Item, bool>> expectedExpression =
item => item.AnalyzeData.Any(subitem => subitem.DepartmentIDs.Any(subitem2 => subitem2 == "Department3"));
//Get Enumerable.Any generic method
var anyPredicate =
typeof(Enumerable).GetMethods().First(m => m.Name == "Any" && m.GetParameters().Length == 2);
#region Build inner most lambda expression : subitem2 => subitem2 == "Department3"
var subitem2Para = Expression.Parameter(typeof(string), "subitem2");
var subitem2CompareValue = Expression.Equal(subitem2Para, Expression.Constant("Department3", typeof(string)));
var subitem2CompareFun = Expression.Lambda<Func<string, bool>>(subitem2CompareValue, subitem2Para);
#endregion
#region Build center lambda expression : subitem => subitem.DepartmentIDs.Any( ... )
var subitemPara = Expression.Parameter(typeof(ItemAnalyzeData), "subitem");
var departmentIDsAnyMethodCall = anyPredicate.MakeGenericMethod(typeof(string));
var subItemDepartmentIDsCall = Expression.Call(departmentIDsAnyMethodCall, Expression.Property(subitemPara, "DepartmentIDs"), subitem2CompareFun);
var subitemCompareFun = Expression.Lambda<Func<ItemAnalyzeData, bool>>(subItemDepartmentIDsCall, subitemPara);
#endregion
#region Build outer most lambda expression : item => item.AnalyzeData.Any( ... )
var itemExpr = Expression.Parameter(typeof(Item), "item");
var analyzeAnyMethodCall = anyPredicate.MakeGenericMethod(typeof(ItemAnalyzeData));
var itemAnalyzeDataCall = Expression.Call(analyzeAnyMethodCall, Expression.Property(itemExpr, "AnalyzeData"), subitemCompareFun);
var itemCompareFun = Expression.Lambda<Func<Item, bool>>(itemAnalyzeDataCall, itemExpr);
#endregion
var method = itemCompareFun.Compile();
var actualLambdaCode = itemCompareFun.ToString();
var expectLambdaCode = expectedExpression.ToString();
Assert.AreEqual(expectLambdaCode, actualLambdaCode);
Assert.IsFalse(method.Invoke(item1));
Assert.IsTrue(method.Invoke(item2));
}

Categories