We Keep Coding

Runtime Sorting and Types with Linq

I have seen numerous articles which all seem to nibble around the issue I am having but none have provided an actual resolution to the problem. Which is to say they all get me to about 98% solution only to fail in some small detail.
I have an IEnumerable that is collected at run time. This IEnumerable could be of ANYTHING. I will not know until runtime. I need to sort it however based on a list of propertyNames and sort directions in a List of KeyValuePair objects provided as an argument.
public static void SortData(IEnumerable<dynamic> dataToSort, List<KeyValuePair<string, string>> sortArgs)
{
}
I first get the Type of the IEnumerable. I have created a method for this. I won't get into this detail here. It's been tested and does return the proper Type.
public static void SortData(IEnumerable<dynamic> dataToSort, List<KeyValuePair<string, string>> sortArgs)
{
Type dataType = TypeService.GetType(data);
}
I then attempt to create an initial IOrderedEnumerable that I can apply the sortArgs sort to.
public static void SortData(IEnumerable<dynamic> dataToSort, List<KeyValuePair<string, string>> sortArgs)
{
Type dataType = TypeService.GetType(dataToSort);
// Create IOrderedEnumerable
//
var query = from dataItem in dataToSort
orderby // ?????
select dataItem;
// apply sortArgs to IOrderedEnumerable
//
for(int argIDX = 0; argIDX < sortArgs.Count; argIDX++)
{
var arg = sortArgs[argIDX];
var sortField = arg.Key.Trim();
var sortDirection = arg.Value.Trim().ToUpper();
if(argIDX == 0)
{
if(sortDirection == "DESC")
{
query = query.OrderByDescending(e => e.GetType().GetProperty(sortField).GetValue(e));
}
else
{
query = query.OrderBy(e => e.GetType().GetProperty(sortField).GetValue(e));
}
}
else
{
if(sortDirection == "DESC")
{
query = query.ThenByDescending(e => e.GetType().GetProperty(sortField).GetValue(e));
}
else
{
query = query.ThenBy(e => e.GetType().GetProperty(sortField).GetValue(e));
}
}
}
// After applying the sort retreive the contents
//
dataToSort = query.ToList();
}
It seems by this point I have all the information I should need to sort the original dataToSort argument. But defining a property to initialize the IOrderedEnumerable is eluding me.
I have tried a number of different techniques I have read about ...
var query = from dataItem in dataToSort
orderby ( X => 1) //ERR: The type of one of the expressions in the OrderBy clause is incorrect. Type inference failed in the call to 'OrderBy'.
select dataItem;
I have tried to create a new Typed list (since I know the type) so that the expressions in the OrderBy clause could be inferred more accurately.
var typedDataToSort = new List<dataType>(); //ERR: dataType is a variable used like a type
foreach(var item in dataToSort)
{
typedDataToSort.Add( item );
}
I have tried to get the PropertyInfo for a property to sort on..
PropertyInfo propInfo = dataType.GetProperty(dataValueField);
var query = from dataItem in dataToSort
orderby (x => propInfo.GetValue(x, null)) //ERR: The type of one of the expressions in the OrderBy clause is incorrect. Type inference failed in the call to 'OrderBy'. Of couse this could not work since `dataType` is an INSTANCE of
select dataItem;
I have just run out of ideas.

To create an IOrderedIEnumerable from a List<T> without manipulating the order you can just execute a noop sort by doing something like this:
var myList = new List<String>(){"test1", "test3", "test2"}
IOrderedIEnumerable<String> query = myList.OrderBy(x => 1);
The same goes for an IEnumerable - however you must ensure that the underlaying type of IEnumerable provides stable order for each iteration.

Dynamically selecting fields in a linq subquery

I am trying to dynamically filter the fields in a select clause similar to what was requested by a user as was described in the post dynamically selecting fields in a linq query. (I describe my problem and fully below so you don't need to go to the link)
What I am trying to do in short is to define a query and then at runtime determine what fields are part of the select statement and which are filterd out.
QueryFilter filter = new QueryFilter {
ShowClientInfo = false,
ShowPackaging = true,
ShowName = true,
ShowWeight = false
};
//Below is a sample linq query that works without desired filtering.
// Comments are to the right of the fields I want filterd with the object above
var testQuery = db.Client
.Where(c => c.ID == clientId)
.Select(c =>
new
{
ClientInfo = c, // <== Supress this
Packaging = c.ClientPackaging // <== Show this
.Where(cp => !cp.IsDeleted)
.Select(cp =>
new
{
Name = cp.Name, // <== Show this
Weight = cp.Weight // <== Suppress this
}
).ToList()
}
).FirstOrDefault();
The first answer of the related question was not sufficient, so I am not going to go into it further. It is insufficient because the linq statement would still query the database for specific fields, and would only just not assign the value on return.
The second awesome answer by Ivan Stoev is what I am trying to have expanded upon. This answer prevents the fields from even being part of the query to the database and is preferred. But this only works for the outermost select of the linq query. The part I am having trouble with is creating a helper method that will work on subqueries in the select statement.
The reason for the trouble, is the first use of the overload is of type IQueryable, the nested queries in the select are of type IEnumerable and thus will not map to the current extension of the answer.
Extension from other post mentioned above that works for the outermost select statement
public static class MyExtensions
{
public static IQueryable<TResult> Select<TSource, TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector, object options)
{
var memberInit = (MemberInitExpression)selector.Body;
var bindings = new List<MemberBinding>();
foreach (var binding in memberInit.Bindings)
{
var option = options.GetType().GetProperty("Show" + binding.Member.Name);
if (option == null || (bool)option.GetValue(options)) bindings.Add(binding);
}
var newSelector = Expression.Lambda<Func<TSource, TResult>>(
Expression.MemberInit(memberInit.NewExpression, bindings), selector.Parameters);
return source.Select(newSelector);
}
}
Here is my sample query that works with the first Select, but the nested select statements do NOT work as they need to be an IEnumerable extension.
public class QueryFilter
{
public bool ShowClientInfo {get;set;}
public bool ShowPackaging {get;set;}
public bool ShowName {get;set;}
public bool ShowWeight {get;set;}
}
QueryFilter filter = new QueryFilter {
ShowClientInfo = false,
ShowPackaging = true,
ShowName = true,
ShowWeight = false
};
var testQuery = db.Client
.Where(c => c.ID == clientId)
.Select(c =>
new
{
ClientInfo = c,
Packaging = c.ClientPackaging
.Where(cp => !cp.IsDeleted)
.Select(cp => // <==== This select does not work
new
{
Name = cp.Name,
Weight = cp.Weight,
packages = cp.ShipmentPackage
.Select(sp => // <==== Neither does this
new
{
Dimension_x = sp.Dimension_x,
Dimension_y = sp.Dimension_y,
Dimension_z = sp.Dimension_z
}, filter //<== additional filter parameter for the select statement
).ToList()
}, filter //<== additional filter parameter for the select statement
).ToList()
}, filter //<== additional filter parameter for the select statement
).FirstOrDefault();
The error I am getting on the two nested queries in the .Select is:
Method 'Select' has 1 parameter(s) but is invoked with 2 argument(s)
I need to create an IEnumerable version of the method that would work in this case.
How would I create the method as an IEnumerable?

Dynamically build select list from linq to entities query

I'm looking for a way to dynamically create a select list from a iQueryable object.
Concrete example, i want to do something like the following:
public void CreateSelectList(IQueryable(of EntityModel.Core.User entities), string[] columns)
{
foreach(var columnID in columns)
{
switch(columnID)
{
case "Type":
SelectList.add(e => e.UserType);
break;
case "Name":
SelectList.add(e => e.Name);
break;
etc....
}
}
var selectResult = (from u in entities select objSelectList);
}
So all properties are known, i however don't know beforehand what properties are to be selected. That will be passed via the columns parameter.
I know i'm going to run into issues with the type of the selectResult type, because when the select list is dynamic, the compiler doesn't know what the properties of the anonymous type needs to be.
If the above is not possible: The scenario I need it for is the following:
I'm trying to create a class that can be implemented to display a paged/filtered list of data. This data can be anything (depends on the implementations).The linq used is linq to entities. So they are directly linked to sql data. Now i want to only select the columns of the entities that i am actually showing in the list. Therefore i want the select to be dynamic. My entity might have a hundred properties, but if only 3 of them are shown in the list, i don't want to generate a query that selects the data of all 100 columns and then only uses 3 of them. If there is a different approach that I haven't thought of, I'm open to ideas
Edit:
Some clarifications on the contraints:
- The query needs to work with linq to entities (see question subject)
- an entity might contain 100 columns, so selecting ALL columns and then only reading the ones i need is not an option.
- The end user decides what columns to show, so the columns to select are determined at run time
- i need to create a SINGLE select, having multiple select statements means having multiple queries on the database, which i don't want

Dynamic select expression to a compile time known type can easily be build using Expression.MemberInit method with MemberBindings created using the Expression.Bind method.
Here is a custom extension method that does that:
public static class QueryableExtensions
{
public static IQueryable<TResult> Select<TResult>(this IQueryable source, string[] columns)
{
var sourceType = source.ElementType;
var resultType = typeof(TResult);
var parameter = Expression.Parameter(sourceType, "e");
var bindings = columns.Select(column => Expression.Bind(
resultType.GetProperty(column), Expression.PropertyOrField(parameter, column)));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var selector = Expression.Lambda(body, parameter);
return source.Provider.CreateQuery<TResult>(
Expression.Call(typeof(Queryable), "Select", new Type[] { sourceType, resultType },
source.Expression, Expression.Quote(selector)));
}
}
The only problem is what is the TResult type. In EF Core you can pass the entity type (like EntityModel.Core.User in your example) and it will work. In EF 6 and earlier, you need a separate non entity type because otherwise you'll get NotSupportedException - The entity or complex type cannot be constructed in a LINQ to Entities query.
UPDATE: If you want a to get rid of the string columns, I can suggest you replacing the extension method with the following class:
public class SelectList<TSource>
{
private List<MemberInfo> members = new List<MemberInfo>();
public SelectList<TSource> Add<TValue>(Expression<Func<TSource, TValue>> selector)
{
var member = ((MemberExpression)selector.Body).Member;
members.Add(member);
return this;
}
public IQueryable<TResult> Select<TResult>(IQueryable<TSource> source)
{
var sourceType = typeof(TSource);
var resultType = typeof(TResult);
var parameter = Expression.Parameter(sourceType, "e");
var bindings = members.Select(member => Expression.Bind(
resultType.GetProperty(member.Name), Expression.MakeMemberAccess(parameter, member)));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var selector = Expression.Lambda<Func<TSource, TResult>>(body, parameter);
return source.Select(selector);
}
}
with sample usage:
var selectList = new SelectList<EntityModel.Core.User>();
selectList.Add(e => e.UserType);
selectList.Add(e => e.Name);
var selectResult = selectList.Select<UserDto>(entities);

What you are going for is possible, but it's not simple. You can dynamically build EF queries using the methods and classes in the System.Linq.Expressions namespace.
See this question for a good example of how you can dynamically build your Select expression.

I believe this is what you need:
var entities = new List<User>();
entities.Add(new User { Name = "First", Type = "TypeA" });
entities.Add(new User { Name = "Second", Type = "TypeB" });
string[] columns = { "Name", "Type" };
var selectResult = new List<string>();
foreach (var columnID in columns)
{
selectResult.AddRange(entities.Select(e => e.GetType().GetProperty(columnID).GetValue(e, null).ToString()));
}
foreach (var result in selectResult)
{
Console.WriteLine(result);
}
This code outputs:
First
Second
TypeA
TypeB
UPDATE (according to comments)
// initialize alist of entities (User)
var entities = new List<User>();
entities.Add(new User { Name = "First", Type = "TypeA", SomeOtherField="abc" });
entities.Add(new User { Name = "Second", Type = "TypeB", SomeOtherField = "xyz" });
// set the wanted fields
string[] columns = { "Name", "Type" };
// create a set of properties of the User class by the set of wanted fields
var properties = typeof(User).GetProperties()
.Where(p => columns.Contains(p.Name))
.ToList();
// Get it with a single select (by use of the Dynamic object)
var selectResult = entities.Select(e =>
{
dynamic x = new ExpandoObject();
var temp = x as IDictionary<string, Object>;
foreach (var property in properties)
temp.Add(property.Name, property.GetValue(e));
return x;
});
// itterate the results
foreach (var result in selectResult)
{
Console.WriteLine(result.Name);
Console.WriteLine(result.Type);
}
This code outputs:
First
TypeA
Second
TypeB

Use reflection to make dynamic LINQ statements in C#

If I have a LINQ statement like
x = Table.SingleOrDefault(o => o.id == 1).o.name;
how can I replace "id" and "name" with passed in variables using reflection? I keep getting object reference not set to instance of an object errors when I try. My attempts are things like
x = (string)Table.SingleOrDefault(o => (int?)o.GetType().GetProperty(idString)
.GetValue(o, null) == 1).GetType().GetField(nameString).GetValue(x);
Any help would be great. Thanks.

You should use Expression Trees instead of reflection. It will perform better, and you'll be able to use it with both LINQ to Objects and LINQ to SQL/Entities.
var source = new List<Test> { new Test { Id = 1, Name = "FirsT" }, new Test { Id = 2, Name = "Second" } };
var idName = "Id";
var idValue = 1;
var param = Expression.Parameter(typeof(Test));
var condition =
Expression.Lambda<Func<Test, bool>>(
Expression.Equal(
Expression.Property(param, idName),
Expression.Constant(idValue, typeof(int))
),
param
).Compile(); // for LINQ to SQl/Entities skip Compile() call
var item = source.SingleOrDefault(condition);
then, you can get Name property using reflection (you'll do it just once, so it's fine to do it using reflection.
var nameName = "Name";
var name = item == null ? null : (string) typeof(Test).GetProperty(nameName).GetValue(item);
Test class is defined as
public class Test
{
public int Id { get; set; }
public string Name { get; set; }
}

You can't use reflection, but you can use dynamic Linq as described here. If you are using Entity Framework for this, you should also be able to use Entity SQL, which is basically a hard-coded SQL string.

Combining Tables With Different Data Using Linq in MVC?

I have Two classes Named OfflineOrderLineItem.cs and OnlineOrderLineItem.cs both have diff Order table named offline and Online
In that i want to Combine the two tables data to search and Display the Fields from both tables
How to do that using linq in mvc4 ??? any idea.....
public virtual IPagedList<OnlineOrderLineItem> SearchOrderLineItems(string PoNumber)
{
var query1 = (from ol in _offlineOrderLineItemRepository.Table
select new
{
ol.Name
}).ToList();
var query2 = (from opv in _onlineOrderLineItemRepository.Table
select new
{
opv.Name
}).ToList();
var finalquery = query1.Union(query2);
if (!String.IsNullOrWhiteSpace(Name))
finalquery = finalquery.Where(c => c.Name == Name);
var orderlineitems = finalquery.ToList(); //its not working it throw a error
return new PagedList<OnlineOrderLineItem>(orderlineitems);//error
}
Error
cannot convert from 'System.Collections.Generic.List<AnonymousType#1>'
to 'System.Linq.IQueryable<Nop.Core.Domain.Management.OnlineOrderLineItem>'
to 'System.Linq.IQueryable<Nop.Core.Domain.Management.OnlineOrderLineItem>'

query1 and query2 are lists of an anonymous type with a single property of type string. (I assmume the ol.Name and opv.Name are strings.) Hence finalQuery and orderlineitems are collections of this anonymous as well. By specifying PagedList<T> you require that the collection passed into the constructor is an enumeration of type T. T is OnlineOrderLineItem, but the enumeration passed into the constructor is the anonymous type which is a different type. Result: compiler error.
To solve the problem I suggest that you define a named helper type that you can use to union the two different types OfflineOrderLineItem and OnlineOrderLineItem:
public class OrderLineItemViewModel
{
public int Id { get; set; }
public string PoNumber { get; set; }
public string Name { get; set; }
// maybe more common properties of `OfflineOrderLineItem`
// and `OnlineOrderLineItem`
}
Then your SearchOrderLineItems method should return a paged list of that helper type:
public virtual IPagedList<OrderLineItemViewModel> SearchOrderLineItems(
string PoNumber)
{
var query1 = from ol in _offlineOrderLineItemRepository.Table
select new OrderLineItemViewModel
{
Id = ol.Id,
PoNumber = ol.PoNumber,
Name = ol.Name,
// maybe more properties
};
// don't use ToList here, so that the later Union and filter
// can be executed in the database
var query2 = from opv in _onlineOrderLineItemRepository.Table
select new OrderLineItemViewModel
{
Id = opv.Id,
PoNumber = opv.PoNumber,
Name = opv.Name,
// maybe more properties
};
// don't use ToList here, so that the later Union and filter
// can be executed in the database
var finalquery = query1.Union(query2);
// again no ToList here
if (!string.IsNullOrWhiteSpace(PoNumber))
finalquery = finalquery.Where(c => c.PoNumber == PoNumber);
var orderlineitems = finalquery.ToList(); // DB query runs here
return new PagedList<OrderLineItemViewModel>(orderlineitems);
}
It is important to use ToList only at the very end of the query. Otherwise you would load the whole tables of all OnlineOrderLineItems and all OfflineOrderLineItems into memory and then filter out the items with the given PoNumber in memory which would be a big overhead and performance desaster.

Instead of
var orderlineitems = finalquery.ToList();
Try
var orderlineitems = finalquery.AsQueryable();
From https://github.com/TroyGoode/PagedList/blob/master/src/PagedList/PagedList.cs, PagedList takes a IQueryable<T>
Queryable.AsQueryable<TElement> Method