We Keep Coding

How could I generate a c# .NET core function that can mimic the Include().ThenInclude.(Where) behaviour?

I'm trying to create a generic blazor component that and has a config object.
So far I've been able to successfully mimic .Where(), .Include().ThenInclude() separately, but now I need to add condition to the include.
My config file has the following structure
class Config
{
public List<string> includes {get;set;}
public List<Expression<Func<T,bool>>> conditions {get;set;}
}
database structure
For the example lets assume the following structure.
Dataset (1)<--(n) DatasetItem
DatasetItem (1)-->(1) Image
DatasetItem (1)-->(1) Label
Image (1)<--(n) Labels
Label(1)<--(n) Tags
where an image can be a part of more than one dataset and it's tags might or might not be included in the same dataset. Imagine an picture of a city park with dogs, people, and cars. Someone would want a dataset of vehicles and consider only the cars and ignore the labels about animals and people.
A Label include the coordinates and one or more TagNames (vehicle, car, red, fourWheeler, etc)
what works
And I can give conditions and includes this way:
var config = new Config<DatasetItem>()
{
includes = new() {"Image", "Image.Labels", "Image.Labels.Tags"}
conditions = new() {di => di.datasetId == 1, di => di.Label.width > 100}
}
the problems and limitations of this approach
For this to work I use the .Include(str) overload, but I don't like that approach since it is prone to fail.
The other problem with this approach is a limitation on the conditions.
For example, I can not add a condition to get only the Tags that belong to a specific category.
It seems that is not possible to add a condition that would give me this filter. Best I got was with the use of All or Any, but those do not give the desire result.
config.conditions.Add(
di => di.Images.Any(i => Labels.Any(l => l.Tags.Any(t => t.Name == "vehicle"))
)
what I'm looking for
Whith regular use of Include, obtaining the desire result would be like this.
DatasetItems.Include(di => di.Images)
.ThenInclude(i => i.Labels)
.ThenInclude(l => l.Tags.Where(t => t.Name == "vehicle"))
So, is there any way to achieve this?
I thing the solution would require to use Expression Trees, Reflection, and Recursive functions using MakeGenericMethod, but I havent found a way to achive this.
the ideal solution would be to find a structure like the one for conditions
List<Expression<Func<T,something>>> that lets me add the condition in this way
includes.add(di => di.Images.Labels.Tags.Where(t => t.Name == "vehicle"))
a failed attempt
I'm including a failed attempt I took to solve this, just in case this helps to narrow down the intention and for if someone can use it for solving the problem. This attempt is far from ideal.
property was a dot-separated string, like "Image.Labels.Tags" and filter was suppoused to be the Where expression.
This function does not compile.
public static IQueryable<T> FilteredInclude<T>(this IQueryable<T> query, string property, Func<dynamic, bool> filter) where T : class
{
var type = typeof(T);
var properties = property.Split('.');
var propertyInfo = type.GetProperty(properties[0]);
var parameter = Expression.Parameter(type, "x");
var propertyExpression = Expression.Property(parameter, propertyInfo);
var lambda = Expression.Lambda(propertyExpression, parameter);
var filteredQuery = query.Where(x => filter(propertyInfo.GetValue(x)));
if (properties.Length == 1)
{
var methodInfo = typeof(QueryableExtensions).GetMethod(nameof(Include));
methodInfo = methodInfo.MakeGenericMethod(type, propertyInfo.PropertyType);
var result = methodInfo.Invoke(null, new object[] { filteredQuery, lambda });
return (IQueryable<T>)result;
}
else
{
var nextProperty = string.Join(".", properties.Skip(1));
var result = filteredQuery.FilteredInclude(nextProperty, filter);
var thenIncludeMethod = typeof(Microsoft.EntityFrameworkCore.Query.Internal.IncludeExpression)
.GetMethod("ThenInclude")
.MakeGenericMethod(propertyInfo.PropertyType);
var resultWithThenInclude = thenIncludeMethod.Invoke(result, new object[] { lambda });
return (IQueryable<T>)resultWithThenInclude;
}
}

How to pass variables to Expression Func Projection before compile

I am trying to write Expression Functions for my projections. I found a good article about that link. But I couldn't figure out how can I pass variables to these functions.
How can I write a projection function for this one
int i = 3;
var query = await _db.Users
.FilterByName(name)
.Select(item => new SearchResultItemViewModel
{
Id = item.Id,
Article = item.FirstName + i.ToString()
});
}))
This one is working. In select SQL string has only Id and Firstname but I can't pass any variable.
var query = await _db.Users
.FilterByName(name)
.Select(item => SearchResultItemViewModel.Projection)
public static Expression<Func<ApplicationUser, SearchResultItemViewModel>> Projection
{
get
{
return item => new SearchResultItemViewModel
{
Id = item.Id,
Article = item.FirstName
};
}
}
This one is working only if you call compile and invoke. SQL string has all rows. Leading to bad performance
var query = await _db.Users
.FilterByName(name)
.Select(item => SearchResultItemViewModel.Projection.Compile().Invoke(item,i))
public static Expression<Func<ApplicationUser,int, SearchResultItemViewModel>> Projection
{
get
{
return( item,i) => new SearchResultItemViewModel
{
Id = item.Id,
Article = item.FirstName + i.ToString()
};
}
}

I don't use EF so this might vary in your specific use case, but this seems to be a question about LINQ Expressions more than anything else.
The first big problem is that you are trying to use an Expression<Func<ApplicationUser, int, SearchResultItemViewModel>> where you really meant Expression<Func<ApplicationUser, SearchResultItemViewModel>> and that's not going to do what you want. Instead of binding to a variable you're invoking the indexed variant of Select. So instead of getting the same value of i for all rows you get the index of the row.
When you create an expression that references a variable, one of two things happens. For local variables (and parameters) the value is copied to an anonymous class instance which is bound to the expression, so you can't change it afterwards. For other variables the expression contains a reference to the variable itself, as well as the containing object for non-static variables.
Which means that you could in principle use a static variable to alter the parameter and never have to recreate the projection expression. There are certainly times when this is useful.
On the other hand, your code above is creating a new instance each time you access the Projection property. So why not just change it to a function and generate the expression you need when you need it?
public static Expression<Func<ApplicationUser, SearchResultItemViewModel>> Projection(int parm)
=> item => new SearchResultItemViewModel
{
Id = item.Id,
Article = item.FirstName + parm.ToString()
};
Each time you invoke the method you'll get back an expression that uses the specified value.
Or you could use an expression visitor to take a template expression and modify the constants in it to whatever you need at the time. Fun, but a bit beyond scope here I think.

What is the "=>" sign in LINQ queries?

It's amazing how little information there is on this. I found tons of tutorials explaining LINQ, but they don't explain this particular operator:
var Results = UserFavoritesContext.UserFavorites.Select(color => color.FavoriteColor);
"x => x.y"
Can someone please explain how this works? I get the general syntax and am able to use it to make queries, but it's like doing something without knowing what you're doing.

Suppose you have a list of people, and you want to iterate over them. You would write something like:
foreach(var person in people)
{
//do something to person
}
Note how you yourself chose the name person. It could've been any word, but you basically said "process every single item of the list as my person variable".
Now look at this LINQ query:
filteredPeopleList = people.Where(person => person.Name == "John");
Again, you basically chose person as a placeholder name for every object in the original list (one at a time). The above Linq query is equivalent to
foreach(var person in people)
{
if(person.Name == "John")
{
filteredPeopleList.Add(person);
}
}
To me, x => x.y is basically saying "for every variable we process (let's call it x), please perform the following operation on it (x.y, get the y property)"
I hope that explains it.
Edit
As a commenter who now deleted his comment mentioned, this isn't exclusively used in LINQ. A lambda expression doesn't have to iterate over an IEnumerable, it can be used to process a single item.
However, LINQ is by far the most common place to encounter lambdas, and I find their use very similar to the foreach loop, which is why I picked this example.

The => operator is used in lambda expressions.
The best way to think of this is a type of syntax for writing a function, where the left side of the operator is the parameters for the function and the right side is the body of the function e.g. this is a valid use of a lambda expression where its being used like a function:
Func<int, int> incrementFunction = i => i + 1;
int incrementedNumber = incrementFunction(1);
The name i in this case is an arbitrary variable name e.g. just like you would name a functions input parameter. Notice I didnt need to declare the input parameters type because the compiler will infer it. Also notice I dont need to include the "return" keyword or enclose the function body in a code block. It doesn't necessarily need any parameters either e.g.
Action myAction = () => Console.Write("something");
myAction();
When you use it in a linq expression, think of it as though the lambda function is being called on every element in the collection (which I believe is exactly what happens with linq to objects).

It's the syntax of a Lambda expression. If it helps you to remember... in a nutshell, the argument to pass (the parameter) is on the left of the => and the method(s) that use it are on the right hand side of it.
I hope this short summary explains it enough :)

That is a lambda expression, and it can be used as a selector from an object
You can conditionally select (or another operation orderby, count, etc) when the expression is true. For example:
Say you had a list of people and their details: ID, Name, City and Profession.
You could select a single person by using lambda to search for their specific ID:
public Person GetByID(int id)
{
Person selectedPerson = people.SingleOrDefault(person => person.ID == id);
return selectedPerson;
}
Same could be applied for a select on a city, this would be:
public List<Person> GetByCity(string city)
{
List<Person> selectedPeople = people.where(person => person.City == city);
return selectedPeople;
}
The lamda expression is where you place your operation variable, so in these cases the condition upon which you select the data. You can use it as a orderby variable much in the same way, in the next example I use two lamdas to perform two seperate functions
public List<Person> GetByCity(string city)
{
List<Person> selectedPeople = people.where(person => person.city == city)
.OrderByDescending(person => person.Name);
return selectedpeople;
}
I hope this helps at all

x => x.y is Lambda Expression introduced with C# 3.0. The general syntax is
parameter => executioncode
The Lambda Expression has 3 parts:
x on left hand side is the parameter.This can be a variable,delegate or an anonymous function.
=> read as "goes to", which acts as separator
x.y is an Expression to be evaluated.
For example, the lambda expression x => x * x specifies a parameter that’s named x and returns the value of x squared (source: MSDN).
Hope this would help you.

They're called Lamda expression:
https://msdn.microsoft.com/en-us/library/bb397687.aspx
var Results = UserFavoritesContext.UserFavorites.Select(color => color.FavoriteColor);
is similar to:
List<UserFavorite> Results = new List<UserFavorite>();
foreach(var item in UserFavorites)
{
Results.Add(item.FavoriteColor);
}

This is not specific to Linq.
It is the way you write a lambda expression.
The arrow => is the delimiter between your lambda parameters and its body.

What is the best way to create strongly typed LINQ queries from some given strings, via reflection

I'm using EF5, unit of work, and repository pattern.
I want to define some limitations on accessing data for specified users.
In database I've designed a table to keep my entity names and their properties which is called EntityProperties, and another table to keep the values of those properties which is called PropertyValues, and each EntityProperty has one or more PropertyValues.
In business layer when user requests data, if any limitation is defined for him, some conditions should be added to the linq statement. What I do is to get the list of entity names and their propeties and values by 'userId', then I add some 'Where' clause to the linq query.
However, the entity names and their properties are of type "String", thus I should use Reflection to manage them.
But I don't know this part, and I don't know how to create LINQ where clause from a given set of condition strings.
For example, let's suppose that a user requests the list orders, and user id is 5. I first query those access limitation tables, and the result is:
"Order", "Id", "74"
"Order", "Id", "77"
"Order", "Id", "115"
It means that this user should only see these three orders, while in Orders table, we have more orders.
So, if I want to use a LINQ query to get orders, like:
var orders = from order in Context.Orders
I need to turn it into something like:
var orders = from order in Context.Orders
// where order id should be in 74,77,115
However, getting to Order entity and Id property from "Order" and "Id" strings requires reflection. Thus two questions:
What is the best way to get strongly typed from strings?
Is there a better way for me to do this, with better performance?

Ok. With the comments, we might go for something like that (assuming you have a navigation property in EntityProperties table, which is a collection of PropertyValues, and named PropertyValueList (if you don't have, just do a join instead of using Include).
Here is some sample code, really rustic, working only with Int32 properties, but this might be the start of a solution.
You may also look at PredicateBuilder...
Anyway
I use an "intermediate class" Filter.
public class Filter
{
public string PropertyName { get; set; }
public List<string> Values { get; set; }
}
Then an helper class, which will return an IQueryable<T>, but... filtered
public static class FilterHelper {
public static IQueryable<T> Filter(this IQueryable<T> queryable, Context context, int userId) {
var entityName = typeof(T).Name;
//get all filters for the current entity by userId, Select the needed values as a `List<Filter>()`
//this could be done out of this method and used as a parameter
var filters = context.EntityProperties
.Where(m => m.entityName == entityName && m.userId = userId)
.Include(m => m.PropertyValueList)
.Select(m => new Filter {
PropertyName = m.property,
Values = m.PropertyValueList.Select(x => x.value).ToList()
})
.ToList();
//build the expression
var parameter = Expression.Parameter(typeof(T), "m");
var member = GetContains( filters.First(), parameter);
member = filters.Skip(1).Aggregate(member, (current, filter) => Expression.And(current, GetContains(filter, parameter)));
//the final predicate
var lambda = Expression.Lambda<Func<T, bool>>(member, new[] { parameter });
//use Where with the final predicate on your Queryable
return queryable.Where(lambda);
}
//this will build the "Contains" part
private static Expression GetContains(Filter filter, Expression expression)
{
Expression member = expression;
member = Expression.Property(member, filter.PropertyName);
var values = filter.Values.Select(m => Convert.ToInt32(m));
var containsMethod = typeof(Enumerable).GetMethods().Single(
method => method.Name == "Contains"
&& method.IsGenericMethodDefinition
&& method.GetParameters().Length == 2)
.MakeGenericMethod(new[] { typeof(int) });
member = Expression.Call(containsMethod, Expression.Constant(values), member);
return member;
}
}
usage
var orders = from order in Context.Orders
select order;
var filteredOrders = orders.Filter(Context, 1);//where 1 is a userId

My answer depends on whether you are happy to alter your access model slightly. I've got a similar situation in an application that I have written and personally I don't like the idea of having to rely on my calling code to correctly filter out the records based on the users authentication.
My approach was to use an OData service pattern to call into my Entity Framework, each of the repositories are exposed via OData independently.
An OData (WCFDataService) has QueryInterceptors which perform on the fly filtering of your data when a query is made. Thus if you asked the OData repository for context.Orders(o => o.Id) you'd only see the orders that that user was permitted to see with no additional clauses.
A good link to the basics is found here but it requires some work to manage the calling user and provide the filtering that you may require. You can provide the query interceptor at every record level.

Insert Linq query part inside existing query

Having the following variable which stores an IQueryable:
var mainQuery = session
.Query<Employee>()
.Select(e => new
{
e.Name,
e.Address
});
And a method which takes that IQueryable as parameter.
public DataTable GetData(IQueryable query)
{
...
}
How can I write code inside GetData() that adds OrderBy() before the Select()?
Final value of query should look like it was built using following Linq expression:
var query = session
.Query<Employee>()
.OrderBy(e => e.Age)
.Select(e => new
{
e.Name,
e.Address
});
This is required because if I add the OrderBy() after Select() I will only be able to sort by the members of the anonymous type (name, address). If an employee would also have and age, I could not sort by it when placing OrderBy() after Select().
Thanks for your help!
UPDATE:
GetData has no knowledge of the structure of the query except that it ends with a Select.
And it has to have that exact signature public DataTable GetData(IQueryable query) - no extra parameters.
What is required it to modify the existing query inside the method and add OrderBy before the Select.
When there will be a correct answer I will accept and vote for it.

Why not just apply the select() after the call to GetData()?
var mainQuery = session.Query<Employee>();
this.GetData(mainQuery);
mainQuery.OrderBy(x => x.Age)
.Select(x => new
{
x.Name,
x.Address
});
Linq expression trees are immutable (source). You have to create a copy of the tree in parts to modify it.
Update: Keep in mind your pattern is trying to separate data access from presentation (or at least that is how it reads). Ordering is a matter of presentation. You may have multiple clients wanting to use GetData to fetch data but each of those clients might want the data to be sorted differently. Your original query projected after the call to GetData anyway so it makes sense to order with the projection.
Now if you REALLY want to order inside the method without changing its contract, you have to walk the expression tree of the linq query and rebuild it from scratch injecting the ordering in the right place. Expression trees are immutable and cannot be modified in-place.

Consider creating ViewModel or DTO for Employee and not to pass anonymous objects around. But anyway you can pass selector into GetData and apply it after sorting Employee
var mainQuery = session.Query<Employee>();
GetData(mainQuery, e => new { e.Name, e.Address });
//object is used because you create anonymous objects and pass them around
public DataTable GetData(IQueryable query,
Expression<Func<Employee, object>> selector)
{
return query.OrderBy(e => e.Age)
.Select(selector)
//...
}