I have this idea to create a "list" of IQueryables that do different kinds of operations.
So basically:
var query1 = Enumerable.Empty<Person>().AsQueryable().Where(e => e.Name == "Ronald");
var query2 = Enumerable.Empty<Person>().AsQueryable().Where(e => e.Age == 43);
var query3 = Enumerable.Empty<Person>().AsQueryable().Select(e => e.EyeColor);
var listOfQueries = new List<IQueryable<Person>
{
query1,
query2,
query3
};
Now, I also have this DbSet full of "Persons" and I would like to "apply" all my queries against that DbSet. How would I do that? Is it even possible?
An updated example:
var personQueryFactory = new PersonQueryFactory();
var personQueryByFirstname = personQueryFactory.CreateQueryByFirstname("Ronald"); //Query by Firstname.
var personQueryByAge = personQueryFactory.CreateQueryByAge(42); //Query by age.
var personQueryByHasChildWithAgeOver = personQueryFactory.CreateQueryByChildAgeOver(25); //Query using a "join" to the child-relationship.
var personQuerySkip = personQueryFactory.Take(5); //Only get the 5 first matching the queries.
var personQuery = personQueryFactory.AggregateQueries //Aggragate all the queries into one single query.
(
personQueryByFirstname,
personQueryByAge,
personQueryByHasChildWithAgeOver,
personQuerySkip
);
var personSurnames = personsService.Query(personQuery, e => new { Surname = e.Surname }); //Get only the surname of the first 5 persons with a firstname of "Ronald" with the age 42 and that has a child thats over 25 years old.
var personDomainObjects = personsService.Query<DomainPerson>(personQuery); //Get the first 5 persons as a domain-object (mapping behind the "scenes") with a firstname of "Ronald" with the age 42 and that has a child thats over 25 years old.
var personDaos = personsService.Query(personQuery); //Get the first 5 persons as a DAO-objects/entityframework-entities with a firstname of "Ronald" with the age 42 and that has a child thats over 25 years old.
The reason for doing this would be to create a more "unified" way of creating and re-using predefined queries, and then to be able to execute them against the DbSet and return the result as a domain-object and not an "entity-framework-model/object"
This is possible, but you need to reframe your approach slightly.
Storing the filters
var query1 = Enumerable.Empty<Person>().AsQueryable().Where(e => e.Name == "Ronald");
var query2 = Enumerable.Empty<Person>().AsQueryable().Where(e => e.Age == 43);
var query3 = Enumerable.Empty<Person>().AsQueryable().Select(e => e.EyeColor);
Reading your intention, you don't actually want to handle IQueryable objects, but rather the parameter that you supply to the Where method.
Edit: I missed that the third one was a Select(), not a Where(). Ive adjusted the rest of the answer as if this had also been a Where(). See the comments below for my response as to why you can't mix Select() with Where() easily.
Func<Person,bool> filter1 = (e => e.Name == "Ronald");
Func<Person,bool> filter2 = (e => e.Age == 43);
Func<Person,bool> filter3 = (e => e.EyeColor == "Blue");
This stores the same information (filter criteria), but it doesn't wrap each filter in an IQueryable of its own.
A short explanation
Notice the Func<A,B> notation. In this case, A is the input type (Person), and B is the output type (bool).
This can be extended further. A Func<string,Person,bool> has two input parameters (string, Person) and one output parameter (bool). A usage example:
Func<string, Person, bool> filter = (inputString, inputPerson) => inputString == "TEST" && inputPerson.Age > 35;
There is always one output parameter (the last type). Every other mentioned type is an input parameter.
Putting the filters in a list
Intuitively, since Func<Person,bool> represents a single filter; you can represent a list of filters by using a List<Func<Person,bool>>.
Nesting generic types get a bit hard to read, but it does work just like any other List<T>.
List<Func<Person,bool>> listOfFilters = new List<Func<Person,bool>>()
{
(e => e.Name == "Ronald"),
(e => e.Age == 43),
(e => e.EyeColor == "Blue")
};
Executing the filters
You're in luck. Because you want to apply all the filters (logical AND), you can do this very easily by stacking them:
var myFilteredData = myContext.Set<Person>()
.Where(filter1)
.Where(filter2)
.Where(filter3)
.ToList();
Or, if you're using a List<Func<Person,bool>>:
var myFilteredData = myContext.Set<Person>().AsQueryable();
foreach(var filter in listOfFilters)
{
myFilteredData = myFilteredData.Where(filter);
}
Fringe cases
However, if you were trying to look for all items which fit one or more filters (logical OR), it becomes slightly more difficult.
The full answer is quite complicated.. You can check it here.
However, assuming you have the filters set in known variables, there is a simpler method:
Func<Person, bool> filterCombined =
e => filter1(e) || filter2(e) || filter3(e);
var myFilteredData = myContext.Set<Person>()
.Where(filterCombined)
.ToList();
One of the problems is that the collection of IQueryable is only valid as long as your DbSet is valid. As soon as your DbContext is Disposed your carefully filled collection is worthless.
So you have to think of another method to reconstruct the query than the one that uses the DbSet<Person>
Although at first glance they seem the same, there is a difference between IEnumerable and IQueryable. An Enumerable has everything in it to enumerate over the resulting sequence.
A Queryable on the other hand holds an Expression and a Provider. The Provider knows where the data can be fetched. This is usually a database, but it can also be a CSV-file or other items where you can fetch sequences. It is the task of the Provider to interpret the Expression and to translate it info a format that the database can understand, usually SQL.
While concatenating the linq statements into one big linq statements, the database is not accessed. Only the Expression is changed.
Once you call GetEnumerator() and MoveNext() (usually by doing ForEach, or ToList(), or similar), the Expression is sent to the Provider who will translate it into a query format that the database understands and perform the query. the result of the query is an Enumerable sequence, so Getenumerator() and MoveNext() of the provider's query result are called.
Because your IQueryable holds this Provider, you can't enumerate anymore after the Provider has been disposed.
When using entity framework, the DbSet holds the Provider. In the Provider is the Database information held by the DbContext. Once you Dispose the DbContext you can't use the IQueryable anymore:
IQueryable<Person> query = null;
using (var dbContext = new MyDbcontext())
{
query = dbContext.Persons.Where(person => person.Age > 20);
}
foreach (var person in query)
{
// expect exception: the DbContext is already Disposed
}
So you can't put the Provider in your collection or possible queries. However, you could remember the Expression. The only thing your require from your Expression is that it returns a Person. You also need a function that takes this Expression and a QueryaProvider for Persons to convert it to an IQueryable.
Let's create a generic function for this, so It can be used for any type, not just for Persons:
static IQueryable<TSource> ToQueryable<TSource>(this IQueryProvider provider,
Expression expression)
{
return provider.CreateQuery(expression);
}
// well, let's add the following also:
static IQueryable<Tsource> ToQueryable<TSource>(this DbContext dbContext,
Expression expression)
{
return dbContext.Set<TSource>.Provider.ToQueryable<TSource>(expression);
}
For help on extension functions see Extension Functions Demystified
Now only once you create your collection of Expressions. For fast lookup make it a Dictionary:
enum PersonQuery
{
ByFirstname,
ByAge,
ByHasChildWithAgeOver,
Skip,
}
public IReadOnlyDictionary<PersonQuery, Expression> CreateExpressions()
{
Dictionary<PersonQuery, Expression> dict = new Dictionary<PersonQuery, Expression>();
using (var dbContext = new MyDbContext())
{
IQueryable<Person> queryByFirstName = dbContext.Persons
.Where(...);
dict.Add(PersonQuery.ByfirstName, queryByFirstName.Expression);
... // etc for the other queries
}
return dict.
}
Usage:
IReadOnlyCollection<Person> PerformQuery(PersonQuery queryId)
{
using (var dbContext = new MyDbContext())
{
// get the Expression from the dictionary:
var expression = this.QueryDictionary[queryId];
// translate to IQueryable:
var query = dbContext.ToQueryable<Person>(expression);
// perform the query:
return query.ToList();
// because all items are fetched by now, the DbContext may be Disposed
}
}
Related
I am trying to get data from linq in asp.net core. I have a table with a Position with a FacultyID field, how do I get it from the Position table with an existing userid. My query
var claimsIdentity = _httpContextAccessor.HttpContext.User.Identity as ClaimsIdentity;
var userId = claimsIdentity.FindFirst(ClaimTypes.NameIdentifier)?.Value.ToString();
var data = _context.Positions.Where(p => p.UserID.ToString() == userId).Select(x => x.FacultyID).???;
What can I add after the mark? to get the data. Thank you so much
There are several things you can do. An example in your case would be:
var data = _context.Positions.Where(p => p.UserID.ToString() == userId).Select(x => x.FacultyID).FirstOrDefault();
If you expect more than 1 results, then you would do:
var data = _context.Positions.Where(p => p.UserID.ToString() == userId).Select(x => x.FacultyID).ToList();
You have to be aware of the difference between a query and the result of a query.
The query does not represent the data itself, it represents the potential to fetch some data.
If you look closely to the LINQ methods, you will find there are two groups: the LINQ methods that return IQueryable<...> and the others.
The IQueryable methods don't execute the query. These functions are called lazy, they use deferred execution. You can find these terms in the remarks section of every LINQ method.
As long as you concatenate IQueryable LINQ methods, the query is not executed. It is not costly to concatenate LINQ methods in separate statements.
The query is executed as soon as you start enumerating the query. At its lowest level this is done using GetEnumerator and MoveNext / Current:
IQueryable<Customer> customers = ...; // Query not executed yet!
// execute the query and process the fetched data
using (IEnumerator<Customer> enumerator = customers.GetEnumerator())
{
while(enumerator.MoveNext())
{
// there is a Customer, it is in property Current:
Customer customer = enumerator.Current;
this.ProcessFetchedCustomer(customer);
}
}
This code, or something very similar is done when you use foreach, or one of the LINQ methods that don't return IQueryable<...>, like ToList, ToDictionary, FirstOrDefault, Sum, Any, ...
var data = dbContext.Positions
.Where(p => p.UserID.ToString() == userId)
.Select(x => x.FacultyID);
If you use your debugger, you will see that data is an IQueryable<Position>. You'll have to use one of the other LINQ methods to execute the query.
To get all Positions in the query:
List<Position> fetchedPositions result = data.ToList();
If you expect only one position:
Position fetchedPosition = data.FirstOrDefault();
If you want to know if there is any position at all:
if (positionAvailable = data.Any())
{
...
}
Be aware: if you use the IQueryable, the data will be fetched again from the DbContext. So if you want to do all three statements efficiently these, make sure you don't use the original data three times:
List<Position> fetchedPositions result = data.ToList();
Position firstPosition = fetchedPostion.FirstOrDefault();
if (firstPosition != null)
{
ProcessPosition(firstPosition);
}
I have following generic queryable (which may already have selections applied):
IQueryable<TEntity> queryable = DBSet<TEntity>.AsQueryable();
Then there is the Provider class that looks like this:
public class Provider<TEntity>
{
public Expression<Func<TEntity, bool>> Condition { get; set; }
[...]
}
The Condition could be defined per instance in the following fashion:
Condition = entity => entity.Id == 3;
Now I want to select all Provider instances which have a Condition that is met at least by one entity of the DBSet:
List<Provider> providers = [...];
var matchingProviders = providers.Where(provider => queryable.Any(provider.Condition))
The problem with this: I'm starting a query for each Provider instance in the list. I'd rather use a single query to achieve the same result. This topic is especially important because of questionable performance. How can I achieve the same results with a single query and improve performance using Linq statements or Expression Trees?
Interesting challenge. The only way I see is to build dynamically UNION ALL query like this:
SELECT TOP 1 0 FROM Table WHERE Condition[0]
UNION ALL
SELECT TOP 1 1 FROM Table WHERE Condition[1]
...
UNION ALL
SELECT TOP 1 N-1 FROM Table WHERE Condition[N-1]
and then use the returned numbers as index to get the matching providers.
Something like this:
var parameter = Expression.Parameter(typeof(TEntity), "e");
var indexQuery = providers
.Select((provider, index) => queryable
.Where(provider.Condition)
.Take(1)
.Select(Expression.Lambda<Func<TEntity, int>>(Expression.Constant(index), parameter)))
.Aggregate(Queryable.Concat);
var indexes = indexQuery.ToList();
var matchingProviders = indexes.Select(index => providers[index]);
Note that I could have built the query without using Expression class by replacing the above Select with
.Select(_ => index)
but that would introduce unnecessary SQL query parameter for each index.
Here is another (crazy) idea that came in my mind. Please note that similar to my previous answer, it doesn't guarantee better performance (in fact it could be worse). It just presents a way to do what you are asking with a single SQL query.
Here we are going to create a query that returns a single string with length N consisting of '0' and '1' characters with '1' denoting a match (something like string bit array). The query will use my favorite group by constant technique to build dynamically something like this:
var matchInfo = queryable
.GroupBy(e => 1)
.Select(g =>
(g.Max(Condition[0] ? "1" : "0")) +
(g.Max(Condition[1] ? "1" : "0")) +
...
(g.Max(Condition[N-1] ? "1" : "0")))
.FirstOrDefault() ?? "";
And here is the code:
var group = Expression.Parameter(typeof(IGrouping<int, TEntity>), "g");
var concatArgs = providers.Select(provider => Expression.Call(
typeof(Enumerable), "Max", new[] { typeof(TEntity), typeof(string) },
group, Expression.Lambda(
Expression.Condition(
provider.Condition.Body, Expression.Constant("1"), Expression.Constant("0")),
provider.Condition.Parameters)));
var concatCall = Expression.Call(
typeof(string).GetMethod("Concat", new[] { typeof(string[]) }),
Expression.NewArrayInit(typeof(string), concatArgs));
var selector = Expression.Lambda<Func<IGrouping<int, TEntity>, string>>(concatCall, group);
var matchInfo = queryable
.GroupBy(e => 1)
.Select(selector)
.FirstOrDefault() ?? "";
var matchingProviders = matchInfo.Zip(providers,
(match, provider) => match == '1' ? provider : null)
.Where(provider => provider != null)
.ToList();
Enjoy:)
P.S. In my opinion, this query will run with constant speed (regarding number and type of the conditions, i.e. can be considered O(N) in the best, worst and average cases, where N is the number of the records in the table) because the database has to perform always a full table scan. Still it will be interesting to know what's the actual performance, but most likely doing something like this just doesn't worth the efforts.
Update: Regarding the bounty and the updated requirement:
Find a fast query that only reads a record of the table once and ends the query if already all conditions are met
There is no standard SQL construct (not even speaking about LINQ query translation) that satisfies both conditions. The constructs that allow early end like EXISTS can be used for a single condition, thus when executed for multiple conditions will violate the first rule of reading the table record only once. While the constructs that use aggregates like in this answer satisfy the first rule, but in order to produce the aggregate value they have to read all the records, thus cannot exit earlier.
Shortly, there is no query that can satisfy both requirements. What about the fast part, it really depends of the size of the data and the number and type of the conditions, table indexes etc., so again there is simply no "best" general solution for all cases.
Based on this Post by #Ivan I created an expression that is slightly faster in some cases.
It uses Any instead of Max to get the desired results.
var group = Expression.Parameter(typeof(IGrouping<int, TEntity>), "g");
var anyMethod = typeof(Enumerable)
.GetMethods()
.First(m => m.Name == "Any" && m.GetParameters()
.Count() == 2)
.MakeGenericMethod(typeof(TEntity));
var concatArgs = Providers.Select(provider =>
Expression.Call(anyMethod, group,
Expression.Lambda(provider.Condition.Body, provider.Condition.Parameters)));
var convertExpression = concatArgs.Select(concat =>
Expression.Condition(concat, Expression.Constant("1"), Expression.Constant("0")));
var concatCall = Expression.Call(
typeof(string).GetMethod("Concat", new[] { typeof(string[]) }),
Expression.NewArrayInit(typeof(string), convertExpression));
var selector = Expression.Lambda<Func<IGrouping<int, TEntity>, string>>(concatCall, group);
var matchInfo = queryable
.GroupBy(e => 1)
.Select(selector)
.First();
var MatchingProviders = matchInfo.Zip(Providers,
(match, provider) => match == '1' ? provider : null)
.Where(provider => provider != null)
.ToList();
The approach I tried here was to create Conditions and nest them into one Expression. If one of the Conditions is met, we get the index of the Provider for it.
private static Expression NestedExpression(
IEnumerable<Expression<Func<TEntity, bool>>> expressions,
int startIndex = 0)
{
var range = expressions.ToList();
range.RemoveRange(0, startIndex);
if (range.Count == 0)
return Expression.Constant(-1);
return Expression.Condition(
range[0].Body,
Expression.Constant(startIndex),
NestedExpression(expressions, ++startIndex));
}
Because the Expressions still may use different ParameterExpressions, we need an ExpressionVisitor to rewrite those:
private class PredicateRewriterVisitor : ExpressionVisitor
{
private readonly ParameterExpression _parameterExpression;
public PredicateRewriterVisitor(ParameterExpression parameterExpression)
{
_parameterExpression = parameterExpression;
}
protected override Expression VisitParameter(ParameterExpression node)
{
return _parameterExpression;
}
}
For the rewrite we only need to call this method:
private static Expression<Func<T, bool>> Rewrite<T>(
Expression<Func<T, bool>> exp,
ParameterExpression parameterExpression)
{
var newExpression = new PredicateRewriterVisitor(parameterExpression).Visit(exp);
return (Expression<Func<T, bool>>)newExpression;
}
The query itself and the selection of the Provider instances works like this:
var parameterExpression = Expression.Parameter(typeof(TEntity), "src");
var conditions = Providers.Select(provider =>
Rewrite(provider.Condition, parameterExpression)
);
var nestedExpression = NestedExpression(conditions);
var lambda = Expression.Lambda<Func<TEntity, int>>(nestedExpression, parameterExpression);
var matchInfo = queryable.Select(lambda).Distinct();
var MatchingProviders = Providers.Where((provider, index) => matchInfo.Contains(index));
Note: Another option which isn't really fast as well
Here is another view of the problem that has nothing to do with expressions.
Since the main goal is to improve the performance, if the attempts to produce the result with single query don't help, we could try improving the speed by parallelizing the execution of the original multi query solution.
Since it's really a LINQ to Objects query (which internally executes multiple EF queries), theoretically it should be a simple matter of turning it into a PLINQ query by inserting AsParallel like this (non working):
var matchingProviders = providers
.AsParallel()
.Where(provider => queryable.Any(provider.Condition))
.ToList();
However, it turns out that EF DbContext is not well suited for multi thread access, and the above simply generates runtime errors. So I had to resort to TPL using one of the Parallel.ForEach overloads that allows us to supply local state, which I used to allocate several DbContext instances during the execution.
The final working code looks like this:
var matchingProviders = new List<Provider<TEntity>>();
Parallel.ForEach(providers,
() => new
{
context = new MyDbContext(),
matchingProviders = new List<Provider<TEntity>>()
},
(provider, state, data) =>
{
if (data.context.Set<TEntity>().Any(provider.Condition))
data.matchingProviders.Add(provider);
return data;
},
data =>
{
data.context.Dispose();
if (data.matchingProviders.Count > 0)
{
lock (matchingProviders)
matchingProviders.AddRange(data.matchingProviders);
}
}
);
If you have a multi core CPU (which is normal nowadays) and a good database server, this should give you the improvement you are seeking for.
I'm using EF5, unit of work, and repository pattern.
I want to define some limitations on accessing data for specified users.
In database I've designed a table to keep my entity names and their properties which is called EntityProperties, and another table to keep the values of those properties which is called PropertyValues, and each EntityProperty has one or more PropertyValues.
In business layer when user requests data, if any limitation is defined for him, some conditions should be added to the linq statement. What I do is to get the list of entity names and their propeties and values by 'userId', then I add some 'Where' clause to the linq query.
However, the entity names and their properties are of type "String", thus I should use Reflection to manage them.
But I don't know this part, and I don't know how to create LINQ where clause from a given set of condition strings.
For example, let's suppose that a user requests the list orders, and user id is 5. I first query those access limitation tables, and the result is:
"Order", "Id", "74"
"Order", "Id", "77"
"Order", "Id", "115"
It means that this user should only see these three orders, while in Orders table, we have more orders.
So, if I want to use a LINQ query to get orders, like:
var orders = from order in Context.Orders
I need to turn it into something like:
var orders = from order in Context.Orders
// where order id should be in 74,77,115
However, getting to Order entity and Id property from "Order" and "Id" strings requires reflection. Thus two questions:
What is the best way to get strongly typed from strings?
Is there a better way for me to do this, with better performance?
Ok. With the comments, we might go for something like that (assuming you have a navigation property in EntityProperties table, which is a collection of PropertyValues, and named PropertyValueList (if you don't have, just do a join instead of using Include).
Here is some sample code, really rustic, working only with Int32 properties, but this might be the start of a solution.
You may also look at PredicateBuilder...
Anyway
I use an "intermediate class" Filter.
public class Filter
{
public string PropertyName { get; set; }
public List<string> Values { get; set; }
}
Then an helper class, which will return an IQueryable<T>, but... filtered
public static class FilterHelper {
public static IQueryable<T> Filter(this IQueryable<T> queryable, Context context, int userId) {
var entityName = typeof(T).Name;
//get all filters for the current entity by userId, Select the needed values as a `List<Filter>()`
//this could be done out of this method and used as a parameter
var filters = context.EntityProperties
.Where(m => m.entityName == entityName && m.userId = userId)
.Include(m => m.PropertyValueList)
.Select(m => new Filter {
PropertyName = m.property,
Values = m.PropertyValueList.Select(x => x.value).ToList()
})
.ToList();
//build the expression
var parameter = Expression.Parameter(typeof(T), "m");
var member = GetContains( filters.First(), parameter);
member = filters.Skip(1).Aggregate(member, (current, filter) => Expression.And(current, GetContains(filter, parameter)));
//the final predicate
var lambda = Expression.Lambda<Func<T, bool>>(member, new[] { parameter });
//use Where with the final predicate on your Queryable
return queryable.Where(lambda);
}
//this will build the "Contains" part
private static Expression GetContains(Filter filter, Expression expression)
{
Expression member = expression;
member = Expression.Property(member, filter.PropertyName);
var values = filter.Values.Select(m => Convert.ToInt32(m));
var containsMethod = typeof(Enumerable).GetMethods().Single(
method => method.Name == "Contains"
&& method.IsGenericMethodDefinition
&& method.GetParameters().Length == 2)
.MakeGenericMethod(new[] { typeof(int) });
member = Expression.Call(containsMethod, Expression.Constant(values), member);
return member;
}
}
usage
var orders = from order in Context.Orders
select order;
var filteredOrders = orders.Filter(Context, 1);//where 1 is a userId
My answer depends on whether you are happy to alter your access model slightly. I've got a similar situation in an application that I have written and personally I don't like the idea of having to rely on my calling code to correctly filter out the records based on the users authentication.
My approach was to use an OData service pattern to call into my Entity Framework, each of the repositories are exposed via OData independently.
An OData (WCFDataService) has QueryInterceptors which perform on the fly filtering of your data when a query is made. Thus if you asked the OData repository for context.Orders(o => o.Id) you'd only see the orders that that user was permitted to see with no additional clauses.
A good link to the basics is found here but it requires some work to manage the calling user and provide the filtering that you may require. You can provide the query interceptor at every record level.
I have a method where I am trying to return all default customer addresses with the matching gender. I would like to be able to build up the filtering query bit by bit by passing in System.Func methods to the where clause.
var emailAddresses = new List<string>();
// get all customers.
IQueryable<Customer> customersQ = base.GetAllQueryable(appContext).Where(o => o.Deleted == false);
// for each customer filter, filter the query.
var genders = new List<string>() { "C" };
Func<Customer, bool> customerGender = (o => genders.Contains(o.Addresses.FirstOrDefault(a => a.IsDefaultAddress).Gender));
customersQ = customersQ.Where(customerGender).AsQueryable();
emailAddresses = (from c in customersQ
select c.Email).Distinct().ToList();
return emailAddresses;
But this method calls the database for every address (8000) times which is very slow.
however if I replace the two lines
Func<Customer, bool> customerGender = (o => genders.Contains(o.Addresses.FirstOrDefault(a => a.IsDefaultAddress).Gender));
customersQ = customersQ.Where(customerGender).AsQueryable();
with one line
customersQ = customersQ.Where(o => genders.Contains(o.Addresses.FirstOrDefault(a => a.IsDefaultAddress).Gender)).AsQueryable();
Then the query only makes one call to the database and is very fast.
My question is why does this make a difference? How can I make the first method work with only calling the database once?
Use expression instead of Func:
Expression<Func<Customer, bool>> customerGender = (o =>
genders.Contains(o.Addresses.FirstOrDefault(a => a.IsDefaultAddress).Gender));
customersQ = customersQ.Where(customerGender).AsQueryable();
When you are using simple Func delegate, then Where extension of Enumerable is called. Thus all data goes into memory, where it is enumerated and lambda is executed for each entity. And you have many calls to database.
On the other hand, when you are using expression, then Where extension of Queryable is called, and expression is converted into SQL query. That's why you have single query in second case (if you use in-place lambda it is converted into expression).
I would like to create a repository model that could take an Expression and use Linq-To-Sql to generate the required SQL statement.
For example, I have a function such as this:
// Possible criteria
Expression<Func<Purchase,bool>> criteria1 = p => p.Price > 1000;
// Function that should take that criteria and convert to SQL statement
static IEnumerable<Customer> GetCustomers (Expression<Func<Purchase,bool>> criteria)
{
// ...
}
Inside the function, I would like to convert criteria to a SQL statement using Linq-To-Sql.
I am aware that you can use DataContext.Log to see the executed queries and DataContext.GetCommand(query).CommandText to see the full query before it is executed. However, I would like just a part of the entire expression generated.
What I am hoping to accomplish is to make my repository abstract the underlying technology (Linq-to-Sql, Dapper, etc). That way I could pass the Expression to the repository, have it generate the right statement and use the right technology to execute it.
You could do something like this:
string sql = DataContext.GetTable<Customer>().Where(criteria).ToString();
ToString() gives you the SQL expression. You could then use regex to pull out the WHERE clause.
This is a code excerpt that I use to build my own predicate to use in the Where function. The compiler can't cope with ienumerables of complex objects, so you have to do it yourself.
Essentially, the code gets passed an ienumerable of (string code, string exchange) tuples, and then builds an expression to retrieve all Security objects that have Security.Code == tuple.Code AND (Security.MasterExchangeForStocksId == tuple.exchange OR SecurityExchangeId == tuple.exchange).
CreateTrEntitiesAsync() simply returns a Entity Framework context, which has a DbSet Security property.
public async Task<Security[]> GetSecurities(IEnumerable<(string code, string exchange)> tickers)
{
using (var ctx = await CreateTrEntitiesAsync())
{
var securityExpr = Expression.Parameter(typeof(Security), "security");
Expression expr = null;
Expression exprToadd;
foreach (var item in tickers)
{
exprToadd = Expression.And(
Expression.Equal(Expression.Property(securityExpr, nameof(Security.Code)), Expression.Constant(item.code)),
Expression.Or(
Expression.Equal(Expression.Property(Expression.Property(securityExpr, nameof(Security.Exchange)), nameof(Exchange.MasterExchangeForStocksId)), Expression.Constant(item.exchange)),
Expression.Equal(Expression.Property(securityExpr, nameof(Security.ExchangeId)), Expression.Constant(item.exchange))
)
);
if (expr == null)
expr = exprToadd;
else
expr = Expression.Or(expr, exprToadd);
}
var criteria = Expression.Lambda<Func<Security, bool>>(expr, new ParameterExpression[] { securityExpr });
var items = ctx.Securities.Where(criteria);
return await items.ToArrayAsync();
}
}