I need to write some dynamic queries for a project I'm working on. I'm finding out that a significant amount of time is being spent by my program on the Count and First methods, so I started to change to .Single, only to find out that there is no such method.
The code below was my first attempt at creating one (mostly copied from the Where method), but it's not working. Help?
public static object Single(this IQueryable source, string predicate, params object[] values)
{
if (source == null) throw new ArgumentNullException("source");
if (predicate == null) throw new ArgumentNullException("predicate");
LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, typeof(bool), predicate, values);
return source.Provider.CreateQuery(
Expression.Call(
typeof(Queryable), "Single",
new Type[] { source.ElementType },
source.Expression, Expression.Quote(lambda)));
}
IMHO, you should just simply can use Single or SingleOrDefault when you are executing the query.
// build your dynamic query
var query = NorthwindConext.Products.Categories
.Where("CategoryID = #0", 2);
// now you can simply get the single item by
var category = query.SingleOrDefault();
So, I do not see the necessity for a "Single" operator for dynnamic linq. Especially, as the IEnumerable or IQueryable returned by the query enumeration should only contain one item.
I don't understand what the difference you fill to be between Single(SingleOrDefault) and First(FirstOrDefault) ?
Moreover EF do not implement the first one and you have to use First(FirstOrDefault) instead.
Also why do you fill you will gain perfomance improvement by creating your own implementation of single , which is by your comment almost a copy of where , which is almost the same as first
so why do not use it , and try to look at query's being generated and analyse them ?
I think Queryable.Single is what you're looking for.
Related
I came across the following piece of code in the Sprache repository :
Parser<string> identifier =
from leading in Parse.WhiteSpace.Many()
from first in Parse.Letter.Once().Text()
from rest in Parse.LetterOrDigit.Many().Text()
from trailing in Parse.WhiteSpace.Many()
select first + rest;
var id = identifier.Parse(" abc123 ");
I see a contradiction here: the from clause docs say the source (Parse.WhiteSpace.Many() or Parse.Letter.Once().Text() in our case) must be IEnumerable:
The data source referenced in the from clause must have a type of IEnumerable, IEnumerable<T>, or a derived type such as IQueryable<T>
But it isn't and the compiler says that's fine!
I thought there is some implicit cast to IEnumerable, but there isn't: Parse.WhiteSpace.Many() returns Parser<IEnumerable<T>> and Parse.Letter.Once().Text() returns Parser<string> (types are not IEnumerable).
1st question: Why does the compiler allow this code?
Also, the final expression select first + rest doesn't take into account leading and trailing variables, but the final result identifier, for sure, uses them inside.
2nd question: By what rule\mechanism leading and trailing variables were added to the identifier?
P.S.
It'd be great if someone shared an all-encompassing doc about internal work of LINQ query syntax. I've found nothing on this topic.
After like five minutes of looking at the code I have observed:
parser is a delegate that returns an intermediate result
public delegate IResult<T> Parser<out T>(IInput input);
there are linq compliant methods declared that allow linq syntax like:
public static Parser<U> Select<T, U>(this Parser<T> parser, Func<T, U> convert)
{
if (parser == null) throw new ArgumentNullException(nameof(parser));
if (convert == null) throw new ArgumentNullException(nameof(convert));
return parser.Then(t => Return(convert(t)));
}
https://github.com/sprache/Sprache/blob/develop/src/Sprache/Parse.cs#L357
It is not true that IEnumerable interface is required for the syntax from x in set to work you just require particular extension method with correct name that accepts correct set of parameters. So the above makes select valid. Here you have where method
public static Parser<T> Where<T>(this Parser<T> parser, Func<T, bool> predicate)
{
if (parser == null) throw new ArgumentNullException(nameof(parser));
if (predicate == null) throw new ArgumentNullException(nameof(predicate));
return i => parser(i).IfSuccess(s =>
predicate(s.Value) ? s : Result.Failure<T>(i,
$"Unexpected {s.Value}.",
new string[0]));
}
https://github.com/sprache/Sprache/blob/develop/src/Sprache/Parse.cs#L614
and so on.
This is separate implementation of the linq abstraction that has nothing to do with collections it is about parsing text. It produces a nested chain of delegates that process given string to verify if it confirms to particular gramma and returns structure that describes parsed text.
that answers first question. For the second you need to follow the code: all from x in set except the first one map to SelectMany function:
public static Parser<V> SelectMany<T, U, V>(
this Parser<T> parser,
Func<T, Parser<U>> selector,
Func<T, U, V> projector)
{
if (parser == null) throw new ArgumentNullException(nameof(parser));
if (selector == null) throw new ArgumentNullException(nameof(selector));
if (projector == null) throw new ArgumentNullException(nameof(projector));
return parser.Then(t => selector(t).Select(u => projector(t, u)));
}
https://github.com/sprache/Sprache/blob/develop/src/Sprache/Parse.cs#L635
and Then method
https://github.com/sprache/Sprache/blob/develop/src/Sprache/Parse.cs#L241
there you will see that if first succeeds (leading white spaces where matched) only than second (the single letter parser) is applied on the remainder of the string. So Again it is not a collection processing its a chain of events that lead to parsing the string.
I have another question about performance with EF.
There's one method to get an object from context:
tDocumentTyp DocumentTypObject = Context.tDocumentTyps.Where(s => s.DocumentTypID == iTypID).FirstOrDefault();
This method takes ~2979 ms.
Then I wrote a method to get the DBSet via reflection and is executed this way:
tDocumentTyp DocumentTypObject = Context.GetEntries<tDocumentTyp>().Where(s => s.DocumentTypID == iTypID).FirstOrDefault();
My method needs ~222 ms to execute.
So my question now is, why is my method much faster than the original one? Or is there anything wrong with my method?
To make this a bit easier, here is my method for getting DBSet via reflection:
public static IEnumerable<T> GetEntries<T>(this AppContext DataContext,
string PropertyName = null, IEnumerable<string> includes = null) where T : IEntity
{
Type ContextType = typeof(AppContext);
PropertyInfo Entity = null;
if (null == PropertyName)
Entity = ContextType.GetProperty(typeof(T).Name)
?? ContextType.GetProperty(typeof(T).Name + "s");
else
Entity = ContextType.GetProperty(PropertyName);
if (null == Entity)
throw new Exception("Could not find the property. If the property is not equal to the tablesname, you have to parametrize it.");
DbQuery<T> set = ((DbSet<T>)Entity.GetValue(DataContext, null));
if (includes != null)
includes.ForEach(f => set = set.Include(f));
return set;
}
The second example is getting the entire table and applying the Where in memory. You are applying the extension method System.Linq.Enumerable.Where which operates on IEnumerable<T>. Note that this is an in-memory implementation. In the first example you using the extension method System.Linq.Queryable.Where which operates on IQueryable<T>. This is a different method, though they share the same name.
If you inspect closely you will also find that in the first example the method parameter is of type Expression<Func<T, bool>> whilst in the second example it is simply Func<T, bool>. This is a very important difference: the expression can be processed to produce a SQL-query.
So why is the second one faster? Well, that is hard to answer without more information about your data source. But as others have noted in the comments, if the database is not indexed then it may well be quicker to select the entire table and execute the filter in memory than to have the SQL server apply the filtering.
We are using Xamarin with SQLiteNet as ORM.
In our data layer class we have the method below.
filter = ri => ri.ItemVersioniId == itemVersionId;
The method is getting the records matching the Id. If the lambda expression is hardcoded, instead of using the "filter" parameter it is much faster... even though it is the same logic.
We would to be able to pass the filter as a parameter but still get a good performance. Any advise?
public virtual List<ResourceItem> GetResourceItems (string itemVersionId, Func<ResourceItem,bool> filter ){
//var t = db.Table<ResourceItem> ().Where (ri => ri.ItemVersionId == itemVersionId); --* this line is 10 times faster
var t = db.Table<ResourceItem> ().Where (filter); --* this line is 10 times slower
return new List<ResourceItem> (t);
}
I'm not sure because it is xamarin specific, but i suggest to use Expression instead of Func.
Expression<Func<ResourceItem,bool>> filter =
ri => ri.ItemVersioniId == itemVersionId;
public virtual List<ResourceItem> GetResourceItems
(string itemVersionId, Expression<Func<ResourceItem,bool>> filter )
{
return db.Table<ResourceItem> ().Where (filter).ToList();
}
I would advise hard coding it. Here is why, but first let me qualify this by saying I am speculating - I have no experience with SQLiteNet - this based on some general, rudimentary knowledge on how LINQ Prodivers work.
When you have it hard coded the lambda expression is converted to SQL at compile time. When you set it to a delegate, it could be a LINQ to Objects query, there is no way to know at compile time that your LINQ Provider can convert that to a SQL statement. Instead this work occurs at runtime and as a result the performance suffers greatly.
I'm having some trouble figuring out the best way to do this, and I would appreciate any help.
Basically, I'm setting up a filter that allows the user to look at a history of audit items associated with an arbitrary "filter" of usernames.
The datasource is a SQL Server data base, so I'm taking the IQueryable "source" (either a direct table reference from the db context object, or perhaps an IQueryable that's resulted from additional queries), applying the WHERE filter, and then returning the resultant IQueryable object....but I'm a little stumped as to how to perform OR using this approach.
I've considered going the route of Expressions because I know how to OR those, but I haven't been able to figure out quite how to do that with a "Contains" type evaluation, so I'm currently using a UNION, but I'm afraid this might have negative impact on performance, and I'm wondering if it may not give me exactly what I need if other filters (in addition to user name filtering shown here) are added in an arbirary order.
Here is my sample code:
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
// Take allowed values...
List<string> searchStrings = new List<string>();
// <SNIP> (This just populates my list of search strings)
IQueryable<X> oReturn = null;
// Step through each iteration, and perform a 'LIKE %value%' query
string[] searchArray = searchStrings.ToArray();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
if (i == 0)
// For first step, perform direct WHERE
oReturn = source.Where(x => x.Username.Contains(value));
else
// For additional steps, perform UNION on WHERE
oReturn = oReturn.Union(source.Where(x => x.Username.Contains(value)));
}
return oReturn ?? source;
}
This feels like the wrong way to do things, but it does seem to work, so my question is first, is there a better way to do this? Also, is there a way to do a 'Contains' or 'Like' with Expressions?
(Editted to correct my code: In rolling back to working state in order to post it, I apparently didn't roll back quite far enough :) )
=============================================
ETA: Per the solution given, here is my new code (in case anyone reading this is interested):
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
List<string> searchStrings = new List<string>(AllowedValues);
// <SNIP> build collection of search values
string[] searchArray = searchStrings.ToArray();
Expression<Func<X, bool>> expression = PredicateBuilder.False<X>();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
expression = expression.Or(x => x.Username.Contains(value));
}
return source.Where(expression);
}
(One caveat I noticed: Following the PredicateBuilder's example, an empty collection of search strings will return false (false || value1 || ... ), whereas in my original version, I was assuming an empty list should just coallesce to the unfiltered source. As I thought about it more, the new version seems to make more sense for my needs, so I adopted that)
=============================================
You can use the PredicateBuilder from the LINQkit to dynamically construct your query.
I have a method that gets rows from my database. It looks like this:
public static IEnumerable<Dictionary<string, object>> GetRowsIter()
{
_resultSet.ReadFirst();
do
{
var resultList = new Dictionary<string, object>();
for (int fieldIndex = 0; fieldIndex < _resultSet.FieldCount; fieldIndex++)
{
resultList.Add(_resultSet.GetName(fieldIndex),
_resultSet.GetValue(fieldIndex));
}
yield return resultList;
} while (_resultSet.ReadRelative(1));
yield break;
}
This is great when I want to return all the rows. But sometimes I want to return only some of the rows.
I am planning on writing my own method (GetRowsIterWhere(string column, object whereValue)), but I am wondering if I can use the linq where on my method.
I admit I don't know how it would work, becuase I am advancing the reader with a ReadRelative(1) to get to the next value. So even if it "thinks" it is skipping rows, it will not really skip them.
I am really concerned with performance here (I am currently refactoring from Linq-To-Datasets because it was way way way too slow.)
So, my question is, do I need to write my own Where type method or can I change the one above to work with the linq where method?
Yes you can use LINQ Where, but you'll need to build the predicate yourself. It isn't tricky. Something like (from memory; no compiler to hand):
var param = Expression.Parameter(typeof(T), "row");
var body = Expression.Equal(
Expression.PropertyOrField(param, column),
Expression.Constant(whereValue));
var lambda = Expression.Lambda<Func<T,bool>>(body, param);
then:
IQueryable<T> source = ...
var filtered = source.Where(lambda);
This will cause the Where to be executed at the server (for example, in TSQL), removing most of the network IO (asusming a sensible filter).