We are using Xamarin with SQLiteNet as ORM.
In our data layer class we have the method below.
filter = ri => ri.ItemVersioniId == itemVersionId;
The method is getting the records matching the Id. If the lambda expression is hardcoded, instead of using the "filter" parameter it is much faster... even though it is the same logic.
We would to be able to pass the filter as a parameter but still get a good performance. Any advise?
public virtual List<ResourceItem> GetResourceItems (string itemVersionId, Func<ResourceItem,bool> filter ){
//var t = db.Table<ResourceItem> ().Where (ri => ri.ItemVersionId == itemVersionId); --* this line is 10 times faster
var t = db.Table<ResourceItem> ().Where (filter); --* this line is 10 times slower
return new List<ResourceItem> (t);
}
I'm not sure because it is xamarin specific, but i suggest to use Expression instead of Func.
Expression<Func<ResourceItem,bool>> filter =
ri => ri.ItemVersioniId == itemVersionId;
public virtual List<ResourceItem> GetResourceItems
(string itemVersionId, Expression<Func<ResourceItem,bool>> filter )
{
return db.Table<ResourceItem> ().Where (filter).ToList();
}
I would advise hard coding it. Here is why, but first let me qualify this by saying I am speculating - I have no experience with SQLiteNet - this based on some general, rudimentary knowledge on how LINQ Prodivers work.
When you have it hard coded the lambda expression is converted to SQL at compile time. When you set it to a delegate, it could be a LINQ to Objects query, there is no way to know at compile time that your LINQ Provider can convert that to a SQL statement. Instead this work occurs at runtime and as a result the performance suffers greatly.
Related
Working with EPiServer Find and trying to build a generic facet funcionality for it to simplify managing which facet should be enabled. I would like to construct two generic methods, one for adding active filters to perform the hits search and one to perform the available facet filters remaining.
The first method will perform the following (specific code for brand filter):
var brandFilter = client.BuildFilter<FashionProduct>();
foreach (var facet in SelectedGroup.Facets.Where(x => x.Selected))
{
brandFilter = brandFilter.Or(x => x.Brand.Match(facet.Key));
}
query = query.Filter(brandFilter);
I would like to be able to call it in a generic way so I could base the available facets on some simple list of strings or objects. Like this:
query = AddFilterFacet<FashionProduct>(query, "Brand", SelectedGroup.Facets)
So the method would take the type of object to filter on, the query to append the filters on, the name of the property to filter on and the list of values to add.
The second method is similar but relates more to perform the following:
facetQuery = facetQuery.TermsFacetFor(x => x.Brand)
...
var brandFacets = facetResult.TermsFacetFor(x => x.Brand).Terms;
Is it possible to build this kind of functionality? The biggest questionmark I have is how to translate the "Brand" input string to be the Brand Property in x => x.Brand
private void AddFilterFacet<T>(IClient client, ref ITypeSearch<T> query, string propertyName, List<FacetOption> facets)
{
var filter = client.BuildFilter<T>();
foreach (var facet in facets)
{
filter = filter.Or(x => x.????.Match(facet.Key));
}
query = query.Filter(filter);
}
The .Or method takes a
System.Linq.Expressions.Expression<Func<T, Find.Api.Querying.Filter>>
so perhaps something can be used to make a proper generic call to it
It's definitely possible to create generic lambda expressions, it's just not easy and requires a lot of reflection code.
I haven't done it in a while, but maybe if you look at the code i created for something similar a while ago (Generic lambda expressions) it'll help. I'm sure someone with fresher experience will help you out here soon enough.
Decimal precision attribute <-- take a look a this answer witch has code to genereate modelBuilder.Entity<CLASS>().Property(OBJECT=> OBJECT.PROPERTY).HasPrecision(12, 10) automatically from an attribute in a class
I sum myself to the hapless lot that fumbles with custom methods in LINQ to EF queries. I've skimmed the web trying to detect a pattern to what makes a custom method LINQ-friendly, and while every source says that the method must be translatable into a T-SQL query, the applications seem very diverse. So, I'll post my code here and hopefully a generous SO denizen can tell me what I'm doing wrong and why.
The Code
public IEnumerable<WordIndexModel> GetWordIndex(int transid)
{
return (from trindex in context.transIndexes
let trueWord = IsWord(trindex)
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
}
public string IsWord(transIndex trindex)
{
Match m = Regex.Match(trindex.word, #"^[a-z]+(\w*[-]*)*",
RegexOptions.IgnoreCase);
return m.Value;
}
With the above code I access a table, transIndex that is essentially a word index of culled from various user documents. The problem is that not all entries are actually words. Nubers, and even underscore lines, such as, ___________,, are saved as well.
The Problem
I'd like to keep only the words that my custom method IsWord returns (at the present time I have not actually developed the parsing mechanism). But as the IsWord function shows it will return a string.
So, using let I introduce my custom method into the query and use it as a grouping parameter, the is selectable into my object. Upon execution I get the omninous:
LINQ to Entities does not recognize the method
'System.String IsWord(transIndex)' method, and this
method cannot be translated into a store expression."
I also need to make sure that only records that match the IsWord condition are returned.
Any ideas?
It is saying it does not understand your IsWord method in terms of how to translate it to SQL.
Frankly it does not do much anyway, why not replace it with
return (from trindex in context.transIndexes
let trueWord = trindex.word
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
What methods can EF translate into SQL, i can't give you a list, but it can never translate a straight forward method you have written. But their are some built in ones that it understands, like MyArray.Contains(x) for example, it can turn this into something like
...
WHERE Field IN (ArrItem1,ArrItem2,ArrItem3)
If you want to write a linq compatible method then you need to create an expresion tree that EF can understand and turn into SQL.
This is where things star to bend my mind a little but this article may help http://blogs.msdn.com/b/csharpfaq/archive/2009/09/14/generating-dynamic-methods-with-expression-trees-in-visual-studio-2010.aspx.
If the percentage of bad records in return is not large, you could consider enumerate the result set first, and then apply the processing / filtering?
var query = (from trindex in context.transIndexes
...
select new WordIndexModel
{
Word,
Instances = grouped.Select(test => test.trans).Distinct()
});
var result = query.ToList().Where(word => IsTrueWord(word));
return result;
If the number of records is too high to enumerate, consider doing the check in a view or stored procedure. That will help with speed and keep the code clean.
But of course, using stored procedures has disadvatages of reusability and maintainbility (because of no refactoring tools).
Also, check out another answer which seems to be similar to this one: https://stackoverflow.com/a/10485624/3481183
I have to query a repository where column a = {myvalue} and column b has any value in a collection.
Here's what I have:
Application[] applicationCollection = GetAllApplications();
var result = repo.GetAll(r => r.Email == myEmail && r.Application.In(applicationCollection));
I can't figure out how to write the "in" part...
Preferrably links to how to do this would be best, so I can learn it, as opposed to just getting the answer :). Thanks all.
(I'm sure this is a repeat question, but my Google/search skills are obviously poor.)
The SQL idea of item in collection is written in C# (including LINQ) as collection.Contains(item). In your example, this might be:
var result = repo.GetAll(r => r.Email == myEmail &&
applicationCollection.Contains(r.Application));
ApplicationCollection.Contains(r.Application)
Use .Contains(collection) instead of in. Here's a link since you wanted one.
If you want to write it the way you've shown, you can write this extension method:
public static bool In<T>(this T item, IEnumerable<T> set)
{
return set.Contains(item);
}
And then use it exactly like you did in your question:
Application[] applicationCollection = GetAllApplications();
var result = repo.GetAll(r =>
r.Email == myEmail &&
r.Application.In(applicationCollection));
This will work fine if you're only working with in-memory sets. If you're doing something like LINQ-to-SQL, however, this won't be compatible.
Note: I'm not trying to imply that this extension method might be a good idea to use - I'm just saying it's possible to write one like that.
I tried to use expandoobjects in LINQ queries to have the ability to query against properties that are created during runtime, for example the headers from a csv file. It all worked fine if typing the LINQ query direct in the code as in the example:
// initialize testdata
List<ExpandoObject> hans = new List<ExpandoObject>();
string[] names = {"Apfel", "Birne", "Banane", "Orange"};
int[] ids = { 1, 2, 3, 4 };
for (int i = 0; i < 4; i++)
{
dynamic horst = new ExpandoObject();
((IDictionary<string, object>)horst).Add("Fruit", names[i]);
((IDictionary<string, object>)horst).Add("ID", ids[i]);
hans.Add(horst);
}
// try some LINQ queries, both are working as intended
var klaus = from dynamic x in hans where x.ID < 3 select x;
//var klaus = hans.Where(x => x.ID < 3).Select(x => x);
Then i tried to read the query from the commandline and create a dynamic LINQ query using a slighty modified version of the evaluant linq compiler.
string expression = System.Console.ReadLine();
LinqCompiler lc = new LinqCompiler(expression);
lc.AddSource<ExpandoObject>("hans", hans);
IEnumerable<ExpandoObject> klaus = (IEnumerable<ExpandoObject>)lc.Evaluate();
As long as as i donĀ“t use WHERE or ORDER BY statements, everything is fine, but if any WHERE or ORDER BY is included in the query, i get an error when compiling the codedom code in the linq compiler: CS1963: An expression tree may not contain a dynamic operation.
The code for the query is created using the following line:
doRequestMethod.Statements.Add(new CodeMethodReturnStatement(new CodeSnippetExpression(Query)));
I suppose that the codedom compiler is building the expression tree in some way different to the way a direct typed in LINQ query is parsed. Any idea to get this to work would be appretiated, including other ideas to dynamically create queries for runtime-generated objects.
To get the error you're getting, I had to fix the LINQ Compiler to support dynamic, by telling it to use C# 4.0 and add a reference to Microsoft.CSharp.dll, so I assume you have done the same.
The problem is a source in LINQ compiler can be any collection, including IQueryable<T>. And if IQueryable<T> is supposed to work correctly, you actually need to treat it as IQueryable<T>, not IEnumerable<T>. The way LINQ compiler solves this is that it treats any source as IQuerybale<T>by using the AsQueryable() extension method.
This means the generated code looks like this:
public object DoRequest(System.Linq.IQueryable<System.Dynamic.ExpandoObject> hans) {
return from dynamic x in hans where x.ID < 3 select x;
}
The problem with this code is that tries to use IQuerybale<T> versions of LINQ methods, that use Expressions. And, as the error message tells you, Expressions don't support dynamic.
I think the easiest way to fix this is to modify LINQ Compiler to use IEnumerable<T>, instead of IQuerybale<T> by changing the AddSource() into:
public void AddSource<T>(string name, IEnumerable<T> source)
{
this.sources.Add(new SourceDescription(name, typeof(IEnumerable<T>), source));
}
Of course, this means it won't work well for database queries, but you can't make database queries work with dynamic anyway.
I'm having some trouble figuring out the best way to do this, and I would appreciate any help.
Basically, I'm setting up a filter that allows the user to look at a history of audit items associated with an arbitrary "filter" of usernames.
The datasource is a SQL Server data base, so I'm taking the IQueryable "source" (either a direct table reference from the db context object, or perhaps an IQueryable that's resulted from additional queries), applying the WHERE filter, and then returning the resultant IQueryable object....but I'm a little stumped as to how to perform OR using this approach.
I've considered going the route of Expressions because I know how to OR those, but I haven't been able to figure out quite how to do that with a "Contains" type evaluation, so I'm currently using a UNION, but I'm afraid this might have negative impact on performance, and I'm wondering if it may not give me exactly what I need if other filters (in addition to user name filtering shown here) are added in an arbirary order.
Here is my sample code:
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
// Take allowed values...
List<string> searchStrings = new List<string>();
// <SNIP> (This just populates my list of search strings)
IQueryable<X> oReturn = null;
// Step through each iteration, and perform a 'LIKE %value%' query
string[] searchArray = searchStrings.ToArray();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
if (i == 0)
// For first step, perform direct WHERE
oReturn = source.Where(x => x.Username.Contains(value));
else
// For additional steps, perform UNION on WHERE
oReturn = oReturn.Union(source.Where(x => x.Username.Contains(value)));
}
return oReturn ?? source;
}
This feels like the wrong way to do things, but it does seem to work, so my question is first, is there a better way to do this? Also, is there a way to do a 'Contains' or 'Like' with Expressions?
(Editted to correct my code: In rolling back to working state in order to post it, I apparently didn't roll back quite far enough :) )
=============================================
ETA: Per the solution given, here is my new code (in case anyone reading this is interested):
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
List<string> searchStrings = new List<string>(AllowedValues);
// <SNIP> build collection of search values
string[] searchArray = searchStrings.ToArray();
Expression<Func<X, bool>> expression = PredicateBuilder.False<X>();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
expression = expression.Or(x => x.Username.Contains(value));
}
return source.Where(expression);
}
(One caveat I noticed: Following the PredicateBuilder's example, an empty collection of search strings will return false (false || value1 || ... ), whereas in my original version, I was assuming an empty list should just coallesce to the unfiltered source. As I thought about it more, the new version seems to make more sense for my needs, so I adopted that)
=============================================
You can use the PredicateBuilder from the LINQkit to dynamically construct your query.