We Keep Coding

LinqKit Predicate Or with Contains evaluates to equals

I am implementing a search for my application. The search works except for when the user's search cascades by searching related entities. I have debugged the code and tested the SQL generated by Entity Framework. I found the problem is Contains() translates to a '=' in SQL when it should be 'LIKE'. Contains() works as intended for my initial predicate on FirstName, MiddleName etc. but not in the if (cascade) code block.
My C# search logic:
public IList<Individual> Find(string search, bool cascade, bool includeInactive)
{
_context.Database.Log = s => System.Diagnostics.Debug.WriteLine(s);
IQueryable<Individual> query = _context.Individuals;
if (!string.IsNullOrWhiteSpace(search))
{
search = search.Trim();
string[] searchParts = search.Split(' ');
ExpressionStarter<Individual> predicate = PredicateBuilder.New<Individual>(false);
foreach (string searchPart in searchParts)
{
predicate = predicate.Or(c =>
c.FirstName.Contains(searchPart) || c.MiddleNames.Contains(searchPart) ||
c.LastName.Contains(searchPart) || c.PreferredName.Contains(searchPart));
if (cascade)
{
predicate = predicate.Or(c =>
c.IndividualOrganisationGroups.Select(og => og.OrganisationGroup).Select(g => g.Group.Name)
.Contains(searchPart));
}
}
query = query.Where(predicate);
}
if (!includeInactive)
{
query = query.Where(c => c.Active);
}
return query.ToList();
}
The SQL generated from EF:
SELECT
[Extent1].[ID] AS [ID],
[Extent1].[RegistrationTypeID] AS [RegistrationTypeID],
[Extent1].[Title] AS [Title],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[MiddleNames] AS [MiddleNames],
[Extent1].[LastName] AS [LastName],
[Extent1].[PreferredName] AS [PreferredName],
[Extent1].[RegistrationNumber] AS [RegistrationNumber],
[Extent1].[Username] AS [Username],
[Extent1].[AzureID] AS [AzureID],
[Extent1].[Notes] AS [Notes],
[Extent1].[Active] AS [Active],
[Extent1].[CreatedDate] AS [CreatedDate],
[Extent1].[CreatedBy] AS [CreatedBy],
[Extent1].[UpdatedDate] AS [UpdatedDate],
[Extent1].[UpdatedBy] AS [UpdatedBy]
FROM [dbo].[Individual] AS [Extent1]
WHERE ([Extent1].[FirstName] LIKE #p__linq__0 ESCAPE '~') OR ([Extent1].[MiddleNames] LIKE #p__linq__1 ESCAPE '~') OR ([Extent1].[LastName] LIKE #p__linq__2 ESCAPE '~') OR ([Extent1].[PreferredName] LIKE #p__linq__3 ESCAPE '~') OR ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[IndividualOrganisationGroup] AS [Extent2]
INNER JOIN [dbo].[OrganisationGroup] AS [Extent3] ON [Extent2].[OrganisationGroupID] = [Extent3].[ID]
INNER JOIN [dbo].[Group] AS [Extent4] ON [Extent3].[GroupID] = [Extent4].[ID]
WHERE ([Extent1].[ID] = [Extent2].[IndividualID]) AND (([Extent4].[Name] = #p__linq__4) OR (1 = 0))
))
-- p__linq__0: '%screen%' (Type = AnsiString, Size = 8000)
-- p__linq__1: '%screen%' (Type = AnsiString, Size = 8000)
-- p__linq__2: '%screen%' (Type = AnsiString, Size = 8000)
-- p__linq__3: '%screen%' (Type = AnsiString, Size = 8000)
-- p__linq__4: 'screen' (Type = AnsiString, Size = 8000)
The problematic SQL (last WHERE clause):
WHERE ([Extent1].[ID] = [Extent2].[IndividualID]) AND (([Extent4].[Name] = #p__linq__4) OR (1 = 0))
What it should look like:
WHERE ([Extent1].[ID] = [Extent2].[IndividualID]) AND (([Extent4].[Name] LIKE #p__linq__4) OR (1 = 0))
So my question is, how do I make this code translate to a LIKE in my Entity Framework SQL?
predicate = predicate.Or(c =>
c.IndividualOrganisationGroups.Select(og => og.OrganisationGroup).Select(g => g.Group.Name)
.Contains(searchPart));

It has nothing to do with LINQKit PredicateBuilder. The reason is that the result of
c.IndividualOrganisationGroups.Select(og => og.OrganisationGroup).Select(g => g.Group.Name)
is IEnumerable<string> (or IQueryable<string>), so your are hitting the Enumerable (or Queryable) Contains method instead of the string.Contains as in the other places.
Instead of Select + Contains, what you really need is the Any extension method.
The most concise syntax with your sample is:
c => c.IndividualOrganisationGroups
.Any(og => og.OrganisationGroup.Group.Name.Contains(searchPart))
Of course if you need to check more properties of the Group (or even w/o that), then use Select + Any:
c => c.IndividualOrganisationGroups
.Select(og => og.OrganisationGroup.Group)
.Any(g => g.Name.Contains(searchPart))
or the most natural way of turning collection with filter to single bool by writing the query in a "normal" way. e.g. Select, Where etc. and putting parameterless Any at the end:
c => c.IndividualOrganisationGroups
.Select(og => og.OrganisationGroup.Group)
.Where(g => g.Name.Contains(searchPart))
.Any())

Have EF Linq Select statement Select a constant or a function

I have a Select statement that is currently formatted like
dbEntity
.GroupBy(x => x.date)
.Select(groupedDate => new {
Calculation1 = doCalculation1 ? x.Sum(groupedDate.Column1) : 0),
Calculation2 = doCalculation2 ? x.Count(groupedDate) : 0)
In the query doCalculation1 and doCalculation2 are bools that are set earlier. This creates a case statement in the Sql being generated, like
DECLARE #p1 int = 1
DECLARE #p2 int = 0
DECLARE #p3 int = 1
DECLARE #p4 int = 0
SELECT (Case When #p1 = 1 THEN Sum(dbEntity.Column1)
Else #p2
End) as Calculation1,
(Case When #p3 = 1 THEN Count(*)
Else #p4
End) as Calculation2
What I want to happen is for the generated sql is to be like this when doCalculation1 is true
SELECT SUM(Column1) as Calculation1, Count(*) as Calculation2
and like this when doCalculation2 is false
SELECT 0 as Calculation1, Count(*) as Calculation2
Is there any way to force a query through EF to act like this?
Edit:
bool doCalculation = true;
bool doCalculation2 = false;
dbEntity
.Where(x => x.FundType == "E")
.GroupBy(x => x.ReportDate)
.Select(dateGroup => new
{
ReportDate = dateGroup.Key,
CountInFlows = doCalculation2 ? dateGroup.Count(x => x.Flow > 0) : 0,
NetAssetEnd = doCalculation ? dateGroup.Sum(x => x.AssetsEnd) : 0
})
.ToList();
generates this sql
-- Region Parameters
DECLARE #p0 VarChar(1000) = 'E'
DECLARE #p1 Int = 0
DECLARE #p2 Decimal(5,4) = 0
DECLARE #p3 Int = 0
DECLARE #p4 Int = 1
DECLARE #p5 Decimal(1,0) = 0
-- EndRegion
SELECT [t1].[ReportDate],
(CASE
WHEN #p1 = 1 THEN (
SELECT COUNT(*)
FROM [dbEntity] AS [t2]
WHERE ([t2].[Flow] > #p2) AND ([t1].[ReportDate] = [t2].[ReportDate]) AND ([t2].[FundType] = #p0)
)
ELSE #p3
END) AS [CountInFlows],
(CASE
WHEN #p4 = 1 THEN CONVERT(Decimal(33,4),[t1].[value])
ELSE CONVERT(Decimal(33,4),#p5)
END) AS [NetAssetEnd]
FROM (
SELECT SUM([t0].[AssetsEnd]) AS [value], [t0].[ReportDate]
FROM [dbEntity] AS [t0]
WHERE [t0].[FundType] = #p0
GROUP BY [t0].[ReportDate]
) AS [t1]
which has many index scans and a spool and a join in the execution plan. It also takes about 20 seconds on average to run on the test set, with the production set going to be much larger.
I want it to run in the same speed as sql like
select reportdate, 1, sum(AssetsEnd)
from vwDailyFundFlowDetail
where fundtype = 'E'
group by reportdate
which runs in about 12 seconds on average and has the majority of the query tied up in a single index seek in the execution plan. What the actual sql output is doesnt matter, but the performance appears to be much worse with the case statements.
As for why I am doing this, I need to generate a dynamic select statements like I asked in Dynamically generate Linq Select. A user may select one or more of a set of calculations to perform and I will not know what is selected until the request comes in. The requests are expensive so we do not want to run them unless they are necessary. I am setting the doCalculation bools based on the user request.
This query is supposed to replace some code that inserts or deletes characters from a hardcoded sql query stored as a string, which is then executed. That runs fairly fast but is a nightmare to maintain

It would technically be possible to pass the Expression in your Select query through an expression tree visitor, which checks for constant values on the left-hand side of ternary operators, and replaces the ternary expression with the appropriate sub-expression.
For example:
public class Simplifier : ExpressionVisitor
{
public static Expression<T> Simplify<T>(Expression<T> expr)
{
return (Expression<T>) new Simplifier().Visit(expr);
}
protected override Expression VisitConditional(ConditionalExpression node)
{
var test = Visit(node.Test);
var ifTrue = Visit(node.IfTrue);
var ifFalse = Visit(node.IfFalse);
var testConst = test as ConstantExpression;
if(testConst != null)
{
var value = (bool) testConst.Value;
return value ? ifTrue : ifFalse;
}
return Expression.Condition(test, ifTrue, ifFalse);
}
protected override Expression VisitMember(MemberExpression node)
{
// Closed-over variables are represented as field accesses to fields on a constant object.
var field = (node.Member as FieldInfo);
var closure = (node.Expression as ConstantExpression);
if(closure != null)
{
var value = field.GetValue(closure.Value);
return VisitConstant(Expression.Constant(value));
}
return base.VisitMember(node);
}
}
Usage example:
void Main()
{
var b = true;
Expression<Func<int, object>> expr = i => b ? i.ToString() : "N/A";
Console.WriteLine(expr.ToString()); // i => IIF(value(UserQuery+<>c__DisplayClass0).b, i.ToString(), "N/A")
Console.WriteLine(Simplifier.Simplify(expr).ToString()); // i => i.ToString()
b = false;
Console.WriteLine(Simplifier.Simplify(expr).ToString()); // i => "N/A"
}
So, you could use this in your code something like this:
Expression<Func<IGrouping<DateTime, MyEntity>>, ClassYouWantToReturn> select =
groupedDate => new {
Calculation1 = doCalculation1 ? x.Sum(groupedDate.Column1) : 0),
Calculation2 = doCalculation2 ? x.Count(groupedDate) : 0
};
var q = dbEntity
.GroupBy(x => x.date)
.Select(Simplifier.Simplify(select))
However, this is probably more trouble than it's worth. SQL Server will almost undoubtedly optimize the "1 == 1" case away, and allowing Entity Framework to produce the less-pretty query shouldn't prove to be a performance problem.
Update
Looking at the updated question, this appears to be one of the few instances where producing the right query really does matter, performance-wise.
Besides my suggested solution, there are a few other choices: you could use raw sql to map to your return type, or you could use LinqKit to choose a different expression based on what you want, and then "Invoke" that expression inside your Select query.

Entity Framework 6: Are Navigation Properties Smart?

So, I already know that code like this does everything in memory:
SomeDbObject.SomeNavigationProperty.Where(x => x.IsSomeBool);
In addition to this (still in memory):
SomeDbObject.SomeNavigationProperty.AsQueryable().Where(x => x.IsSomeBool);
So, I came up with a friendly solution that helps me make sure that the whole call will be executed as a SQL command (this is just a property on SomeDbObject):
public IQueryable<AnotherDbObject> QueryableSomeNavigationProperty
{
get
{
return SomeStaticContext.AnotherDbObjects.Where(x => x.ForeignKeyForSomeDbObject == this.Id);
}
}
So, as you can see, this basically does what the Navigation property would have done, but it just creates the command in expression-treeable format, so that subsequent clauses will be built into the SQL command. For example, the same statement as before, would now return an IQueryable upon which we can add a Where clause:
SomeDbObject.QueryableSomeNavigationProperty.Where(x => x.IsSomeBool);
Now, the question is, what happens if I want to query another navigation property in the where clause. For example:
SomeDbObject.QueryableSomeNavigationProperty.Where(x => SomeDbObject.AnotherNavigationProperty.Any());
So, do I need to make another method that returns an IQueryable for SomeDbObject.AnotherNavigationProperty? Or, does EF do the right thing here and build it into a SQL statement?
I can clarify, if needed, but I think this covers the gist of what I am looking for.
Thanks!

Ok, everyone. I ran a bunch of simulations and the verdict is in. See the comments for results in each scenario. Comments denote when the SQL came spitting out! :)
Hope this helps the next poor soul who is confused by what EF6 does at what point!
class Program
{
private static readonly Action<string> DebugWriteLine = s => System.Diagnostics.Debug.WriteLine(s);
private static readonly Action<string> WriteLine = s => { System.Console.WriteLine(s); DebugWriteLine(s); };
static void Main(string[] args)
{
Statics.Entities.Database.Log = WriteLine;
WhereClauseOnSimpleProperty();
WhereClauseOnNavigationProperty();
WhereClauseOnICollection();
WhereClauseOnIQueryable();
WhereClauseOnIQueryableWithIQueryable();
System.Console.ReadKey();
}
static void WhereClauseOnSimpleProperty()
{
WriteLine("Get objects with a where clause (simple property).");
WriteLine(" Calling: var users = entities.Users.Where(u => u.FirstName == \"Julie\");");
var users = Statics.Entities.Users.Where(u => u.FirstName == "Julie");
WriteLine(" Calling: users.ToList();");
var usersList = users.ToList();
// SQL got built and called here (NOTE: SQL call is not made until the data needs to be "realized"):
/* SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Uin] AS [Uin],
[Extent1].[ClientId] AS [ClientId],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName]
FROM [dbo].[Users] AS [Extent1]
WHERE 'Julie' = [Extent1].[FirstName]
*/
WriteLine(" There are " + usersList.Count + " users.");
}
static void WhereClauseOnNavigationProperty()
{
WriteLine("Get objects with a where clause (1-to-many navigation property).");
WriteLine(" Calling: var users = Entities.Users.Where(u => u.FirstName == \"Julie\" && u.Votes.Any());");
var users = Statics.Entities.Users.Where(u => u.FirstName == "Julie" && u.Votes.Any());
WriteLine(" Calling: users.ToList();");
var usersList = users.ToList();
// SQL got built and called here (NOTE: using the ICollection navigation property on the lambda parameter "u" builds just one SQL statement):
/* SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Uin] AS [Uin],
[Extent1].[ClientId] AS [ClientId],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName]
FROM [dbo].[Users] AS [Extent1]
WHERE ('Julie' = [Extent1].[FirstName]) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[Votes] AS [Extent2]
WHERE [Extent1].[Id] = [Extent2].[UserId]
))
*/
WriteLine(" There are " + usersList.Count + " users.");
}
static void WhereClauseOnICollection()
{
WriteLine("Get objects with a where clause (simple property) from an ICollection.");
WriteLine(" Calling: var users = Entities.Users.First(u => u.FirstName == \"Julie\" && u.Votes.Any());");
var user = Statics.Entities.Users.First(u => u.FirstName == "Julie" && u.Votes.Any());
// SQL got built and called here (NOTE: data is realized immediately because we are allocating a single object):
/* SELECT TOP (1)
[Extent1].[Id] AS [Id],
[Extent1].[Uin] AS [Uin],
[Extent1].[ClientId] AS [ClientId],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName]
FROM [dbo].[Users] AS [Extent1]
WHERE ('Julie' = [Extent1].[FirstName]) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[Votes] AS [Extent2]
WHERE [Extent1].[Id] = [Extent2].[UserId]
))
*/
WriteLine(" Calling: var votes = user.Votes.AsQueryable().Where(v => v.VoteValue > 0);");
var votes = user.Votes.AsQueryable().Where(v => v.VoteValue > 0);
// SQL got built and called here (NOTE: there "where" clause is executed in app memory/time [it's not in the SQL call]):
/* SELECT
[Extent1].[Id] AS [Id],
[Extent1].[UserId] AS [UserId],
[Extent1].[VoteValue] AS [VoteValue]
FROM [dbo].[Votes] AS [Extent1]
WHERE [Extent1].[UserId] = #EntityKeyValue1
*/
WriteLine(" Calling: votes.ToList();");
var votesList = votes.ToList();
WriteLine(" There are " + votesList.Count + " votes.");
}
static void WhereClauseOnIQueryable()
{
WriteLine("Get objects with a where clause (1-to-many navigation property) from an IQueryable.");
WriteLine(" Calling: var users = Entities.Users.First(u => u.FirstName == \"Julie\" && u.Votes.Any());");
var user = Statics.Entities.Users.First(u => u.FirstName == "Julie" && u.Votes.Any());
// SQL got built and called here:
/* SELECT TOP (1)
[Extent1].[Id] AS [Id],
[Extent1].[Uin] AS [Uin],
[Extent1].[ClientId] AS [ClientId],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName]
FROM [dbo].[Users] AS [Extent1]
WHERE ('Julie' = [Extent1].[FirstName]) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[Votes] AS [Extent2]
WHERE [Extent1].[Id] = [Extent2].[UserId]
))
*/
WriteLine(" Calling: var votes = user.QueryableVotes.Where(v => user.Votes.AsQueryable().Contains(v));");
var votes = user.QueryableVotes.Where(v => user.Votes.AsQueryable().Contains(v));
// SQL got built and called here (NOTE: this is just the "user.Votes.AsQueryable().Contains(v)" part of the query):
/* SELECT
[Extent1].[Id] AS [Id],
[Extent1].[UserId] AS [UserId],
[Extent1].[VoteValue] AS [VoteValue]
FROM [dbo].[Votes] AS [Extent1]
WHERE [Extent1].[UserId] = #EntityKeyValue1
*/
WriteLine(" Calling: votes.ToList();");
var votesList = votes.ToList();
// NOTE: EF6 dies here because it had already computed "user.Votes.Contains(v)" (see above), and that can't go into the query.
WriteLine(" There are " + votesList.Count + " votes.");
}
static void WhereClauseOnIQueryableWithIQueryable()
{
WriteLine("Get objects with a where clause (1-to-many navigation property as an IQueryable) from an IQueryable.");
WriteLine(" Calling: var users = Entities.Users.First(u => u.FirstName == \"Julie\" && u.Votes.Any());");
var user = Statics.Entities.Users.First(u => u.FirstName == "Julie" && u.Votes.Any());
// SQL got built and called here:
/* SELECT TOP (1)
[Extent1].[Id] AS [Id],
[Extent1].[Uin] AS [Uin],
[Extent1].[ClientId] AS [ClientId],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName]
FROM [dbo].[Users] AS [Extent1]
WHERE ('Julie' = [Extent1].[FirstName]) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[Votes] AS [Extent2]
WHERE [Extent1].[Id] = [Extent2].[UserId]
))
*/
WriteLine(" Calling: var votes = user.QueryableVotes.Where(v => user.QueryableVotes.Contains(v));");
var votes = user.QueryableVotes.Where(v => user.QueryableVotes.Contains(v)); // Yes, I know this is reduntant...just making sure the SQL looks right.
WriteLine(" Calling: votes.ToList();");
var votesList = votes.ToList();
// SQL got built and called here (NOTE: making all expressions true IQueryables will build the "correct" [one call to rule them all] SQL expression):
/* SELECT
[Extent1].[Id] AS [Id],
[Extent1].[UserId] AS [UserId],
[Extent1].[VoteValue] AS [VoteValue]
FROM [dbo].[Votes] AS [Extent1]
WHERE ([Extent1].[UserId] = #p__linq__0) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[Votes] AS [Extent2]
WHERE ([Extent2].[UserId] = #p__linq__1) AND ([Extent2].[Id] = [Extent1].[Id])
))
*/
WriteLine(" There are " + votesList.Count + " votes.");
}
// SPECIAL NOTE: The clauses should follow these guidelines:
/*
* 1. If the condition operates on the lambda parameter, then use the ICollection navigation property to achieve one statement.
* For example: var user = Statics.Entities.Users.First(u => u.FirstName == "Julie" && u.Votes.Any());
* 2. If the condition operates on a "non-navigation" property of the lambda parameter, then use the IQueryable expression to acheive one statement.
* For example: var votes = user.QueryableVotes.Where(v => user.QueryableVotes.Contains(v));
*/
}
public partial class User
{
public IQueryable<Vote> QueryableVotes
{
get
{
return Statics.Entities.Votes.Where(v => v.UserId == this.Id);
}
}
}

linq to sql startwith performance indexed columns

I use entity code first.
Indexed columns:
SourceCatalogId
Disabled
CategoryPath
40 000 rows in the Table,
My problem is the query takes 40s!!
var result = DBContext.Set<SourceProduct>()
.Include(x => x.SalesHistories, x => x.SourceCatalog)
.Where(p => p.SourceCatalogId == 2)
.where(p => p.Disabled == false)
.where(x => x.CategoryPath.StartsWith("MyPath"))
.orderby(x => x.ShortDesignation)
.Skip(1)
.Take(10)
.toList();
SQL via sql profiler:
exec sp_executesql N'SELECT TOP (10)
[Project1].[SourceProductId] AS [SourceProductId],
[Project1].[SourceSKU] AS [SourceSKU],
[Project1].[SourceCatalogId] AS [SourceCatalogId],
[Project1].[ManufacturerReference] AS [ManufacturerReference],
[Project1].[Disabled] AS [Disabled],
[Project1].[EAN] AS [EAN],
[Project1].[ShortDesignation] AS [ShortDesignation],
[Project1].[FullDesignation] AS [FullDesignation],
[Project1].[Description] AS [Description],
[Project1].[Url] AS [Url],
[Project1].[CategoryPath] AS [CategoryPath],
[Project1].[Condition] AS [Condition],
[Project1].[BuyingPriceHT] AS [BuyingPriceHT],
[Project1].[ShippingPriceHT] AS [ShippingPriceHT],
[Project1].[PublicSellingPriceHT] AS [PublicSellingPriceHT],
[Project1].[PictureUrl1] AS [PictureUrl1],
[Project1].[PictureUrl2] AS [PictureUrl2],
[Project1].[PictureUrl3] AS [PictureUrl3],
[Project1].[PictureUrl4] AS [PictureUrl4],
[Project1].[Quantity] AS [Quantity],
[Project1].[AddDate] AS [AddDate],
[Project1].[UpdateDate] AS [UpdateDate],
[Project1].[Followers] AS [Followers]
FROM ( SELECT [Project1].[SourceProductId] AS [SourceProductId], [Project1].[SourceSKU] AS [SourceSKU], [Project1].[SourceCatalogId] AS [SourceCatalogId], [Project1].[ManufacturerReference] AS [ManufacturerReference], [Project1].[Disabled] AS [Disabled], [Project1].[EAN] AS [EAN], [Project1].[ShortDesignation] AS [ShortDesignation], [Project1].[FullDesignation] AS [FullDesignation], [Project1].[Description] AS [Description], [Project1].[Url] AS [Url], [Project1].[CategoryPath] AS [CategoryPath], [Project1].[Condition] AS [Condition], [Project1].[BuyingPriceHT] AS [BuyingPriceHT], [Project1].[ShippingPriceHT] AS [ShippingPriceHT], [Project1].[PublicSellingPriceHT] AS [PublicSellingPriceHT], [Project1].[PictureUrl1] AS [PictureUrl1], [Project1].[PictureUrl2] AS [PictureUrl2], [Project1].[PictureUrl3] AS [PictureUrl3], [Project1].[PictureUrl4] AS [PictureUrl4], [Project1].[Quantity] AS [Quantity], [Project1].[AddDate] AS [AddDate], [Project1].[UpdateDate] AS [UpdateDate], [Project1].[Followers] AS [Followers], row_number() OVER (ORDER BY [Project1].[ShortDesignation] ASC) AS [row_number]
FROM ( SELECT
[Extent1].[SourceProductId] AS [SourceProductId],
[Extent1].[SourceSKU] AS [SourceSKU],
[Extent1].[SourceCatalogId] AS [SourceCatalogId],
[Extent1].[ManufacturerReference] AS [ManufacturerReference],
[Extent1].[Disabled] AS [Disabled],
[Extent1].[EAN] AS [EAN],
[Extent1].[ShortDesignation] AS [ShortDesignation],
[Extent1].[FullDesignation] AS [FullDesignation],
[Extent1].[Description] AS [Description],
[Extent1].[Url] AS [Url],
[Extent1].[CategoryPath] AS [CategoryPath],
[Extent1].[Condition] AS [Condition],
[Extent1].[BuyingPriceHT] AS [BuyingPriceHT],
[Extent1].[ShippingPriceHT] AS [ShippingPriceHT],
[Extent1].[PublicSellingPriceHT] AS [PublicSellingPriceHT],
[Extent1].[PictureUrl1] AS [PictureUrl1],
[Extent1].[PictureUrl2] AS [PictureUrl2],
[Extent1].[PictureUrl3] AS [PictureUrl3],
[Extent1].[PictureUrl4] AS [PictureUrl4],
[Extent1].[Quantity] AS [Quantity],
[Extent1].[AddDate] AS [AddDate],
[Extent1].[UpdateDate] AS [UpdateDate],
[Extent1].[Followers] AS [Followers]
FROM [dbo].[SourceProducts] AS [Extent1]
WHERE ([Extent1].[SourceCatalogId] = #p__linq__0) AND (0 = [Extent1].[Disabled]) AND ([Extent1].[CategoryPath] LIKE #p__linq__1 ESCAPE N''~'')
) AS [Project1]
) AS [Project1]
WHERE [Project1].[row_number] > 0
ORDER BY [Project1].[ShortDesignation] ASC',N'#p__linq__0 bigint,#p__linq__1 nvarchar(4000)',#p__linq__0=2,#p__linq__1=N'MyPath%'
In the last one before where clause, if I remove "escape N''~''" in:
WHERE ([Extent1].[SourceCatalogId] = #p__linq__0) AND (0 = [Extent1].[Disabled]) AND ([Extent1].[CategoryPath] LIKE #p__linq__1 ESCAPE N''~'')
the query takes 4s.
Is it normal ? Index uses ? How i can solve it with startWith ?
EDIT
Index attribut for categoryPath:
[Index("IX_SourceProduct_SourceCatalogId_Disabled_CategoryPath", 3), StringLength(400)]
public string CategoryPath { get; set; }
EDIT2
OK i thing that I'm pretty close, I think the probleme is stored procedure.
string search = "julien";
var list = db.Users.Where(x => x.Name.StartsWith(search));
string query = list.ToString();
=>
SELECT
[Extent1].[UserId] AS [UserId],
[Extent1].[Name] AS [Name]
FROM [dbo].[Users] AS [Extent1]
WHERE [Extent1].[Name] LIKE #p__linq__0 ESCAPE N'~'
var list2 = db.Users.Where(x => x.Name.StartsWith("julien"));
string query2 = list2.ToString();
=>
SELECT
[Extent1].[UserId] AS [UserId],
[Extent1].[Name] AS [Name]
FROM [dbo].[Users] AS [Extent1]
WHERE [Extent1].[Name] LIKE N'julien%'
So if I use variable in the query in get a stored procedure, if I use const I get select.
In the stored procedure( generated by entity) makes appear #p__linq__0 so add ESCAPE N'~'
to avoid wildCaractere in the variable.
So now the question is simplier. How avoid query with variable ? it's possible ?
thanks

So what you need to do here is take the value of a variable and use it as a constant in an Expression that you are generating. This is actually quite possible. What we'll need is an expression that accepts the parameter you want as the parameter of your real selector, as second parameter that is a placeholder for the constant value, and then the value that you want to be a constant. We can then replace all instances of the parameter with the value of the constant, leaving just a function that maps the real parameter to the result:
public static Expression<Func<TSource, TResult>> EmbedConstant
<TSource, TResult, TConstant>(
this Expression<Func<TSource, TConstant, TResult>> expression,
TConstant constant)
{
var body = expression.Body.Replace(
expression.Parameters[1],
Expression.Constant(constant));
return Expression.Lambda<Func<TSource, TResult>>(
body, expression.Parameters[0]);
}
This relies on the following methods for replacing all instances of one expression with another:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
This allows you to map this:
string search = "julien";
var list = db.Users.Where(x => x.Name.StartsWith(search));
string query = list.ToString();
Into this:
string search = "julien";
Expression<Func<User, string, bool>> predicate =
(item, searchTerm) => item.Name.StartsWith(searchTerm);
var list = db.Users.Where(predicate.EmbedConstant(search));
string query = list.ToString();

Force Entity Framework to use SQL parameterization for better SQL proc cache reuse

Entity Framework always seems to use constants in generated SQL for values provided to Skip() and Take().
In the ultra-simplified example below:
int x = 10;
int y = 10;
var stuff = context.Users
.OrderBy(u => u.Id)
.Skip(x)
.Take(y)
.Select(u => u.Id)
.ToList();
x = 20;
var stuff2 = context.Users
.OrderBy(u => u.Id)
.Skip(x)
.Take(y)
.Select(u => u.Id)
.ToList();
the above code generates the following SQL queries:
SELECT TOP (10)
[Extent1].[Id] AS [Id]
FROM ( SELECT [Extent1].[Id] AS [Id], row_number() OVER (ORDER BY [Extent1].[Id] ASC) AS [row_number]
FROM [dbo].[User] AS [Extent1]
) AS [Extent1]
WHERE [Extent1].[row_number] > 10
ORDER BY [Extent1].[Id] ASC
SELECT TOP (10)
[Extent1].[Id] AS [Id]
FROM ( SELECT [Extent1].[Id] AS [Id], row_number() OVER (ORDER BY [Extent1].[Id] ASC) AS [row_number]
FROM [dbo].[User] AS [Extent1]
) AS [Extent1]
WHERE [Extent1].[row_number] > 20
ORDER BY [Extent1].[Id] ASC
Resulting in 2 Adhoc plans added to the SQL proc cache with 1 use each.
What I'd like to accomplish is to parameterize the Skip() and Take() logic so the following SQL queries are generated:
EXEC sp_executesql N'SELECT TOP (#p__linq__0)
[Extent1].[Id] AS [Id]
FROM ( SELECT [Extent1].[Id] AS [Id], row_number() OVER (ORDER BY [Extent1].[Id] ASC) AS [row_number]
FROM [dbo].[User] AS [Extent1]
) AS [Extent1]
WHERE [Extent1].[row_number] > #p__linq__1
ORDER BY [Extent1].[Id] ASC',N'#p__linq__0 int,#p__linq__1 int',#p__linq__0=10,#p__linq__1=10
EXEC sp_executesql N'SELECT TOP (#p__linq__0)
[Extent1].[Id] AS [Id]
FROM ( SELECT [Extent1].[Id] AS [Id], row_number() OVER (ORDER BY [Extent1].[Id] ASC) AS [row_number]
FROM [dbo].[User] AS [Extent1]
) AS [Extent1]
WHERE [Extent1].[row_number] > #p__linq__1
ORDER BY [Extent1].[Id] ASC',N'#p__linq__0 int,#p__linq__1 int',#p__linq__0=10,#p__linq__1=20
This results in 1 Prepared plan added to the SQL proc cache with 2 uses.
I have some fairly complex queries and am experiencing significant overhead (on the SQL Server side) on the first run, and much faster execution on subsequent runs (since it can use the plan cache). Note that these more advanced queries already use sp_executesql as other values are parameterized so I'm not concerned about that aspect.
The first set of queries generated above basically means any pagination logic will create a new entry in the plan cache for each page, bloating the cache and requiring the plan generation overhead to be incurred for each page.
Can I force Entity Framework to parameterize values? I've noticed for other values e.g. in Where clauses, sometimes it parameterizes values, and sometimes it uses constants.
Am I completely out to lunch? Is there any reason why Entity Framework's existing behavior is better than the behavior I desire?
Edit:
In case it's relevant, I should mention that I'm using Entity Framework 4.2.
Edit 2:
This question is not a duplicate of Entity Framework/Linq to SQL: Skip & Take, which merely asks how to ensure that Skip and Take execute in SQL instead of on the client. This question pertains to parameterizing these values.

Update: the Skip and Take extension methods that take lambda parameters described below are part of Entity Framework from version 6 and onwards. You can take advantage of them by importing the System.Data.Entity namespace in your code.
In general LINQ to Entities translates constants as constants and variables passed to the query into parameters.
The problem is that the Queryable versions of Skip and Take accept simple integer parameters and not lambda expressions, therefore while LINQ to Entities can see the values you pass, it cannot see the fact that you used a variable to pass them (in other words, methods like Skip and Take don't have access to the method's closure).
This not only affects the parameterization in LINQ to Entities but also the learned expectation that if you pass a variable to a LINQ query the latest value of the variable is used every time you re-execute the query. E.g., something like this works for Where but not for Skip or Take:
var letter = "";
var q = from db.Beattles.Where(p => p.Name.StartsWith(letter));
letter = "p";
var beattle1 = q.First(); // Returns Paul
letter = "j";
var beattle2 = q.First(); // Returns John
Note that the same peculiarity also affects ElementAt but this one is currently not supported by LINQ to Entities.
Here is a trick that you can use to force the parameterization of Skip and Take and at the same time make them behave more like other query operators:
public static class PagingExtensions
{
private static readonly MethodInfo SkipMethodInfo =
typeof(Queryable).GetMethod("Skip");
public static IQueryable<TSource> Skip<TSource>(
this IQueryable<TSource> source,
Expression<Func<int>> countAccessor)
{
return Parameterize(SkipMethodInfo, source, countAccessor);
}
private static readonly MethodInfo TakeMethodInfo =
typeof(Queryable).GetMethod("Take");
public static IQueryable<TSource> Take<TSource>(
this IQueryable<TSource> source,
Expression<Func<int>> countAccessor)
{
return Parameterize(TakeMethodInfo, source, countAccessor);
}
private static IQueryable<TSource> Parameterize<TSource, TParameter>(
MethodInfo methodInfo,
IQueryable<TSource> source,
Expression<Func<TParameter>> parameterAccessor)
{
if (source == null)
throw new ArgumentNullException("source");
if (parameterAccessor == null)
throw new ArgumentNullException("parameterAccessor");
return source.Provider.CreateQuery<TSource>(
Expression.Call(
null,
methodInfo.MakeGenericMethod(new[] { typeof(TSource) }),
new[] { source.Expression, parameterAccessor.Body }));
}
}
The class above defines new overloads of Skip and Take that expect a lambda expression and can hence capture variables. Using the methods like this will result in the variables being translated to parameters by LINQ to Entities:
int x = 10;
int y = 10;
var query = context.Users.OrderBy(u => u.Id).Skip(() => x).Take(() => y);
var result1 = query.ToList();
x = 20;
var result2 = query.ToList();
Hope this helps.

The methods Skip and Top of ObjectQuery<T> can be parametrized. There is an example at MSDN.
I did a similar thing in a model of my own and sql server profiler showed the parts
SELECT TOP (#limit)
and
WHERE [Extent1].[row_number] > #skip
So, yes. It can be done. And I agree with others that this is a valuable observation you made here.