We Keep Coding

How to use DbFunction translation in EF Core?

I'm looking for something like EF.Functions.FreeText that was implemented in SQL Server but using the MATCH...AGAINST syntax of MySQL.
This is my current workflow:
AspNetCore 2.1.1
EntityFrameworkCore 2.1.4
Pomelo.EntityFrameworkCore.MySql 2.1.4
The problem is that MySQL uses two functions and I don't know how to interpret that with DbFunction and separate the arguments for each one. Does anyone know how to implement this?
This should be the Linq syntax:
query.Where(x => DbContext.FullText(new[] { x.Col1, x.Col2, x.Col3 }, "keywords"));
And this should be the result generated in SQL:
SELECT * FROM t WHERE MATCH(`Col1`, `Col2`, `Col3`) AGAINST('keywords');
I'm trying to follow the following examples using the HasTranslation function:
https://github.com/aspnet/EntityFrameworkCore/issues/11295#issuecomment-511440395
https://github.com/aspnet/EntityFrameworkCore/issues/10241#issuecomment-342989770
Note: I know it can be solved with FromSql, but it's not what I'm looking for.

Your use case is very similar to mine when I needed ROW_NUMBER support in EF Core.
Example:
// gets translated to
// ROW_NUMBER() OVER(PARTITION BY ProductId ORDER BY OrderId, Count)
DbContext.OrderItems.Select(o => new {
RowNumber = EF.Functions.RowNumber(o.ProductId, new {
o.OrderId,
o.Count
})
})
Use anonymous classes instead of arrays
The first thing you have to do is to switch from using an array to an anonymous class, i.e. you change the call from
DbContext.FullText(new[] { x.Col1, x.Col2, x.Col3 }, "keywords")
to
DbContext.FullText(new { x.Col1, x.Col2, x.Col3 }, "keywords")
The sort order of the parameters will stay as it is defined by in the query,
i.e new { x.Col1, x.Col2 } will be translated to Col1, Col2
and new { x.Col2, x.Col1 } to Col2, Col1.
You can even to the following: new { x.Col1, _ = x.Col1, Foo = "bar" } that is going to be translated to Col1, Col1, 'bar'.
Implement custom IMethodCallTranslator
If you need some hints then you can look through my code on Azure DevOps: RowNumber Support or if you can wait a few days then I will provide a blog post about the implementation of custom functions.
Updated (31 july, 2019)
Blog posts:
Entity Framework Core: Custom Functions (using IMethodCallTranslator)
Entity Framework Core: Custom Functions (using HasDbFunction)
Updated (july 27, 2019)
Thanks to the comments below I see that some clarification is required.
1) As pointed out in the comment below there is another approach. With HasDbFunction I could save me some typing like the code for registration of the translator with EF but I would still need the RowNumberExpression because the function has 2 sets of parameters (for PARTITION BY and ORDER BY) and the existing SqlFunctionExpression doesn't support that. (or did I missed something?) The reason I've chosen the approach with IMethodCallTranslator is because I want the configuration of this feature to be done during setting up of the DbContextOptionsBuilder and not in OnModelCreating. That is, it’s a personal preference of mine.
In the end the thread creator can use HasDbFunction to implement the desired feature as well. In my case the code would look something like the following:
// OnModelCreating
var methodInfo = typeof(DemoDbContext).GetMethod(nameof(DemoRowNumber));
modelBuilder.HasDbFunction(methodInfo)
.HasTranslation(expressions => {
var partitionBy = (Expression[])((ConstantExpression)expressions.First()).Value;
var orderBy = (Expression[])((ConstantExpression)expressions.Skip(1).First()).Value;
return new RowNumberExpression(partitionBy, orderBy);
});
// the usage with this approach is identical to my current approach
.Select(c => new {
RowNumber = DemoDbContext.DemoRowNumber(
new { c.Id },
new { c.RowVersion })
})
2) An anonymous type can’t enforce the type(s) of its members, so you can get a runtime exception if the function is called with, say, integer instead of string. Still, it can be valid solution. Depending on the customer you are working for the solution may be more or less viable, in the end the decision lies with the customer. Not providing any alternatives is a possible solution as well but not a satisfying one.
Especially, if the usage of SQL is not desired (because you get even less support from compiler) so the runtime exception may be a good compromise after all.
But, if the compromise is still not acceptable then we can make a research on how to add support for arrays.
First approach could be the implementation of a custom IExpressionFragmentTranslator to “redirect” the handling of arrays to us.
Please note, it is just a prototype and needs more investigation/testing :-)
// to get into EF pipeline
public class DemoArrayTranslator : IExpressionFragmentTranslator
{
public Expression Translate(Expression expression)
{
if (expression?.NodeType == ExpressionType.NewArrayInit)
{
var arrayInit = (NewArrayExpression)expression;
return new DemoArrayInitExpression(arrayInit.Type, arrayInit.Expressions);
}
return null;
}
}
// lets visitors visit the array-elements
public class DemoArrayInitExpression : Expression
{
private readonly ReadOnlyCollection<Expression> _expressions;
public override Type Type { get; }
public override ExpressionType NodeType => ExpressionType.Extension;
public DemoArrayInitExpression(Type type,
ReadOnlyCollection<Expression> expressions)
{
Type = type ?? throw new ArgumentNullException(nameof(type));
_expressions = expressions ?? throw new ArgumentNullException(nameof(expressions));
}
protected override Expression Accept(ExpressionVisitor visitor)
{
var visitedExpression = visitor.Visit(_expressions);
return NewArrayInit(Type.GetElementType(), visitedExpression);
}
}
// adds our DemoArrayTranslator to the others
public class DemoRelationalCompositeExpressionFragmentTranslator
: RelationalCompositeExpressionFragmentTranslator
{
public DemoRelationalCompositeExpressionFragmentTranslator(
RelationalCompositeExpressionFragmentTranslatorDependencies dependencies)
: base(dependencies)
{
AddTranslators(new[] { new DemoArrayTranslator() });
}
}
// Register the translator
services
.AddDbContext<DemoDbContext>(builder => builder
.ReplaceService<IExpressionFragmentTranslator,
DemoRelationalCompositeExpressionFragmentTranslator>());
For testing I introduced another overload containing Guid[] as parameter.
Although, this method doesn't make sense in my use case at all :)
public static long RowNumber(this DbFunctions _, Guid[] orderBy)
And adjusted the usage of the method
// Translates to ROW_NUMBER() OVER(ORDER BY Id)
.Select(c => new {
RowNumber = EF.Functions.RowNumber(new Guid[] { c.Id })
})

It is possible to query a NotMapped property?

i'm working with EF6 code first, and i used this answer to map a List<stirng> in my entitie.
This is my class
[Key]
public string SubRubro { get; set; }
[Column]
private string SubrubrosAbarcados
{
get
{
return ListaEspecifica == null || !ListaEspecifica.Any() ? null : JsonConvert.SerializeObject(ListaEspecifica);
}
set
{
if (string.IsNullOrWhiteSpace(value))
ListaEspecifica.Clear();
else
ListaEspecifica = JsonConvert.DeserializeObject<List<string>>(value);
}
}
[NotMapped]
public List<string> ListaEspecifica { get; set; } = new List<string>();
It works perfectly to storage my list as Json, but now i need to perform a linq query, and i'm trying this
var c = db.CategoriaAccesorios.Where(c => c.ListaEspecifica.Contains("Buc")).First();
And it's throwing
System.NotSupportedException: The specified type member
'ListaEspecifica' is not supported in LINQ to Entities. Only
initializers, entity members, and entity navigation properties are
supported.
what is logical.
Is there any way to perform a query like this?

The problem here is that LINQ to Entities does not understand how to convert your query to the back-end (SQL) language. Because you're not materializing (i.e. converting to .NET) the results of the query until you filter it, LINQ tries to convert your query to SQL itself. Since it's not sure how to do that, you get a NotSupportedException.
If you materialize the query first (I.e. call a .ToList()) then filter, things will work fine. I suspect this isn't what you want, though. (I.e. db.CategoriaAccesorios.ToList().Where(c => c.ListaEspecifica.Contains("Buc")).First();)
As this answer explains, your issue is the EF to SQL Conversion. Obviously you want some way to workaround it, though.
Because you are JSON serializing, there are actually a couple options here, most particularly using a LIKE:
var c =
(from category
in db.CategoriaAccessorios
where SqlMethods.Like(c.SubrubrosAbarcados, "%\"Buc\"%")
select category).First()
If EF Core, allegedly Microsoft.EntityFrameworkCore.EF.Functions.Like should replace SqlMethods.Like.
If you have SQL Server 2016+, and force the SubrubrosAbarcados to be a JSON type, it should be possible to use a raw query to directly query the JSON column in particular.
If you're curious about said aspect, here's a sample of what it could look like in SQL Server 2016:
CREATE TABLE Test (JsonData NVARCHAR(MAX))
INSERT INTO Test (JsonData) VALUES ('["Test"]'), ('["Something"]')
SELECT * FROM Test CROSS APPLY OPENJSON(JsonData, '$') WITH (Value VARCHAR(100) '$') AS n WHERE n.Value = 'Test'
DROP TABLE Test

I was able to do something like this via CompiledExpression.
using Microsoft.Linq.Translations;
// (...) namespace, class, etc
private static readonly CompiledExpression<MyClass, List<string>> _myExpression = DefaultTranslationOf<MyClass>
.Property(x => x.MyProperty)
.Is(x => new List<string>());
[NotMapped]
public List<string> MyProperty
{
get { return _myExpression.Evaluate(this); }
}
I hope there are better / prettier solutions though ;)

MySQL Linq using .Contains(variable)

Setup info:
VS2013 / C#
EF6
MySQL database
.Net Connector 6.9.5
I'm trying to create a method that returns a collection of Account records using a partial name as the search criteria. If I hard code a string value using the IQueryable .Contains() extension method, it returns data. However, when I attempt to use a variable no data is returned.
Public class Test() {
MyEntities db = new MyEntities();
//Works....but the search criteria is hard coded.
public IQueryable<Account> WorksButValueHardCoded() {
return (from a in db.Accounts
where a.accountname.Contains("Test")
select a);
}
//Does not return anything
public IQueryable<Account> DoesNotReturnAnyData() {
//Obviously I would use a parameter, but even this test fails
string searchText = "Test";
return (from a in db.Accounts
where a.accountname.Contains(searchText)
select a);
}
}
I can see in the LINQ generated SQL used the LIKE operator, but I don't understand how the variable is injected as it reads:
SELECT
`Extent1`.`accountid`,
`Extent1`.`accountname`
FROM `account` AS `Extent1`
WHERE `Extent1`.`accountname` LIKE '%p__linq__0%'
So...why does it work with the hard coded value and not a string variable?

I ran into the same problem and followed the error step by step with Glimpse (nice tool to inspect what the server is doing). It turned out that the SQL-Statement is built correctly, because I got results by executing it on the database.
The problem could be the replacement of the string variables in the statement. I guess LINQ isn't replacing just the string you pass but fills the variable with spaces to the VARCHAR limit so your query looks like
SELECT`Extent1`.`accountid`, `Extent1`.`accountname`
FROM `account` AS `Extent1`
WHERE `Extent1`.`accountname` LIKE '%Test ... %'
Add
.Trim()
to your string and it works.
public IQueryable<Account> DoesNotReturnAnyData() {
string searchText = "Test";
// Use Trim() here
return (from a in db.Accounts
where a.accountname.Contains(searchText.Trim())
select a);
}

There is nothing wrong in theory i can see
Update
This is working fine for me on sqlServer
public IQueryable<Account> DoesNotReturnAnyData(MyEntities db,string searchText) {
return (from a in db.Accounts
where a.accountname.Contains(searchText )
select a)
}

This is a reported bug with MySQL Entity Framework 6.9.5
Bug #74918 : Incorrect query result with Entity Framework 6:
https://bugs.mysql.com/bug.php?id=74918
It has been fixed in MySQL Connector/Net 6.7.7 / 6.8.5 / 6.9.6 releases.
Changelog:
With Entity Framework 6, passing in a string reference to the "StartWith"
clause would return incorrect results.
Alternatively, a workaround is to use .Substring(0) which forces Entity not to use LIKE (might affect performance).
return (from a in db.Accounts
where a.accountname.Contains(searchText.Substring(0))

Fluent NHibernate does not create IN part of WHERE clause

I have Fluent NHibernate Linq queries where I check values based on run time arrays. A basic example would be something like:
var array = [1,2,3,4,5,6];
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<MyObject>().Where(x => array.Contains(x.CompareVal)).ToList();
}
I would expect the generated SQL statement to look something like this:
SELECT CompareVal, Column1, Column2
FROM MyObject
WHERE CompareVal IN (1,2,3,4,5,6)
However, what I'm finding instead is that the generated SQL statement simply emits the WHERE clause (proven by watching in Profiler) and selects the entire table, and then seems to run the filter in memory once it gets all the data back.
Something to note - I have a Generic Repository class that all of these calls are funneled through. The Query method is as follows:
public IList<T> Query(Func<T, bool> criteria)
{
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<T>().Where(criteria).ToList();
}
}
Obviously this (lack of a where clause) is not acceptable in a table with a large amount of data. What can I do to force NHibernate to generate the query correctly with the WHERE clause and still keep a generic pattern for repositories?

Does it make a difference if you change your Query method to the following ?
public IList<T> Query(Expression<Func<T, bool>> criteria)
{
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<T>().Where(criteria).ToList();
}
}
This is how I usually proceed with a generic Query :
public List<TOut> GetEntitiesLinq<TIn,TOut>(Expression<Func<IQueryable<TIn>,IQueryable<TOut>>> myFunc)
{
var t = (myFunc.Compile())(_session.Query<TIn>()) ;
return t.ToList();
}
Then how I would use it in your case :
var myObjList = myQueryManager.GetEntitiesLinq<MyObject,MyObject>(x=>x.Where(myObj => array.Contains(myObj.CompareVal)));
Hope this will help

Use Any:
return session.Query<MyObject>().Where(x => array.Any(y => y == x.CompareVal)).ToList();
Your repository pattern (using plain Func) automatically materializes your query to list, if you want something to be deferredly executed, use IQueryable, don't use Func only
Something to note - I have a Generic Repository class that all of
these calls are funneled through. The Query method is as follows:
public IList<T> Query(Func<T, bool> criteria)
{
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<T>().Where(criteria).ToList();
}
}
Your repository just mimic what is already provided out of the box by NHibernate

Can you use QueryOver and WhereRestrictionOn instead?
session.QueryOver<MyObject>().WhereRestrictionOn(o => o.CompareVal).IsIn(array).List();

LINQ translates when not part of a property

I have a query like this:
var q = db.GetTable<Person>().Where(x => x.Employer.CEO != null);
This made up query will return the ID of the CEO of the company a given person works for. This works find and dandy, but if I do something like this, it get the failed to translate to SQL error:
public class Person
{
public bool HasCEO
{
get
{
return this.Employer.CEO != null;
}
}
}
I want to be able to do this and wrap the longer expression within a property so that I don't have to repeat a nested table get:
var q = db.GetTable<Person>().Where(x => x.HasCEO);
How do I create LINQ properties to achieve my desired result?
I'm using C# 4.0 if that matters.

You can't. LINQ to SQL works by examining the expression tree of the query, and does not delve into the implementation of properties to determine whether a row meets your criteria.
What you can do is create a view in SQL server that dynamically calculates this property, and query that instead of the table.