We Keep Coding

Using Function logic in LINQ Query net core 3

I have the following enum:
public enum WorkType
{
Type1,
Type2,
Type3,
Type4,
Type5,
Type6
}
and a class
public class Work {
public WorkType Type {get; set;}
....
}
and an extension method:
public static partial class WorkTypeExtensions
{
public static bool IsHighValueWork(this WorkType value)
{
switch (value)
{
case WorkType.Type1:
case WorkType.Type2:
return true;
default:
return false;
}
}
}
and SQL Linq query
public List<Work> GetHighValueWork()
{
var query = Context.Work.Where( w => w.IsHighValueWork());
return query.ToList();
}
This is a simplified version of my problem. This query used to work, but it is not working any more after the code was converted from net core 2.1 to 3.1. The error msg is
The query could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(). I don't want to change it to
public List<Work> GetHighValueWork()
{
var query = Context.Work.Where( w => w.Type == WorkType.Type1 || w.Type == WorkType.Type2);
return query.ToList();
}
Because actual function is very complex. I searched it seems LINQ Expression Func can be used, but I haven't figured that yet. What is the best way to do this?

IsHighValueWork is just a simple C# method. There is no way to convert that function to SQL by EF.
It is really explained well in that link, why it was working in .net core 2.1. It seems that, in previous versions when EF Core couldn't convert an expression that was part of a query to either SQL or a parameter, it automatically evaluated the expression on the client.
And it is really bad. Because, as noted:
For example, a condition in a Where() call which can't be translated can cause all rows from the table to be transferred from
the database server, and the filter to be applied on the client.
So, it seems previously you were just loading all data to the client and then applying filter on the client side.
So, the problem with your code is, that Func cant be translated into Sql.
Either fetch all data into app explicitly and filter then or use second version of you code.
Context.Work.ToList()
.Where( w => w.Type.IsHighValueWork());
But, I don't recommend to use that version. It is better to use second version like so:
Func<Work, bool> IsHighValueWork = (work) =>
work.Type == WorkType.Type1 || work.Type == WorkType.Type2;
And then:
var query = Context.Work.Where(IsHighValueWork);

It is possible to query a NotMapped property?

i'm working with EF6 code first, and i used this answer to map a List<stirng> in my entitie.
This is my class
[Key]
public string SubRubro { get; set; }
[Column]
private string SubrubrosAbarcados
{
get
{
return ListaEspecifica == null || !ListaEspecifica.Any() ? null : JsonConvert.SerializeObject(ListaEspecifica);
}
set
{
if (string.IsNullOrWhiteSpace(value))
ListaEspecifica.Clear();
else
ListaEspecifica = JsonConvert.DeserializeObject<List<string>>(value);
}
}
[NotMapped]
public List<string> ListaEspecifica { get; set; } = new List<string>();
It works perfectly to storage my list as Json, but now i need to perform a linq query, and i'm trying this
var c = db.CategoriaAccesorios.Where(c => c.ListaEspecifica.Contains("Buc")).First();
And it's throwing
System.NotSupportedException: The specified type member
'ListaEspecifica' is not supported in LINQ to Entities. Only
initializers, entity members, and entity navigation properties are
supported.
what is logical.
Is there any way to perform a query like this?

The problem here is that LINQ to Entities does not understand how to convert your query to the back-end (SQL) language. Because you're not materializing (i.e. converting to .NET) the results of the query until you filter it, LINQ tries to convert your query to SQL itself. Since it's not sure how to do that, you get a NotSupportedException.
If you materialize the query first (I.e. call a .ToList()) then filter, things will work fine. I suspect this isn't what you want, though. (I.e. db.CategoriaAccesorios.ToList().Where(c => c.ListaEspecifica.Contains("Buc")).First();)
As this answer explains, your issue is the EF to SQL Conversion. Obviously you want some way to workaround it, though.
Because you are JSON serializing, there are actually a couple options here, most particularly using a LIKE:
var c =
(from category
in db.CategoriaAccessorios
where SqlMethods.Like(c.SubrubrosAbarcados, "%\"Buc\"%")
select category).First()
If EF Core, allegedly Microsoft.EntityFrameworkCore.EF.Functions.Like should replace SqlMethods.Like.
If you have SQL Server 2016+, and force the SubrubrosAbarcados to be a JSON type, it should be possible to use a raw query to directly query the JSON column in particular.
If you're curious about said aspect, here's a sample of what it could look like in SQL Server 2016:
CREATE TABLE Test (JsonData NVARCHAR(MAX))
INSERT INTO Test (JsonData) VALUES ('["Test"]'), ('["Something"]')
SELECT * FROM Test CROSS APPLY OPENJSON(JsonData, '$') WITH (Value VARCHAR(100) '$') AS n WHERE n.Value = 'Test'
DROP TABLE Test

I was able to do something like this via CompiledExpression.
using Microsoft.Linq.Translations;
// (...) namespace, class, etc
private static readonly CompiledExpression<MyClass, List<string>> _myExpression = DefaultTranslationOf<MyClass>
.Property(x => x.MyProperty)
.Is(x => new List<string>());
[NotMapped]
public List<string> MyProperty
{
get { return _myExpression.Evaluate(this); }
}
I hope there are better / prettier solutions though ;)

A "Composable" Full Text Search with a Code First Model

UPDATE 18 Sep 2013
It looks like there isn't an easy way to do this. I'm holding out for a solution that involves some extension to Entity Framework.
If you'd like to see these features in Entity Framework, vote for them on the user voice site, perhaps here and here
There are several similar questions on SO but I can't find a question new and similar enough to have the answer I'm looking for.
If this looks like information overload, jump down to In Summary.
Background
I'm writing a WebApi REST service to expose some pre-existing data through an OData end point. I'm using the EntitySetContoller<TEntity, TKey> to do all the grunt work for me. As well as the standard OData parameters, that are routed and translated by the base class, I've added some custom parameters, to allow specific functionality for my controller.
My database server is MS SQL Server with a full text index on the [BigText] NVarChar[4000] column of the [SomeEntity] table.
I have one limitation, I must use a Code First model.
// Model POCO
public class SomeEntity
{
public int Id { get; set; }
public string BigText { get; set; }
}
// Simple Controller
public class SomeEntityController : EntitySetController<SomeEntity, int>
{
private readonly SomeDbContext context = new SomeDbContext();
public override IQueryable<SomeEntity> Get()
{
var parameters = Request.GetQueryNameValuePairs()
.ToDictionary(p => p.Key, p => p.Value);
if (parameters.ContainsKey("BigTextContains")
(
var searchTerms = parameters["BigTextContains"];
// return something special ...
)
return this.context.SomeEntities;
}
// ... The rest is omitted for brevity.
}
The Problem
How to implement the // return something special ... part of my example?
Obviously, the niave
return this.context.SomeEntities.Where(e =>
e.BigText.Contains(searchTerm));
is completely wrong, it composes to a WHERE clause like
[BigText] LIKE '%' + #searchTerm + '%'
This doesn't use Full Text Searching so, doesn't support complex search terms and otherwise, performs terribley.
This approach,
return this.context.SomeEntities.SqlQuery(
"SELECT E.* FROM [dbo].[SomeEntity] E " +
"JOIN CONTAINSTABLE([SomeEntity], [BigText], #searchTerm) FTS " +
" ON FTS.[Key] = E.[Id]",
new object[] { new SqlParameter("#searchTerm", searchTerm) })
.AsQueryable();
Looks promising, it actually uses Full Text Searching, and is quite functional. However, you'll note that DbSqlQuery, the type returned from the SqlQuery function does not implement IQueryable. Here, it is coerced to the right return type with the AsQueryable() extension but, this breaks the "chain of composition". The only statement that will be performed on the server is the one specified in the code above. Any additional clauses, specified on the OData URL will be serviced on the API hosting web server, without benefitting from the indices and specialised set based functionality of the database engine.
In Summary
What is the most expedient way of accessing MS SQL Server's Full Text Search CONTAINSTABLE function with an Entity Framework 5 Code First model and acquiring a "composable" result?
Do I need to write my own IQueryProvider? Can I extend EF in some way?
I don't want to use Lucene.Net, I don't want to use a Database Generated Model. Perhaps I could add extra packages or wait for EF6, would that help?

It is not perfect, but you can accomplish what you are after with 2 calls to the database.
The first call would retrieve a list of matching key's from CONTAINSTABLE and then the second call would be your composable query utilizing the IDs that you returned from the first call.
//Get the Keys from the FTS
var ids = context.Database.SqlQuery<int>(
"Select [KEY] from CONTAINSTABLE([SomeEntity], [BigText], #searchTerm)",
new object[] { new SqlParameter("#searchTerm", searchTerm) });
//Use the IDs as an initial filter on the query
var composablequery = context.SomeEntities.Where(d => ids.Contains(d.Id));
//add on whatever other parameters were captured to the 'composablequery' variable
composablequery = composablequery.Where(.....)

I had this same issue recently:
EF 5 Code First FTS Queriable
Let me extend that post.
Your first option was mine first as well - using SqlQuery
I also needed to do more filtering, so instead of always writing full sql I used QueryBuilder, to which I made some changes and added more functions to fit my needs(I could upload it somewhere if needed):
QueryBuilder
After I have found another idea which I implemented.
Someone already mention it here, and that is to use SqlQuery that will return HashSet of Ids and that you can use it in EF queries with Contains.
This is better but not most optimal since you need 2 queries and Id list in memory.
Example:
public IQueryable<Company> FullTextSearchCompaniesByName(int limit, int offset, string input, Guid accountingBureauId, string orderByColumn)
{
FtsQueryBuilder ftsQueryBuilder = new FtsQueryBuilder();
ftsQueryBuilder.Input = FtsQueryBuilder.FormatQuery(input);
ftsQueryBuilder.TableName = FtsQueryBuilder.GetTableName<Company>();
ftsQueryBuilder.OrderByTable = ftsQueryBuilder.TableName;
ftsQueryBuilder.OrderByColumn = orderByColumn;
ftsQueryBuilder.Columns.Add("CompanyId");
if (accountingBureauId != null && accountingBureauId != Guid.Empty)
ftsQueryBuilder.AddConditionQuery<Guid>(Condition.And, "" , #"dbo.""Company"".""AccountingBureauId""", Operator.Equals, accountingBureauId, "AccountingBureauId", "");
ftsQueryBuilder.AddConditionQuery<bool>(Condition.And, "", #"dbo.""Company"".""Deleted""", Operator.Equals, false, "Deleted", "");
var companiesQuery = ftsQueryBuilder.BuildAndExecuteFtsQuery<Guid>(Context, limit, offset, "Name");
TotalCountQuery = ftsQueryBuilder.Total;
HashSet<Guid> companiesIdSet = new HashSet<Guid>(companiesQuery);
var q = Query().Where(a => companiesIdSet.Contains(a.CompanyId));
return q;
}
However EF 6 now has something called Interceptors that can be used to implement queriable FTS, and it is pretty simple and generic(last post):
EF 6 Interceptors for FTS.
I have tested this and it works fine.
!! REMARK: EF Code First, even with version 6, does not support Custom Stored Procedures.
There are only some for predefined CUD operations if I understood it well:
Code First Insert/Update/Delete Stored Procedure Mapping, so it can't be done with it.
Conclusion: if you can use EF 6 go for third options, is gives all you need.
If you are stucked with EF 5 or less, second option is better then first but not most optimal.

EF4 - Linq to SQLite and List.Contains strange behaviour

I have some issue querying my local sqlite-database. Below is the current code:
public static Dictionary<Guid, string> GetHashedComanyMembers(List<Guid> IdsToSearch)
{
using (var dbContext = new DbEntities(GetConnectionString()))
{
var test = dbContext.<myTable>.Where(cm => IdsToSearch.Contains(cm.IdToFind));
}
}
The variable test contains an emtpt dataset. And now to my question.
If is set a variable
var firstGuid = IdsToSearch[0]
and i change the next line to
var test = dbContext.<myTable>.Where(cm => cm.IdToFind == firstGuid);
test contains the two sets i want to have.
If i let me show the (first) generated statement and run it on the database directly it also fails to find something. But the (second) statement succeeds.
So does anyone have any idea where i'm going wrong or could point out the error?
Thanks in advance, Steve
The first version:
SELECT
[Extent1].[ValueId] AS [ValueId],
[Extent1].[ValueFirstName] AS [ValueFirstName],
[Extent1].[ValueLastName] AS [ValueLastName],
[Extent1].[ValueLastChanged] AS [ValueLastChanged],
[Extent1].[ValueLastChangedBy] AS [ValueLastChangedBy]
FROM [tblValue] AS [Extent1]
WHERE '28826de0-27ea-42ab-9104-234ee289de16' = [Extent1].[ValueId]
Second:
SELECT
[Extent1].[ValueId] AS [ValueId],
[Extent1].[ValueFirstName] AS [ValueFirstName],
[Extent1].[ValueLastName] AS [ValueLastName],
[Extent1].[ValueLastChanged] AS [ValueLastChanged],
[Extent1].[ValueLastChangedBy] AS [ValueLastChangedBy]
FROM [tblValue] AS [Extent1]
WHERE [Extent1].[ValueId] = #p__linq__0
Edit
If i call th ToList() method on my table before the Where() everything is fine because the whole table-content will be enumerated an i call the Contains on this list. But that can't be the solution to Select the whole table first.
So i think the problem is the conversation done by Linq because the generated statements checks agains the Guid as string but it is internally stored as Blob.
In the second statement the guid is not set directly. There is an placeholder that will be replaced by DBMS? And this query is alright.
So any ideas where to start?

First SQL to Linq Conversion

I am new to LINQ, can somebody guide me that how can I convert the following SQL Query in Linq.
Select tblIns.InsID, tblProg.ProgName
from tblIns, tblProg
Where tblIns.InsID = tblProg.InsID
I am working on MVC2 Project, I have dataContext and Reposetories , please find below the code where I need this query:
public IQueryable<tblInstitute> InsRepeater()
{
return from Inst in _db.tblInstitutes
from Progs in _db.tblPrograms
Where Inst.InstituteID = Progs.InstituteID
Select Inst.InstituteID, Progs.ProgramName
}

The first thing you need is a data context which emulates your database. Here is an example of how to do this with linq to sql. You can also do it with entity framework (EF) or any other provider.
Once you have the tables created, the query then translates pretty straight forward:
var results = from insEntity in tablIns
from progEntity in tablProg
where insEntity.InsID equals progEntity.InsID
select new { insEntity.InsID, progEntity.ProgName };
With the question you have asked, this is as much as I think will be useful. In the future it's best to write questions explaining what you are trying to do, what you have tried, and then where you are stuck. The question should be specific enough to get you just over the next hump.
Per your edit: The query you have needs to have lowercase where and select and it needs to end the statement with a semi-colon (assuming it is c#). Then you select statement needs to select a new object. The results would look something like this:
public IQueryable<tblInstitute> InsRepeater()
{
return from Inst in _db.tblInstitutes
from Progs in _db.tblPrograms
where Inst.InstituteID equals Progs.InstituteID
select Inst; // for the current method header
//select new { Inst.InstituteID, Progs.ProgramName }; // to use this one you'll have to create a new type with the properties you want to return
}

First you need to
1. Create entity classes.
2. The data context.
3. Define relationships
4. Query
Reference
from I in tblIns
from P in tblProg
Where I.InsID = P.InsID
Select I.InsID, P.ProgName

Assuming you have your foreign keys set up properly in the database you can just do this, there is no need to to the joins yourself:
from x in db.tblProgs
select x.tblIns.id, x.Progname