We Keep Coding

Entity Framework Core, emitting/generating expression in select

I'm using Entity Framework Core in .Net Core 2.2, with the recently released Oracle.EntityFrameworkCore library.
I'd like to be able to generate a query like this...
select nvl(nullablecolumn, 'N') from table;
I think I'm right in saying that I can't do this, at least not out of the box... I can however do something similar, using something like this (but then if I end up writing this, why not write actual SQL and skip Entity Framework???)...
from row in table
select new { somedata = row.nullablecolumn ?? "N" };
The above linq query gets me the same sort of answer as I'm after... question is, can I do some expression tree magic to get the same result?
For example, this question looks like it generates an expression tree for a "like" query, so how would I generate an expression tree (or modify the existing expression tree) to make the select side of the statement emit nvl()?
This would be useful where you have Entity Framework Value Conversions...
Bonus points (if I could give bonus points) if you can give me a clue on how to create an expression tree to manipulate the where side of the query... AutoMapper manages this somehow with "Projections"?
Any thoughts/pointers would be greatly appreciated...

To translate your own method invocation to proper SQL function you can use HasDbFunction from docs.
You have to define own static method as
public static class OracleDbFunction
{
public static string Nvl(string string1, string replace_with) => throw new NotImplementedException(); // You can provide matching in-memory implementation for client-side evaluation of query if needed
}
register it in your DbContext
protected overridevoid OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasDbFunction(typeof(OracleDbFunction).GetMethod(nameof(OracleDbFunctionExtensions.Nvl, builder =>
{
builder.HasName("nvl");
});
}
and use it in your Where expression as
.Where(i => OracleDbFunction.Nvl(i.nullablecolumn, "N") == "N")
You can also use attribute DbFunctionAttribute on OracleDbFunctionExtensions.Nvl to avoid registering it in OnModelCreating

Can't add calculated value to IQueryable

I'm running an EF statement where I need to calculate de deductibles. After long trying, I can't seem to add a custom function in a .Select() statement. Instead I'm trying to add the values after my .Select() statement.
The problem here is, in my CalculateDeductibles() I can't seem to add any values to item.Deductibles.
The GetDeductibles(item.RequestId) is a rather heavy funtion that does several extra queries, so I'm trying to prevent to convert my IQueryable to an IList object.
So there are actually 2 questions:
Can I have the GetDeductibles() function directly in my .Select() statement?
Can I somehow (with keeping an eye on performance) add the value after I did my .Select()
Code:
public IQueryable<ReinsuranceSlip> GetReinsuranceSlipsOverview(int userId, int companyId, string owner, string ownerCompany)
{
IQueryable<ReinsuranceSlip> model = null;
model = _context.Request
.Where(w => w.RequestGroup.ProgramData.MCContactId == userId)
.Select(x => new ReinsuranceSlip()
{
Id = x.Id,
RequestId = x.Id,
LocalPolicyNumber = x.LocalPolicyNumber,
BusinessLine = x.RequestGroup.ProgramData.BusinessLine.DisplayName,
BusinessLineId = x.RequestGroup.ProgramData.BusinessLine.Id,
ParentBroker = x.RequestGroup.ProgramData.Broker.Name,
LocalBroker = x.Broker.Name,
InceptionDate = x.InceptionDate,
RenewDate = x.RenewDate,
//Deductibles = CalculateDeductibles(x)
});
CalculateDeductibles(model);
return model;
}
private void CalculateDeductibles(IQueryable<ReinsuranceSlip> model)
{
//model.ForEach(m => m.Deductibles = GetDeductibles(m.RequestId));
foreach (var item in model)
{
item.Deductibles = GetDeductibles(item.RequestId);
}
}

Updated and Sorry for the first version of this answer. I didn't quite understand.
Answer 1: IQueryable is using to creating a complete SQL statement to call in SQL Server. So If you want to use IQueryable, your methods need to generate statements and return it. Your GetDetuctibles method get request Id argument but your queryable model object didn't collect any data from DB yet, and it didn't know x.Id value. Even more, your GetCarearDetuctiples get an argument so and with that argument generates a queryable object and after some calculations, it returns decimal. I mean yes you can use your methods in select statement but it's really complicated. You can use AsExpendable() LINQ method and re-write your methods return type Expression or Iqueryable.
For detailed info you should check. This:
Entity Navigation Property IQueryable cannot be translated into a store expression and this: http://www.albahari.com/nutshell/predicatebuilder.aspx
And you also should check this article to understand IQueryable interface: https://samueleresca.net/2015/03/the-difference-between-iqueryable-and-ienumerable/
Answer 2: You can use the IEnumerable interface instead IQueryable interface to achieve this. It will be easy to use in this case. You can make performance tests and improve your methods by time.
But if I were you, I'd consider using Stored Procedures for performance gain.

You'll have to understand the differences between an IEnumerable and an IQueryable.
An IEnumerable object holds everything to enumerate over the elements in the sequence that this object represents. You can ask for the first element, and once you've got it, you can repeatedly ask for the next element until there is no more next element.
An IQueryable works differently. An IQueryable holds an Expression and a Provider. The Expression is a generic description of what data should be selected. The Provider knows who has to execute the query (usually a database), and it knows how to translate the Expression into a format that the Provider understands.
There are two types of LINQ functions: the ones that return IQueryable<TResult> and the ones that return TResult. Functions form the first type do not execute the query, they will only change the expression. They use deferred execution. Functions of the second group will execute the query.
When the query must be executed, the Provider takes the Expression and tries to translate it into the format that the process that executes the query understand. If this process is a relational database management system this will usually be SQL.
This translation is the reason that you can't add your own functionality: the Expression must be translatable to SQL, and the only thing that your functions may do is call functions that will change the Expression to something that can be translated into SQL.
In fact, even entity framework does not support all LINQ functionalities. There is a list of Supported and Unsupported LINQ methods
Back to your questions
Can I have GetDeductibles directly in my query?
No you can't, unless you can make it thus simple that it will only change the Expression using only supporte LINQ methods. You'll have to write this in the format of an extension function. See extension methods demystified
Your GetDeductibles should have an IQueryable<TSource> as input, and return an IQueryable<TResult> as output:
static class QueryableExtensions
{
public static IQueryable<TResult> ToDeductibles<TSource, TResult, ...>(
this IQueryable<TSource> source,
... other input parameters, keySelectors, resultSelectors, etc)
{
IQueryable<TResult> result = source... // use only supported LINQ methods
return result;
}
}
If you really need to call other local functions, consider calling AsEnumerable just before calling the local functions. The advantage above ToList is that smart IQueryable providers, like the one in Entity Framework will not fetch all items but the items per page. So if you only need a few ones, you won't have transported all data to your local process. Make sure you throw away all data you don't need anymore before calling AsEnumerable, thus limiting the amount of transported data.
Can I somehow add the value after I did my .Select()
LINQ is meant to query data, not to change it. Before you can change the data you'll have to materialize it before changing it. In case of a database query, this means that you have a copy of the archived data, not the original. So if you make changes, you'll change the copies, not the originals.
When using entity framework, you'll have to fetch every item that you want to update / remove. Make sure you do not select values, but select the original items.
NOT:
var schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.Select(school = new
{
... // you get a copy of the values: fast, but not suitable for updates
})
.FirstOrDefault();
BUT:
School schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.FirstOrDefault()
Now your DbContext has the original School in its ChangeTracker. If you change the SchoolToUpdate, and call SaveChanges, your SchoolToUpdate is compared with the original School, to check if the School must be updated.
If you want, you can bypass this mechanism, by Attaching a new School directly to the ChangeTracker, or call a Stored procedure.

LINQ to Entities Where clause with custom method

I'm using Entity Framework Core (2.0) and I have the following doubt.
I'm not sure about what happens when I do this:
context.Customers.Where(c => MyCustomMethod(c));
bool MyCustomMethod(Customer c)
{
return c.Name.StartsWith("Something");
}
Does it translate to SQL without problems?
Is it different than writing:
context.Customers.Where(c => c.StartsWith("Something"));
In short, will I be able to wrap my validations for the Where clase inside a method? Does it break the translation to SQL?

No, you cannot call your custom method in EF LINQ query because EF will not be able to generate expression tree of the method and so it cannot translate it to SQL.
For more info about expression trees, refer to the link.

if you need to get string from method you can write same query like this
from customer in contetx.Customer
let str = GetString()
where Name.Any(c=> c.StartsWith(str) )
select customer;
string GetString()
{
return "Something";
}
i dnt know this is helpfull, but this can achieve

Can I use predicates from my Domain Model in Entity Framework Code First?

I've got a class with a bool method, e.g.:
public bool IsInFuture()
{ return this.Date > DateTime.Now; }
And I store it in a database using EF Code First. If I'll try to use that predicate in Linq operations, I'll get an exception, as it can not be translated into SQL:
await context.Where(order => order.IsInFuture()).ToListAsync();
That predicate's logic can be rather complicated and I wouldn't like to duplicate it in my code. It there any way I can "inline" it's code into Linq operations? I'm pretty sure, this problem has a solution.
Thanks in advance!

You can't use the predicate directly. How should EF know how to convert it into an SQL WHERE statement?
You have a few alternatives:
Rephrase the query to use standard operations, e.g. smaller-than. This will duplicate the logic from the predicate, though.
Use a raw SQL query. This will also duplicate the logic, but in SQL.
Load the entire list of entities and then filter in-memory (i.e. Where() after ToList()). Like this, you can reuse the predicate. Note that this only a good idea for very small data sets.

Fluent syntax for IQueryable extension method?

I'll let the code speak for itself:
public interface ISoftDeletable {
bool IsDeleted {get; set;}
}
public static class Extensions {
public IQueryable<T> Active<T>(this IQueryable<T> q) where T : ISoftDeletable {
return q.Where(t => !t.IsDeleted);
}
}
public partial class Thing : ISoftDeletable {
...
}
...
var query = from tc in db.ThingContainers
where tc.Things.Active().Any(t => t.SatisfiesOtherCondition)
select new { ... }; // throws System.NotSupportedException
The error is:
LINQ to Entities does not recognize the method 'System.Collections.Generic.IQueryable`1[Thing] ActiveThing' method, and this method cannot be translated into a store expression.
You get the idea: I'd like a fluent kind of way of expressing this so that for any ISoftDeletable I can add on a 'where' clause with a simple, reusable piece of code. The example here doesn't work because Linq2Entities doesn't know what to do with my Active() method.
The example I gave here is a simple one, but in my real code, the Active() extension contains a much more intricate set of conditions, and I don't want to be copying and pasting that all over my code.
Any suggestions?

You have two unrelated problems in the code. The first one is that Entity Framework cannot handle casts in expressions, which your extension method does.
The solution to this problem is to add a class restriction to your extension method. If you do not add that restriction, the expression is compiled to include a cast:
.Where (t => !((ISoftDeletable)t.IsDeleted))
The cast above confuses Entity Framework, so that is why you get a runtime error.
When the restriction is added, the expression becomes a simple property access:
.Where (t => !(t.IsDeleted))
This expression can be parsed just fine with entity framework.
The second problem is that you cannot apply user-defined extension methods in query syntax, but you can use them in the Fluent syntax:
db.ThingContainers.SelectMany(tc => tc.Things).Active()
.Any(t => t.SatisfiesOtherCondition); // this works
To see the problem we have to look at what the actual generated query will be:
db.ThingContainers
.Where(tc => tc.Things.Active().Any(t => t.StatisfiesOtherCondition))
.Select(tc => new { ... });
The Active() call is never executed, but is generated as an expression for EF to parse. Sure enough, EF does not know what to do with such a function, so it bails out.
An obvious workaround (although not always possible) is to start the query at the Things instead of the ThingContainers:
db.Things.Active().SelectMany(t => t.Container);
Another possible workaround is to use Model Defined Functions, but that is a more involved process. See this, this and this MSDN articles for more information.

While #felipe has earned the answer credit, I thought I would also post my own answer as an alternative, similar though it is:
var query = from tc in db.ThingContainers.Active() // ThingContainer is also ISoftDeletable!
join t in db.Things.Active() on tc.ID equals t.ThingContainerID into things
where things.Any(t => t.SatisfiesOtherCondition)
select new { ... };
This has the advantage of keeping the structure of the query more or less the same, though you do lose the fluency of the implicit relationship between ThingContainer and Thing. In my case, the trade of works out that it's better to specify the relationship explicitly rather than having to specify the Active() criteria explicitly.