It worked in 2.1 but started failing on 3.1
System.InvalidCastException: 'Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'System.Linq.Expressions.MethodCallExpression'.'
var ordersCounts = await ordersCountsQuery
.Select(x => new TransactionDTO
{
Value = new MoneyDTO(ordersCountsQuery.Sum(a => a.TotalPrice.Amount)) // worked perfectly on 2.1
})
.FirstOrDefaultAsync();
a.TotalPrice.Amount is decimal on Sql server database
In EF Core 2.1 I have set it up
.ConfigureWarnings(warnings => warnings.Throw(RelationalEventId.QueryClientEvaluationWarning))
It looks like you may be hitting this breaking change between 2.2 and 3.0 versions of the entity framework:
https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-3.0/breaking-changes#linq-queries-are-no-longer-evaluated-on-the-client
Old behavior
Before 3.0, when EF Core couldn't convert an expression that was part
of a query to either SQL or a parameter, it automatically evaluated
the expression on the client. By default, client evaluation of
potentially expensive expressions only triggered a warning.
New behavior
Starting with 3.0, EF Core only allows expressions in the top-level
projection (the last Select() call in the query) to be evaluated on
the client. When expressions in any other part of the query can't be
converted to either SQL or a parameter, an exception is thrown.
If this is the case, you'll have to re-write your query to suit (this may have to be two separate queries).
I would note that your query seems to be a bit strangely structured:
var ordersCounts = await ordersCountsQuery
.Select(x => new TransactionDTO
{
Value = new MoneyDTO(ordersCountsQuery.Sum(a => a.TotalPrice.Amount)) // worked perfectly on 2.1
})
.FirstOrDefaultAsync();
Get a list of MoneyDTO (one for every item in the query), each one containing the sum of the total list and select the first.
The following would do the same and likely be slightly more performant (only returning a single value:
var totalPrice = ordersCountsQuery.Sum(a => a.TotalPrice.Amount);
var orderCounts = new TransactionDTO
{
Value = new MoneyDTO(totalPrice)
};
We are using Entity Framework Core 3 with SqlServer Database. Business program needs to create many columns which are not in the Database, due to storage, high querying cost etc. Currently, the team is Copying the whole Database Layer, and Creating whole another layer adding computed members in new entities. Currently taking database layer and applying AutoMapper to new layer. For some reason, this does not seem like optimal method.
In this example, we require computed members called
FullName => FirstName + LastName
AccountValue => Quantity * StockPrice
Entity Framework 3 does not allow Client Side Evaluation anymore, https://devblogs.microsoft.com/dotnet/announcing-ef-core-3-0-and-ef-6-3-general-availability/
so curious what is standardized way for computed members in Entity Framework Core 3?
Reading this article, curious wondering what is up to date, and can be used?
Or does Entity Framework Core 3 Offer New Syntax?
https://daveaglick.com/posts/computed-properties-and-entity-framework
1) We could Materialize the entities. Seems okay, however this forces developer to remember utilize ToList(), had issues where developers forget, causing long db scanning queries, or clientside evaluation caused error.
var result = ctx.Customers
.ToList()
.Select(c => new
{
FullName = c.FullName,
AccountValue = c.AccountValue
});
2) Create Queryable Extension. This only extracts the computed columns, or forces developers to create computed members all in one class (breaks SRP single responsibility idea). Unless there is an alternative modification which address this. This also brings composition chain issues, and possible performance problems like option 1.
public static IQueryable<CustomerData> SelectCustomerData(this IQueryable<Customer> customers) { return customers.Select(c => new CustomerData {
FullName = c.FirstName + " " + c.LastName,
AccountValue = c.Holdings.Sum(h => h.Quantity * h.Stock.Price) }); }
3) Expression Projection, does not allow assignment in Select without Linq Expression Project. Company does not allow this third party tool, unless built from Microsoft.
public readonly Expression<Func<Customer, decimal>> AccountValueExpression = c => c.Holdings.Sum(h => h.Quantity * h.Stock.Price);
Or does Entity Framework Core 3 offer newer syntax?
Solution needs to be where, (a) person can extract some or All of the existing members of original DBEntity, (b) and some or all of New Members,
Example, need FirstName (Existing) and AccountValue (New Member)
Or FullName, FirstName, LastName, StockPrice,
Or Everything, FirstName, LastName, FullName ,Quantity, StockPrice, AccountValue, etc, etc
Any mix or match from entities.
Actually migrating from 2.2 to Core 3, however 2.2 has ClientSide Evaluation Disabled. Cannot utilize third party tools, like Linq.Translations, or DelegateCompiler unless they are created from Microsoft vendor .
Prefer not to use SqlServer Computed columns, as we are relying on DBA team. Additionally there are more intricate calculations.
Client side evaluation is evil, thus developers of EF Core 3 has made a good decision to forbid it. The code which could evaluates on client often leads to annoying performance issues. So I wouldn't recommend you to use computed properties in EF Core 2.* as well.
what is standardized way for computed members in Entity Framework Core
If you want to do a computation, sorting, modification, etc. as a part of your query, you should project your entity into DTO at first. In such a case, the query will be compiled into SQL query (and not evaluated on client).
For this task you can use AutoMapper library. It automatically maps properties with the same name. Other properties (computed properties) can be mapped using custom expression.
var config = new MapperConfiguration(cfg =>
{
cfg.CreateMap<Customer, CustomerDto>()
.ForMember(x => x.FullName, x => x.MapFrom(z => z.FirstName + " " + z.LastName))
.ForMember(x => x.AccountValue, x => x.MapFrom(z => z.Quantity * z.StockPrice));
});
var mapper = config.CreateMapper();
Then, you can use ProjectTo extension method. ProjectTo internally call Select so it doesn't materialize entity. Hence, Where statement is parsed into SQL query.
var customers = await context.Customers
.ProjectTo<CustomerDto>(mapper.ConfigurationProvider)
.Where(x => x.FullName == "full name" && x.AccountValue > 4)
.ToListAsync();
Projection of entities is often a good practice. It allows you to select just a few columns from DB and offers you other stuff that is not possible when you are returning just plain entities (e.g. sorting):
var config = new MapperConfiguration(cfg =>
{
cfg.CreateMap<Invoice, InvoiceDto>();
cfg.CreateMap<Customer, CustomerDto>()
.ForMember(x => x.Invoices, x => x.MapFrom(z => z.Invoices.OrderBy(x => x.Date)));
});
var mapper = config.CreateMapper();
// Customers with invoices sorted by date
var customers = await context.Customers
.ProjectTo<CustomerDto>(mapper.ConfigurationProvider)
.ToListAsync();
AutoMapper can be also used with DI. However, it is 3rd party library. If your company doesn't permit it, you can create your own mapping layer by hand. Which includes a lot of monkey work ..
Assuming it's supported by your backend, you can mirror your computed properties with computed database columns.
public string FullName => FirstName + LastName;
entity.Property(e => e.FullName).HasComputedColumnSql("FirstName + LastName");
Then you can trivially filter, order by, project etc those properties.
I haven't tried this, but it is just a thought - How about creating a custom Extension method of DbFunctions?
I have used EF.Functions.Like method which is an implementation of the SQL LIKE operation. On relational databases this is usually directly translated to SQL.
Check these links -
DbFunctions Class (Microsoft.EntityFrameworkCore)
src/EFCore/DbFunctionsExtensions.cs
Stack Overflow - Entity framework EF.Functions.Like vs string.Contains
First of all, let me quote the EF 3 breaking changes notice here:
Starting with 3.0, EF Core only allows expressions in the top-level projection (the last Select() call in the query) to be evaluated on the client. When expressions in any other part of the query can't be converted to either SQL or a parameter, an exception is thrown.
I have just successfully tested the following query:
var list = context.Customers.Include(c => c.Stocks).Select(c => new
{
FullName = c.FirstName + " " + c.LastName,
TotalInvestment = c.Stocks.Sum(s => s.Price*s.Quantity)
});
list.ToList();
/*
SELECT ([c].[FirstName] + N' ') + [c].[LastName] AS [FullName], (
SELECT SUM([s].[Price] * [s].[Quantity])
FROM [Stocks] AS [s]
WHERE [c].[Id] = [s].[CustomerId]) AS [TotalInvestment]
FROM [Customers] AS [c]
*/
But let's explore the topic a bit further and say you want to query your table on a computed field without bringing all evaluation to client side.
var list = context.Customers.Include(c => c.Stocks)
.Where(c => string.Concat(string.Concat(c.FirstName, " "), c.LastName) == "John Doe") // notice how we have to do string.Concat twice. observe what hppens if you use the next line instead
//.Where(c => string.Concat(c.FirstName, " ", c.LastName) == "John Doe"); // this query will fail to evaluate on SQL and will throw an error, because EF's default concat translator implementation only caters for two parameters
;
list.ToList();
/* the following SQL has been generated
SELECT [c].[Id], [c].[FirstName], [c].[LastName], [s].[Id], [s].[CustomerId], [s].[Price], [s].[Quantity]
FROM [Customers] AS [c]
LEFT JOIN [Stocks] AS [s] ON [c].[Id] = [s].[CustomerId]
WHERE (([c].[FirstName] + N' ') + [c].[LastName]) = N'John Doe'
ORDER BY [c].[Id], [s].[Id]
*/
Apparently, EF3 comes with a few pre-built functions that allow you to do that: see IMethodCallTranslator implementations (such as StringMethodTranslator for example) to get a bit more insight on how it's doing that (and other translators available to you out of the box).
Okay, what if your translator isn't implemented, I hear you ask. This is where things get a bit more exciting. I have successfully done this for EF 2.2 as outlined in this SO answer (which I invite you to check out). The code unfortunately doesn't directly translate to EF3 1:1 but I'm fairly confident the same approach will work.
UPD: see my github repo for PoC of custom DB functions working with EF Core 3.1
I have a view in SQL lets call it MyCustomView.
If I was to write a simple SQL query to count and sum I could do something like: SELECT COUNT(*), SUM(ISNULL(ValueA, ValueB)) FROM MyCustomView
Is it possible to translate that query in EF Core? Diggin around I found the answers mentioning the user of GroupBy 1 (however this doesn't seem to work for views), i.e.
context
.Query<MyCustomView>()
.GroupBy(p => 1)
.Select(grp => new { count = grp.Count(), total = Sum(p=>p.ValueA ?? p.ValueB)}
The issue I am having is that whenever I attempt to run the query I get a complaint about having to run the group by on the client. However If I was to replace the .Query<MyCustomView>() with a DbSet property from the context then that query works fine. So I am guessing it has to do with the fact that I am trying to execute the operation on a View. Is there a way to achieve this behaviour with a View or am I out of luck again with EF Core :(
Querying Views are notoriously slow when they are not indexed. Instead you can convert your View results into a list first, then query that list. It will eliminate the querying the view on the SQL side and should speed up the overall process.
context
.Query<MyCustomView>()
.ToList()
.GroupBy(p => 1)
.Select(grp => new { count = grp.Count(), total = Sum(p=>p.ValueA ?? p.ValueB)}
I will say, the proper solution (if you can do it) is to index the view.
For anyone that is curious (or until someone else manages to provide an anwser) I managed to get it work by creating a linq query like this:
const a = 1;
context
.Query<MyCustomView>()
// For some reason adding the below select lets it execute
.Select(p => new { p.ValueA, p.ValueB })
.GroupBy(p => a)
.Select(grp => new { count = grp.Count(), total = Sum(p=>p.ValueA ?? p.ValueB)})
.First();
Also according the EF Core team this has been sorted in EF Core 3+, unfortunately I haven't got the luxury to upgrade to 3.
I am trying to filter a list by a search string. It says in the doc on the blue note that:
IQueryable gives you the database provider implementation of Contains.
IEnumerable gives you the .NET Framework implementation of Contains
The default setting of SQL Server instances is case-insensitive.
Using ToUpper to make an explicit case-insensitive call should be avoided because it has a performance penalty.
My filtering is as follows:
IQueryable<ApplicationUser> customers =
from u in _context.Users
where (u.Customer != null && u.IsActive)
select u;
if (!string.IsNullOrEmpty(searchString))
{
customers = customers.Where(s => s.Email.Contains(searchString));
}
This solution however is case-sensitive, and I don't really understand why: since I'm using IQueryable, it should use the database provider implementation, that is case-insensitive by default, right?
I'm using EF Core 2 and currently just running a local MSSQLLocalDB.
starting from version 2.1 of the EF Core, you can use HasConversion(). But the information in the database will be stored in lowercase:
builder.Property(it => it.Email).HasConversion(v => v.ToLowerInvariant(), v => v);
I solved a similar problem. This change solved all my problems.
You would be better off using LIKE operator, e.g.
if (!String.IsNullOrEmpty(searchString))
{
customers = customers.Where(x => EF.Functions.Like(x.Email, $"%{searchString}%"));
}
StringComparison is answer for me.
customers = customers.Where(s => s.Email.Contains(searchString, StringComparison.CurrentCultureIgnoreCase));
OR
customers = customers.Where(s => s.Email.Contains(searchString, StringComparison.InvariantCultureIgnoreCase));
works for me.
I'am using wrapper to get some data from table User
IQueryable<StarGuestWrapper> WhereQuery =
session.Linq<User>().Where(u => u.HomeClub.Id == clubId && u.IsActive).Select(
u =>
new StarGuestWrapper()
{
FullName = u.Name + " " + u.LastName,
LoginTime = u.SomeDateTime,
MonthsAsMember = u.SomeIntergerValue,
StarRating = u.SomeOtherInteregValue,
UserPicture = u.Photo.PhotoData,
InstructorFullName = u.SomeInstructorName,
TalkInteractionDuringSession = u.SomeBoolValue,
GoalInteractionDuringSession = u.SomeOtherBoolValue
});
I use this without a problem as a IQueryable so I can do useful things before actually running the query. Like :
WhereQuery.Skip(startRowIndex).Take(maximumRows).ToList();
and so on.
The problem occurs using 'where' statement on query.
For example:
WhereQuery.Where(s => s.StarRating == 1)
will throw an exception in runtime that 'StarRating' doesn't exist in User table - of course it doesn't it's a wrappers property. It will work if I materialize query by
WhereQuery.AsEnumerable().Where(s => s.StarRating == 1)
but then it loses all the sens of using IQueryable and I don't want to do this.
What is strange and interesting that not all properties from wrapper throw error, all the bool values can be used in where statement. Example :
WhereQuery.Where(s => s.TalkInteractionDuringSession)
It works in EntityFramework , why do I get this error in NHibernate and how to get it working the way I want it to ?
Keep in mind the older nHibernate Linq provider is only a partial implementation and is no longer being actively worked on. A new and more complete linq provider is being developed now and will be part of NH3.0 (you can checkout the trunk and build it to see if it addresses this problem).
My recommendation is to change your code to call ToList() at the point when you explicitly wish to hit the database. You are passing a future valued query back from your repository at which point anything could technically happen to the query. Even EF and LINQ2SQL cannot translate any possible linq query into SQL.
I do realize this isn't what you want to be able to do, but I think you are trying to bend the framework to do something in a way this isn't very natural at all.