I'm trying to do somethig like that:
string queryText = searchtext.ToUpper().RemoveDiacritics();
var result = from p in People
where p.Name.ToUpper().RemoveDiacritics().Contains(queryText)
select p;
And I get this error: LINQ to Entities does not recognize the method RemoveDiacritics.
I need to apply the method RemoveDiacritics to the field Name from database.
Somebody knows how I can do that.
Thanks,
regards!
You have to be aware of the difference between IEnumerable and IQueryable
The advantage of IQueryable is that your query will be executed by the prodiver of the IQueryable, which is usually a database. Transferring your data from the database to local memory is a relatively slow process, so it is best to limit the data that must be transferred by letting the database do as much as possible.
The disadvantage of IQueryable is that it only knows the funcitonality of the provider. In case of SQL check the list of
Supported and Unsupported LINQ Methods (LINQ to Entities)
This means that you can't use your own functions like RemoveDiacritics.
The solution of your problem depends on how often you have to do this query, and whether this query must be performed very fast
If the query isn't done very often and it doesn't have to be at light speed, it is enough to add AsEnumerable()
var result = People
.AsEnumerable()
.Where(p => p.Name.ToUpper().RemoveDiacritics()....);
If your query is done very often and has to be really fast, consider extending your People table with a value that contains the value of Name.ToUpper().RemoveDiacritics()
class People
{
public string Name {get; set;}
public string NameWithoutDiacritics {get; set;}
...
}
Whenever your model Adds or Updates a People object, make sure NameWithoutDiacritics contains the correct value
public void Update(People people)
{
using (var dbContext = new MyDbContext())
{
var foundPeople = people.Find(people.Id);
foundPeople.Name = people.Name;
foundPeople.NameWithoutDiacritics = people.Name
.ToUpper()
.RemoveDiacritics();
...
dbContext.SaveChanges();
}
}
Change your query:
var result = People.Where(p => p.NameWithoutDiacritics.Contains(queryText));
It is impossible because EF does NOT know how to translate 'RemoveDiacritics' method to SQL. So If You need to do this logic from database then maybe You should create stored procedure, otherwise You need to get all records to memory and then apply this filtering.
Related
I'm running an EF statement where I need to calculate de deductibles. After long trying, I can't seem to add a custom function in a .Select() statement. Instead I'm trying to add the values after my .Select() statement.
The problem here is, in my CalculateDeductibles() I can't seem to add any values to item.Deductibles.
The GetDeductibles(item.RequestId) is a rather heavy funtion that does several extra queries, so I'm trying to prevent to convert my IQueryable to an IList object.
So there are actually 2 questions:
Can I have the GetDeductibles() function directly in my .Select() statement?
Can I somehow (with keeping an eye on performance) add the value after I did my .Select()
Code:
public IQueryable<ReinsuranceSlip> GetReinsuranceSlipsOverview(int userId, int companyId, string owner, string ownerCompany)
{
IQueryable<ReinsuranceSlip> model = null;
model = _context.Request
.Where(w => w.RequestGroup.ProgramData.MCContactId == userId)
.Select(x => new ReinsuranceSlip()
{
Id = x.Id,
RequestId = x.Id,
LocalPolicyNumber = x.LocalPolicyNumber,
BusinessLine = x.RequestGroup.ProgramData.BusinessLine.DisplayName,
BusinessLineId = x.RequestGroup.ProgramData.BusinessLine.Id,
ParentBroker = x.RequestGroup.ProgramData.Broker.Name,
LocalBroker = x.Broker.Name,
InceptionDate = x.InceptionDate,
RenewDate = x.RenewDate,
//Deductibles = CalculateDeductibles(x)
});
CalculateDeductibles(model);
return model;
}
private void CalculateDeductibles(IQueryable<ReinsuranceSlip> model)
{
//model.ForEach(m => m.Deductibles = GetDeductibles(m.RequestId));
foreach (var item in model)
{
item.Deductibles = GetDeductibles(item.RequestId);
}
}
Updated and Sorry for the first version of this answer. I didn't quite understand.
Answer 1: IQueryable is using to creating a complete SQL statement to call in SQL Server. So If you want to use IQueryable, your methods need to generate statements and return it. Your GetDetuctibles method get request Id argument but your queryable model object didn't collect any data from DB yet, and it didn't know x.Id value. Even more, your GetCarearDetuctiples get an argument so and with that argument generates a queryable object and after some calculations, it returns decimal. I mean yes you can use your methods in select statement but it's really complicated. You can use AsExpendable() LINQ method and re-write your methods return type Expression or Iqueryable.
For detailed info you should check. This:
Entity Navigation Property IQueryable cannot be translated into a store expression and this: http://www.albahari.com/nutshell/predicatebuilder.aspx
And you also should check this article to understand IQueryable interface: https://samueleresca.net/2015/03/the-difference-between-iqueryable-and-ienumerable/
Answer 2: You can use the IEnumerable interface instead IQueryable interface to achieve this. It will be easy to use in this case. You can make performance tests and improve your methods by time.
But if I were you, I'd consider using Stored Procedures for performance gain.
You'll have to understand the differences between an IEnumerable and an IQueryable.
An IEnumerable object holds everything to enumerate over the elements in the sequence that this object represents. You can ask for the first element, and once you've got it, you can repeatedly ask for the next element until there is no more next element.
An IQueryable works differently. An IQueryable holds an Expression and a Provider. The Expression is a generic description of what data should be selected. The Provider knows who has to execute the query (usually a database), and it knows how to translate the Expression into a format that the Provider understands.
There are two types of LINQ functions: the ones that return IQueryable<TResult> and the ones that return TResult. Functions form the first type do not execute the query, they will only change the expression. They use deferred execution. Functions of the second group will execute the query.
When the query must be executed, the Provider takes the Expression and tries to translate it into the format that the process that executes the query understand. If this process is a relational database management system this will usually be SQL.
This translation is the reason that you can't add your own functionality: the Expression must be translatable to SQL, and the only thing that your functions may do is call functions that will change the Expression to something that can be translated into SQL.
In fact, even entity framework does not support all LINQ functionalities. There is a list of Supported and Unsupported LINQ methods
Back to your questions
Can I have GetDeductibles directly in my query?
No you can't, unless you can make it thus simple that it will only change the Expression using only supporte LINQ methods. You'll have to write this in the format of an extension function. See extension methods demystified
Your GetDeductibles should have an IQueryable<TSource> as input, and return an IQueryable<TResult> as output:
static class QueryableExtensions
{
public static IQueryable<TResult> ToDeductibles<TSource, TResult, ...>(
this IQueryable<TSource> source,
... other input parameters, keySelectors, resultSelectors, etc)
{
IQueryable<TResult> result = source... // use only supported LINQ methods
return result;
}
}
If you really need to call other local functions, consider calling AsEnumerable just before calling the local functions. The advantage above ToList is that smart IQueryable providers, like the one in Entity Framework will not fetch all items but the items per page. So if you only need a few ones, you won't have transported all data to your local process. Make sure you throw away all data you don't need anymore before calling AsEnumerable, thus limiting the amount of transported data.
Can I somehow add the value after I did my .Select()
LINQ is meant to query data, not to change it. Before you can change the data you'll have to materialize it before changing it. In case of a database query, this means that you have a copy of the archived data, not the original. So if you make changes, you'll change the copies, not the originals.
When using entity framework, you'll have to fetch every item that you want to update / remove. Make sure you do not select values, but select the original items.
NOT:
var schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.Select(school = new
{
... // you get a copy of the values: fast, but not suitable for updates
})
.FirstOrDefault();
BUT:
School schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.FirstOrDefault()
Now your DbContext has the original School in its ChangeTracker. If you change the SchoolToUpdate, and call SaveChanges, your SchoolToUpdate is compared with the original School, to check if the School must be updated.
If you want, you can bypass this mechanism, by Attaching a new School directly to the ChangeTracker, or call a Stored procedure.
I've been reading up on Projections and while I find them fascinating, I also find they're not very transparant and Microsoft's documentation seems incomplete.
This article on it by Eugene Prystupa is quite excellent, but still leaves me with questions.
For example, is it required to use an Anonymous Type to do the shaping/projecting? Or could one also use custom named Types?
For example, would the following two code samples result in the same generated SQL?
var q = ctx.CustomerAddresses.Select(x =>
new {
CustomerAddress = ca,
ca.Customer,
ca.Address
}).Take(3);
var w = ctx.CustomerAddresses.Select(x =>
new CustomerAddressHelper() {
CustomerAddress1 = ca,
Customer1 = ca.Customer,
Address1 = ca.Address
}).Take(3);
public class CustomerAddressHelper
{
public CustomerAddress CustomerAddress1;
public Customer Customer1;
public Address Address1;
}
I always prefer to use a strongly typed projection model, instead of dynamic - the generated SQL would be the same.
As far as SQL is concerned, all it needs to know is what columns it needs to return to satisfy the request. Whether you are populating a dynamic object, or a custom type object, is irrelevant from the SQL point of view.
A good/quick way to check what the SQL being executed actually is, is to run SQL Profiler, then watch what hits the DB when the results of the Linq-to-sql query are materialised
I have a Linq query similar to as follows,
var dataByWheels = this.Db.Cars.Cast<IVehicle>().Where(v => (v.NumWheels == 4));
IVehicle in this case is an interface which defines that the class must have a NumWheels property. The Cars class is similar to as follows,
[Table]
public class Car : IVehicle
{
[Column]
public double Weight { get; set; }
...
public int NumWheel
{
get { return 4; }
}
}
So in this case the NumWheels property is not stored in the database, however if the cast is done then it should be OK. However an exception is raised,
The member 'MyProject.IVehicle.NumWheels' has no supported translation to SQL
Which implies that it is not performing th cast. When I insert a ToList() into the chain after Cast it does work but I imagine this is prematurely resolving the query and creating a giant List of the entire table and would like to avoid since I am working on the phone .
What am I doing wrong?
(Note I changed the names and queries to make this simpler)
This is happening because when you issue a Where statement against a LINQ to SQL class, the LINQ to SQL engine expects that you are performing a WHERE statement against the database directly and tries to map the statement to raw T-SQL which it cannot do since NumWheels is not a database column.
To overcome this, you COULD do a ToList on it first, but beware of the performance implications if you do. This code will cycle through the entire Cars table and do the filtering in-memory. I can't think of another way to accomplish your goal, however.
var dataByWheels = this.Db.Cars.ToList().Where(v => (v.NumWheels == 4));
The problem isn't your LINQ, but rather your use context. This particular LINQ query is being used on a Queryable and the Cast will force it to go to the DB and pull back all the results at that point. You can then do the cast on the enumerable and proceed, but it may very well not be what you are looking for in this case.
The error simply indicates that you have written an expression that can not be mapped to SQL and executed on the SQL server. It's an EF thing, not a LINQ thing.
I am currently having to work on a project which uses linq2sql as its database accessing framework, now there are a lot of linq queries which basically do the following:
var result = from <some_table>
join <some_other_table>
join <another_table>
select <some_other_domain_model> // This is a non linq2SQL poco
return result.Where(<Some_Predicate>);
So for example assume you read 3 tables, and then collate the contents into one big higher level model, for sending to a view. Now ignore the mixing of domains, as that doesn't bother me too much, its the final where clause which does.
Now I have not used Linq2Sql much before so would I be right in saying what is going to happen is:
Generate SQL based off the from, join, join, select linq
Retrieve all rows
Map all this data into one big model (in memory)
Loop through all models and then return only the applicable ones
As this is the crux of my question, it would make sense in my mind if the above flow is what would happen, but it has been debated by people who apparently know the framework a lot better than the 4th step is somehow factored into the SQL generation so it will not be pulling back all records, but I dont know how it could be doing that as it NEEDS all the data up front to populate this which it then applies a separate where clause on, so I assume by the 4th point the rows have all been read and are already in memory.
I am trying to push for them to move their where clause into the linq so that it filters out un-needed records at the database level, however I was wondering if anyone can advise as to if my assumptions above are right?
== Edit ==
Have added comment to draw more attention to the fact that the is not a linq2sql generated object and is some random poco hand rolled elsewhere, just to narrow down where my main focus is on the context of the question. As the question is LESS about "does it matter where I put the where clause" and more about "Does the where clause still get factored into the underlying query when it is applied to a non linq2sql object generated from a linq2sql query".
Here is another more concise example of what I mean hopefully drawing the point more towards where my lack of understanding is:
/*
I am only going to put auto properties into the linq2sql entities,
although in the real world they would be a mix of private backing
fields with public properties doing the notiftying.
*/
[global::System.Data.Linq.Mapping.TableAttribute(Name="dbo.some_table_1")]
public class SomeLinq2SqlTable1
{
[global::System.Data.Linq.Mapping.ColumnAttribute(Storage="some_table_1_id", AutoSync=AutoSync.OnInsert, DbType="Int NOT NULL IDENTITY", IsPrimaryKey=true, IsDbGenerated=true)]
public int Id {get;set;}
}
[global::System.Data.Linq.Mapping.TableAttribute(Name="dbo.some_table_2")]
public class SomeLinq2SqlTable2
{
[global::System.Data.Linq.Mapping.ColumnAttribute(Storage="some_table_2_id", AutoSync=AutoSync.OnInsert, DbType="Int NOT NULL", IsPrimaryKey=true, IsDbGenerated=true)]
public int Id {get;set;}
[global::System.Data.Linq.Mapping.ColumnAttribute(Storage="some_table_2_name", AutoSync=AutoSync.OnInsert, DbType="Varchar NOT NULL", IsPrimaryKey=false)]
public string Name {get;set;}
}
[global::System.Data.Linq.Mapping.TableAttribute(Name="dbo.some_table_3")]
public class SomeLinq2SqlTable3
{
[global::System.Data.Linq.Mapping.ColumnAttribute(Storage="some_table_3_id", AutoSync=AutoSync.OnInsert, DbType="Int NOT NULL", IsPrimaryKey=true, IsDbGenerated=true)]
public int Id {get;set;}
[global::System.Data.Linq.Mapping.ColumnAttribute(Storage="some_table_3_other", AutoSync=AutoSync.OnInsert, DbType="Varchar NOT NULL", IsPrimaryKey=false)]
public string Other {get;set;}
}
/*
This is some hand rolled Poco, has NOTHING to do with Linq2Sql, think of it as
a view model of sorts.
*/
public class SomeViewModel
{
public int Id {get;set;}
public string Name {get;set;}
public string Other {get;set;}
}
/*
Here is psudo query to join all tables, then populate the
viewmodel item from the query and finally do a where clause
on the viewmodel objects.
*/
var result = from // Linq2SqlTable1 as t1
join // Linq2SqlTable2.id on Linq2SqlTable1.id as t2
join // Linq2SqlTable3.id on Linq2SqlTable1.id as t3
select new ViewModel { Id = t1.Id, Name = t2.Name, Other = t3.Other }
return result.Where(viewModel => viewModel.Name.Contains("some-guff"));
So given the example above, will the final Where statement be factored into the underlying query, or will the where on the viewModel cause a retrieval and then evaluate in memory?
Sorry for the verbosity to this question but there is very little documentation about it, and this is quite a specific question.
You do not need to push the Where clause any higher. It is fine where it is, as long as result is IQueryable<T> (for some T). LINQ is composable. Indeed, there's absolutely no difference between using the LINQ syntax as using the extension-method syntax, and either would work identically. Basically, when you create a query, it is only building a model of what has been requested. Nothing is executed until you start iterating it (foreach, ToList(), etc). So adding an extra Where on the end is fine: that will get built into the composed query.
You can verify this very simply by monitoring the SQL connection; you'll see that it includes the where clause in the TSQL, and filters at the SQL server.
This allows for some interesting scenarios, for example a flexible search:
IQueryable<Customer> query = db.Customers;
if(name != null) query = query.Where(x => x.Name == name);
if(region != null) query = query.Where(x => x.Region == region);
...
if(dob != null) query = query.Where(x => x.DoB == dob);
var results = query.Take(50).ToList();
In terms of your assumptions, they are incorrect - it is really:
build composable query, composing (separately) from, join, join, select
further compose the query, adding a where (no different to the above compositions)
at some point later, iterate the query
generate sql from the fully-composed query
retreive rows
map into model
yield the results
note that the sql generation only happens when the query is iterated; until then you can keep composing it all day long. It doesn't touch the SQL server until it is iterated.
I did my little research about LINQtoSQL best practices, cause I'm always using this technology with my projects. Take a look to my blog post. maybe it can help you.
http://msguy.net/post/2012/03/20/LINQ-to-SQL-Practices-and-approaches.aspx
The provider knows how the populated properties from your custom model are mapped (because of the select clause in your query) to the actual columns on the database table. So it knows what column on the table it needs to filter when you filter on a property of your custom model. Think of your selected model (weather it be a designer defined entity with all of the columns of the table, or a custom model defined somewhere, or an anonymous type with just the data you need) as just the selected columns before the FROM clause in the SQL query. selecting an anonymous model makes it easily recognizable that the fields in the model correspond to the SELECT list in SQL.
Most important: always remember that var result = from ... is just a query... until it gets iterated result.ToArray(). Try to call your variable query instead of result, and the world may get new new colors when you look again.
Class Customer has the following method, which returns an IQueryable with the linq query to get the active sales for a customer:
public IQueryable<SalesHeader> QueryCurrentSales()
{
// this.SalesHeaders is an association defined in the DBML between
// the Customer and SalesHeader tables (SalesHeader.CustomerId <-> Customer.Id).
return this.SalesHeaders
.Where(sh => sh.Status == 1).AsQueryable();
}
The idea is to centralise the query definition in a single point, so that later we can get the SalesHeader rows, do a count, paginate using the PaginatedList class (from NerdsDinner), and add further Where conditions when searching Sales within active SalesHeaders.
However, after running the SQL Sever profiler, I've discovered that doing the following, gets all the rows and then does the Count in memory.
// cust is an object of type Customer.
int rows = cust.QueryCurrentSales().count();
I guess that the problem lies in the fact that we have used the AsQueryable() method, but I don't know what would be the correct way to accomplish what we intended.
You haven't shown what SalesHeaders is. The fact that you feel you need to do AsQueryable suggests that it's returning an IEnumerable<T> rather than an IQueryable<T>... which is probably the problem. If you could show SalesHeaders, we may be able to help.
It's possible that instead of using AsQueryable you may be able to just cast to IQueryable<SalesHaeder> if the SalesHeaders property is returning an expression which could actually be returned as IQueryable<SalesHeaders> in the first place - but in that case I'd suggest changing the property type instead.
EDIT: Okay, if it's an entity set then that's probably tricky to fix directly.
I know it's ugly, but how about something like:
public IQueryable<SalesHeader> QueryCurrentSales()
{
// You *may* be able to get away with this in the query; I'm not sure
// what LINQ to SQL would do with it.
int id = this.Id;
return context.SalesHeaders
.Where(sh => sh.Status == 1 && sh.CustomerId == id);
}
Here I'm assuming you have some way of getting to the data context in question - I can't remember offhand whether there's an easy way of doing this in LINQ to SQL, but I'd expect so.
This works right. Doing count() on SQL Server.
// cust is an object of type Customer.
int rows = cust.QueryCurrentSales().AsQueryable<SalesHeaders>().count();