I need to go through all my chemicals and send out a warning to users if the expiration date is either 2 days away or 15 days away.
I have it set like that so the warning is sent out twice. Once 2 weeks before it expires, and once 2 days before it expires.
So I have this loop:
foreach (var chemical in _context.Chemical.Where(c => (c.DateOfExpiration - DateTime.Now).TotalDays == 15 || (c.DateOfExpiration - DateTime.Now).TotalDays == 2))
{
// send out notices
var base = _context.Base.Find(chemical.BaseId);
var catalyst = _context.Catalyst.Find(chemical.CatalystId);
_warningService.SendSafetyWarning(chemical, base, catalyst.Name);
}
But whenever I run it, I get this error:
System.InvalidOperationException: 'The LINQ expression 'DbSet<Chemical>
.Where(p => (p.DateOfExpiration - DateTime.Now).TotalDays == 15 || (p.DateOfExpiration - DateTime.Now).TotalDays == 2)'
could not be translated.
I'm not sure why it's giving me this error when I run it.
Is there a better way of doing what I'm trying to do?
Thanks!
DateTime.TotalDays returns a double.
You might have more luck changing your code to:
foreach (var chemical in _context.Chemical.Where(c => ((int)(c.DateOfExpiration - DateTime.Now).TotalDays == 15) || ((int)(c.DateOfExpiration - DateTime.Now).TotalDays == 2)))
{
// send out notices
var base = _context.Base.Find(chemical.BaseId);
var catalyst = _context.Catalyst.Find(chemical.CatalystId);
_warningService.SendSafetyWarning(chemical, base, catalyst.Name);
}
There are two solutions to this problem.
I'll first explain why it is throwing an error. EF is converting your code to an SQL Expression. When working with Expression you cannot parse in any C# code and expect it to work. EF needs to be able to translate it to SQL. That is why you are getting the exception saying that it can't translate your code.
When working directly in the Where clause on the DbSet or the DbContext you are actualy talking to the Where method for an IQueryable. Which has a parameter of Expression<Func<bool, T>> where T is your DbSet<T> type.
You either need to simplify your query so it can be translated to SQL. Or you can convert your IQueryable to an IEnumerable.
When you want to run your query on the SQL server, you need to use the first approach. But when you can run your query on the client you can use the second approach.
The first approached as mentioned by vivek nuna is using EntityFunctions. It might not be enough for your use case, because it has limited functionality.
The statement made that DateTime.Now is different on each iteratoion is not true. Because the query is converted once, it isn't running the query on every iteration. It simply doesn't know how to translate your query to SQL.
https://learn.microsoft.com/en-us/dotnet/api/system.data.objects.entityfunctions?view=netframework-4.8
The second approach means adding AsEnumerable after your DbSet. This will query all your data from your SQL server, and evaluate in C#. This is usually not recommended if you have a large data set. Example:
var chemicals = _context.Chemical.AsEnumerable(); // this will get the complete collection
chemicals.Where(i => i.Value == true); // everything will work now
And answer to a "better" approach. It's a bit cleaner code:
If you have navigation properties for your Base and Catalyst you can also include this in your query.
Note that you can still query server side before pulling everything client side using a Where on the _context.Chemical and then including and calling AsEnumerable.
var chemicals = _context.Chemical.Include(i => i.Base).Include(i => i.Catalyst).AsEnumerable();
foreach (var chemical in chemicals.Where(i => true)) // set correct where clause
{
_warningService.SendSafetyWarning(chemical, chemical.Base, chemical.Catalyst);
}
You can declare the variable before and then pass it to your expression. Like DateOfExpiration == nextFifteenDays …..
var nextFifteenDays = DateTime.Now.Adddays(15);
var nextTwoDays = DateTime.Now.AddDays(2);
Or you can use EntityFunctions.Diffdays
Reason for the error is well explained by #neil that DateTime.Now will keep on changing, so can’t be used.
Related
I am having trouble managing large dataset with linq and telerik model, I cant figure out the problem or how linq is working to execute the query.
I am queryng the database using linq to a 300000+ records and it seems that linq is executing the query before applying the take and skip paramenters.
I am executing this query using linq:
var result = repository.Documents.Where(p => p.TenantId = 1 && p.TipoDocumento == SriDocType && p.RucReceptor == ruc).OrderByDescending(p => p.FechaEmision).Take(20).Skip(0).ToList();
Then I run the same query by using sql:
var result = ((EddocumentRepository)repository).Model.ExecuteQuery<Riverminds.ShardLayer.Eddocument>("SELECT TOP 20 * FROM eddocuments WHERE TenantId = 1 and TipoDocumento = 1 and RucReceptor = '0990017514001' Order By FechaEmision Desc", (new List<System.Data.Common.DbParameter>()).ToArray()).ToList();
On the first Query I get a timeout exception, it takes more than a minute the query, if I change the maxexecutiontime, it will work but will take like 2 minutes.
Now If I run the second query that basically is the same thing but with sql text, it takes just a second or 2 seconds. It is really crazy but it is happening, and I need to use linq because I am working with Kendo Asp Net Mvc and using the ToDataSourceResult that is the same thing than linq. It takes a lot of time executing the query.
Any Idea?.
UPDATE
Doing some Linq Queries with the help of the posted comments, I can figure out that the problem is the paramenters, when I use "ruc" paramenters the LINQ query throw a timeout exception because it takes a lot of time to execute it, The same query with SQL takes 1 second.
Deleting the ruc condition, with Linq takes the same time as SQL, 1 second, I check mappings and it seems ok, the column is a 255 nvarchar nullable, so I think something is wrong with the parameter ruc. I am posting the mapping detail of the column and the linq and sql that takes 1 second.
configuration.HasProperty(x => x.RucReceptor).HasFieldName("_rucReceptor").WithDataAccessKind(DataAccessKind.ReadWrite).ToColumn("RucReceptor").IsNullable().HasColumnType("nvarchar").HasLength(255);
var result = repository.Documents.Where(p => p.TenantId = 1 && p.TipoDocumento == SriDocType).OrderByDescending(p => p.FechaEmision).Take(20).Skip(0).ToList();
var result = ((EddocumentRepository)repository).Model.ExecuteQuery<Riverminds.ShardLayer.Eddocument>("SELECT TOP 20 * FROM eddocuments WHERE TenantId = 1 and TipoDocumento = 1 Order By FechaEmision Desc", (new List<System.Data.Common.DbParameter>()).ToArray()).ToList();
Thanks for your Help
Santiago Munoz
This difference in the query speed usually is due to one of the following:
The generated SQL statement is not the most efficient, in your case I don't believe it is the case as your query is pretty straight forward. However you could inspect the generated SQL statement by executing
string sql = repository.Documents.Where(p => p.TenantId = 1 && p.TipoDocumento == SriDocType && p.RucReceptor == ruc).OrderByDescending(p => p.FechaEmision).Take(20).Skip(0).ToString()
There might be parameter type mismatch causing implicit type conversion at server side that prevents the SQL server to utilize the existing indexes. The usual suspects are "string" type of properties. From your examples I see this: RucReceptor = '0990017514001'. Check the mapping for this column if it is Unicode but in the db is varchar this definitely will affect the performance negatively. Fix the types in the mapping to correspond to those in the DB and it should run fast.
Hope this helps.
Here i am using the below query and its taking lots of time around 14 to 15 seconds for retrieving the large amount of data.
In below Query the CreatedDate is of DateTimeOffset data type.
var naId = UnitOfWork.SalesPhases.FirstOrDefault(p => p.PhaseName =="NA").SalesPhaseId;
var rejectedId = UnitOfWork.SalesPhases.FirstOrDefault(p => p.PhaseName =="Rejected").SalesPhaseId;
var data = UnitOfWork.Leads.Query().AsEnumerable()
.Where(p =>(p.SalesPhaseId == naId || p.SalesPhaseId == rejectedId) &&
p.CreatedDate.Date >= fromDate && p.CreatedDate.Date <= toDate).Select(m =>
new
{
m.LeadId,
m.LeadOwnerId,
m.SalesPhaseId,
m.LeadActivities,
m.Employee,
m.SalesPhase,
m.CompanyName,
m.CreatedDate,
m.LeadHistories,
m.LeadAddresses
}).ToList();
I tried using the AsQueryable instead of the AsEnumerable but it gives the below error:
"The specified type member 'Date' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported."
Can you help me out to reduce the execution time of the query?
Your use of AsEnumerable is forcing the filtering to be done locally. It's pulling in all the data, then filtering it in your app. That's clearly very inefficient. Now, it seems that part of your query can't be directly expressed in LINQ to SQL. I see two options here.
Firstly you could do most of your filtering in SQL, but then do the date filtering locally:
var data = UnitOfWork.Leads.Query()
// Do this part of the query in SQL
.Where(p => p.SalesPhaseId == naId ||
p.SalesPhaseId == rejectedId)
.AsEnumerable()
// Do the rest of the query in-process
.Where(p => p.CreatedDate.Date >= fromDate &&
p.CreatedDate.Date <= toDate)
.Select(...)
That's suitable if the first part will filter it down massively, and then you only need to do local processing of a small set of data.
Alternatively, you could work out what your date filtering means in terms of DateTime. It looks like you could do:
// This may not be required, depending on the source.
fromDate = fromDate.Date;
// This will be, although you may be able to get rid of the ".Date" part.
toDate = toDate.Date.AddDays(1);
var data = UnitOfWork.Leads.Query()
// Do this part of the query in SQL
.Where(p => (p.SalesPhaseId == naId ||
p.SalesPhaseId == rejectedId) &&
p.CreatedDate >= fromDate &&
p.CreatedDate < toDate)
.Select(...)
That's created an equivalent query, but without using the Date property in the query itself.
Everything after AsEnumerable() is executed locally rather than on the server. See also
https://stackoverflow.com/a/2013876/141172
This means that all rows in the table are returned from the database, and then filtered in your C# code.
Remove that call so that the filtering happens server-side.
EDIT
Noticed Jon's comment and it reminded me that he reimplemented LINQ to Objects as a learning exercise. His comments about the AsEnumerable() reimplementation are worth reading
I can describe its behaviour pretty easily: it returns source.
That's all it does. There's no argument validation, it doesn't create another iterator. It just returns source.
You may well be wondering what the point is... and it's all about changing the compile-time type of the expression. I'm going to take about IQueryable in another post (although probably not implement anything related to it) but hopefully you're aware that it's usually used for "out of process" queries - most commonly in databases.
Now it's not entirely uncommon to want to perform some aspects of the query in the database, and then a bit more manipulation in .NET - particularly if there are aspects you basically can't implement in LINQ to SQL (or whatever provider you're using). For example, you may want to build a particular in-memory representation which isn't really amenable to the provider's model.
https://msmvps.com/blogs/jon_skeet/archive/2011/01/14/reimplementing-linq-to-objects-part-36-asenumerable.aspx
Your code should like this..
var naId = UnitOfWork.SalesPhases.FirstOrDefault(p => p.PhaseName =="NA").SalesPhaseId;
var rejectedId = UnitOfWork.SalesPhases.FirstOrDefault(p => p.PhaseName =="Rejected").SalesPhaseId;
var data = UnitOfWork.Leads.Query().AsQueryable()
.Where(p =>(p.SalesPhaseId == naId || p.SalesPhaseId == rejectedId) &&
p.CreatedDate>= fromDate.Date && p.CreatedDate <= toDate.Date).Select(m =>
new
{
m.LeadId,
m.LeadOwnerId,
m.SalesPhaseId,
m.LeadActivities,
m.Employee,
m.SalesPhase,
m.CompanyName,
m.CreatedDate,
m.LeadHistories,
m.LeadAddresses
}).ToList();
Firstly, You need to use .ToQueryable instead of .ToIEnumerable().
Secondly, you cannot use .Date to datetime properties inside a entity framework linq query. That only works for in-memory collections like list and arrays.
I need to have parent and parent.child.count()....in the query.. when i do this it is taking 20 seconds....its not a huge database...Any ideas for optimization...
var plist = context.persons
.Select(p => new
{
p.fullName,
c.personID,
p.Status,
p.Birthdate,
p.Accounts.Count
}).ToList();
Here is a great article on using count() when you really meant to use any()
http://blogs.teamb.com/craigstuntz/2010/04/21/38598/
Do you need to use .count or could you use .any?
http://msdn.microsoft.com/en-us/library/bb534972.aspx
Since this is entity framework, open up the sql profiler and take a look at what sql queries are being sent to the database. It sounds like you may see that a single query is sent to fetch the group identifiers, and then another set of queries (one for each group) might be fetching the count. If that's happening, you'll have to post the linq query for someone to resolve the issue.
Based on the code you sent, it doesn't look like things should be taking that long. I have a few suggestions:
Use LinqPad to do this query. It will let you see the SQL that gets generated. Then run that SQL code in SQL Server Management Studio, and tell it to include the actual execution plan. This will help you learn whether there's a particular point in the query that's taking a lot of time. For example, if you don't have an index on the Account table's PersonId reference, this query will take a lot longer.
Look at how you're using this data. It's very rare that you really need to have all the people in your entire system in memory at the same time. In fact, I suspect that simply getting all this person data out of the database is probably taking a lot more time than the Count() is.
Are you displaying this data? If so, wouldn't it be better to "page" the results, only showing maybe ten entries at a time? You can use the .Take(int) method before calling .ToList() to get only as many entries as you need.
If you're processing and aggregating this data for the sake of site metrics, it's probably better to set up your query to return the end result before it gets evaluated.
If you can describe how this data is being used, or provide a screenshot of the SQL's execution, we can provide more feedback.
I solved a similar problem using the GroupBy method.
IEnumerable> accounts = Accounts.GroupBy(x => x.personID);
accounts.Count() will return the number of accounts that belong to the person.
accounts.Key will return the personID of the group.
I had a somehow similar problem, I tried these and worked out better :
child.count(x=> x.paretnID == inputParentID)
child.where(x=> x.parentID == inputParentID)
my original code which took around 15-20 seconds on each iteration was:
return (isEdit) ? db.ChasisBuys.Single(x => x.ChasisBuyID == long.Parse(Request.QueryString["chbid"])).Chasises.Count(y => y.Bikes.Count > 0 && y.ColorID == buyItems[(int)index].ColorID && y.ChasisTypeID == buyItems[(int)index].ChasisTypeID).ToString() : "-";
new code which runs good is :
**return (isEdit) ? db.Chasises.Where(x => x.ChasisBuyID == long.Parse(Request.QueryString["chbid"])).Count(y => y.Bikes.Count > 0 && y.ColorID == buyItems[(int)index].ColorID && y.ChasisTypeID == buyItems[(int)index].ChasisTypeID).ToString() : "-";**
Database has around 1000 records in chasises , about 5 in chasisBuys and about 20 in Bikes.
my opinion is that Linq to SQL queries does not do preevaluations such in logical statements which for instance if you write "return a && b && c;" if statement a is false other statements are not evaluated and I was expecting such thing in linq to sql but it's not the case.
I'am using wrapper to get some data from table User
IQueryable<StarGuestWrapper> WhereQuery =
session.Linq<User>().Where(u => u.HomeClub.Id == clubId && u.IsActive).Select(
u =>
new StarGuestWrapper()
{
FullName = u.Name + " " + u.LastName,
LoginTime = u.SomeDateTime,
MonthsAsMember = u.SomeIntergerValue,
StarRating = u.SomeOtherInteregValue,
UserPicture = u.Photo.PhotoData,
InstructorFullName = u.SomeInstructorName,
TalkInteractionDuringSession = u.SomeBoolValue,
GoalInteractionDuringSession = u.SomeOtherBoolValue
});
I use this without a problem as a IQueryable so I can do useful things before actually running the query. Like :
WhereQuery.Skip(startRowIndex).Take(maximumRows).ToList();
and so on.
The problem occurs using 'where' statement on query.
For example:
WhereQuery.Where(s => s.StarRating == 1)
will throw an exception in runtime that 'StarRating' doesn't exist in User table - of course it doesn't it's a wrappers property. It will work if I materialize query by
WhereQuery.AsEnumerable().Where(s => s.StarRating == 1)
but then it loses all the sens of using IQueryable and I don't want to do this.
What is strange and interesting that not all properties from wrapper throw error, all the bool values can be used in where statement. Example :
WhereQuery.Where(s => s.TalkInteractionDuringSession)
It works in EntityFramework , why do I get this error in NHibernate and how to get it working the way I want it to ?
Keep in mind the older nHibernate Linq provider is only a partial implementation and is no longer being actively worked on. A new and more complete linq provider is being developed now and will be part of NH3.0 (you can checkout the trunk and build it to see if it addresses this problem).
My recommendation is to change your code to call ToList() at the point when you explicitly wish to hit the database. You are passing a future valued query back from your repository at which point anything could technically happen to the query. Even EF and LINQ2SQL cannot translate any possible linq query into SQL.
I do realize this isn't what you want to be able to do, but I think you are trying to bend the framework to do something in a way this isn't very natural at all.
I need to filter my queries by dates but I don't care in this case about time portion of it that is stored in SQL Database.
I first tried to something like
var now = DateTime.Now.Date;
Where(x => x.CreatedDate.Date.Compare(now) == 0)
but this seems to all get locally checked making the query slow. How can I do this without making it do the check locally?
I am pretty much trying to just find all results that would say have happened today(2020-01-06).
There are a limited number of methods you can use on translatable types when constructing your Lambda / Linq expressions. This is because each method would need additional code so that it could be translated into a sql store expression. It means that you must check that any methods you want to use and expect to be translated into a sql store expression are supported.
In this case the DateTime.Compare is not supported.
The easiest thing to do here is a simple range comparison because the time is included in your persisted value.
var start = DateTime.Now.Date;
var end = start.AddDays(1);
Where(x => x.CreatedDate >= start && x.CreatedDate < end)
This will result in a sargable query.
Use
var now = DateTime.Now.Date
...WHERE(CreatedDate.Date == now)
I just checked that above translates to the following SQL query:
WHERE ((CONVERT(date, [x].[CreatedDate]) = '2019-01-07T00:00:00.000')
I used this (link) method to see what LINQ translates to