I'm trying to gain performance on this query, and I'd like to know if calling a Select() before the Where() I could have some improvement:
public async Task<List<PostValues>> GetValuesToTheDashboard(DataFilter filter, CancellationToken cancellationToken) {
long startTimestanp = Helpers.UnixTimeNow(filter.StartDate);
long endTimestanp = Helpers.UnixTimeNow(filter.EndDate);
return await
_context.CatchDetails.Where(
x => x.Monitoring.Client.Id == filter.CustomerId && x.Data.published >= startTimestanp
&& x.Data.published <= endTimestanp
&& ((filter.Sentiment == Sentiments.ALL) || x.Sentiment_enum == filter.Sentiment)
&& (filter.MonitoringId == 0 || x.Monitoring.id == filter.MonitoringId)
&& (filter.KeywordId == 0 || x.Keyword.Id == filter.KeywordId)
&& (filter.MotiveId == 0 || x.Motive.Id == filter.MotiveId)
&& (filter.SocialNetwork.Count == 0 || filter.SocialNetwork.Any(s => x.Data.social_media == s))
&& (filter.Busca == "" || x.Data.content_snippet.Contains(filter.Busca))
&& (filter.Gender.Count == 0 || filter.Gender.Any(g => x.Data.extra_author_attributes.gender_enum == g)))
.Select(s => new PostValues() {
CatchDetailsId=s.Id,
Monitoring=s.Monitoring.name,
Keyword=s.Keyword.Text,
Motive=s.Motive.Name,
Sentiment=s.Sentiment_enum,
Gender=s.Data.extra_author_attributes.gender_enum,
SocialMedia=s.Data.social_media,
Country=s.Data.extra_author_attributes.world_data.country_code,
State=s.Data.extra_author_attributes.world_data.region,
Published=s.Data.published
}).ToListAsync(cancellationToken);
}
There might be a way how to improve performance, but it won't be with switching Select and Where (as Chetan mentioned in the comment).
You could build the query in a sequence and based on the filter get a simpler query in the end. This would go like this:
var query = _context.CatchDetails.Where(
x => x.Monitoring.Client.Id == filter.CustomerId && x.Data.published >= startTimestanp
&& x.Data.published <= endTimestanp);
if (filter.Sentiment != Sentiments.ALL)
{
query = query.Where(x => x.Sentiment_enum == filter.Sentiment);
}
if (filter.MonitoringId != 0)
{
query = query.Where(x => x.Monitoring.id == filter.MonitoringId);
}
[...]
return await
query.Select(s => new PostValues() {
[...]
}).ToListAsync(cancellationToken);
Do not forget, the variable query is already on memory of the application when the SQL returns data. If there is many results it could throw memory exception.
I suggest that you limit the range of date on that search.
Related
I have the following LINQ query. Is there a way to outsource the central part into a method?
IQueryable<vwResCache> clientDataset = db.vwResCaches.Where(r =>
clientResources.Contains(r.ID_Resource) &&
r.StartDate < prospectDate &&
(
r.StartDate >= retrospectDate ||
r.StartDate < retrospectDate && (
(clientConfig.FinishReq_FinishedPA && clientConfig.FinishReq_DeliveryNotesPrinted && (!r.FinishedPA.Value || !r.DeliveryNotesPrinted.Value)) ||
(clientConfig.FinishReq_FinishedPA && !clientConfig.FinishReq_DeliveryNotesPrinted && !r.FinishedPA.Value) ||
(!clientConfig.FinishReq_FinishedPA && clientConfig.FinishReq_DeliveryNotesPrinted && !r.DeliveryNotesPrinted.Value)
)
));
As you can see, the must indented part depends on the local object clientConfig. I want to exclude that into s separate method like so:
IQueryable<vwResCache> clientDataset = db.vwResCaches.Where(r =>
clientResources.Contains(r.ID_Resource) &&
r.StartDate < prospectDate &&
(
r.StartDate >= retrospectDate ||
r.StartDate < retrospectDate && GetFilterCriteria(clientConfig, r)
));
This method should look something like:
FilterExpression GetFilterExpression(ClientConfig clientConfig, c)
{
if (clientConfig.FinishReq_FinishedPA && clientConfig.FinishReq_DeliveryNotesPrinted)
return new FilterExpression(c, r => !r.FinishedPA.Value || !r.DeliveryNotesPrinted.Value);
if (clientConfig.FinishReq_FinishedPA)
return new FilterExpression(c, r => !r.FinishedPA.Value);
if (clientConfig.FinishReq_DeliveryNotesPrinted)
return new FilterExpression(c, r => !r.DeliveryNotesPrinted.Value);
}
Is such a thing possible? Or would the effort not justify the benefit?
Calling .Where(...) multiple times has the same effect as combining the individual filters with &&.
Unfortunately, there's no simple equivalent for combining filters with ||, so your filter function would need to include the StartDate logic for each branch.
Something like this should work:
static Expression<Func<vwResCache, bool>> BuildClientConfigFilter(
DateTime retrospectDate,
ClientConfig clientConfig)
{
if (clientConfig.FinishReq_FinishedPA
&& clientConfig.FinishReq_DeliveryNotesPrinted)
{
return r => r.StartDate >= retrospectiveDate
|| (r.StartDate < retrospectiveDate
&& (!r.FinishedPA.Value || !r.DeliveryNotesPrinted.Value));
}
if (clientConfig.FinishReq_FinishedPA
&& !clientConfig.FinishReq_DeliveryNotesPrinted)
{
return r => r.StartDate >= retrospectiveDate
|| (r.StartDate < retrospectiveDate
&& !r.FinishedPA.Value);
}
if (!clientConfig.FinishReq_FinishedPA
&& clientConfig.FinishReq_DeliveryNotesPrinted)
{
return r => r.StartDate >= retrospectiveDate
|| (r.StartDate < retrospectiveDate
&& !r.DeliveryNotesPrinted.Value;
}
return r => r.StartDate >= retrospectiveDate;
}
Usage:
IQueryable<vwResCache> clientDataset = db.vwResCaches
.Where(r => clientResources.Contains(r.ID_Resource))
.Where(r => r.StartDate < prospectDate)
.Where(BuildClientConfigFilter(retrospectDate, clientConfig));
public int being = 0;
public void Insert(Currency current, int number)
{
being = db.Currency.Where(x => x.ForDate == current.ForDate)
.Where(x => x.TitleId == current.TitleId)
.Where(x => x.Value == current.Value).Count(x=>x.Id > 0);
if (being == 0)
{
db.Currency.AddOrUpdate(current);
}
}
it's my code works so slowly, because of getting date but it is not necessary, i don't know other way.
maybe something like :
db.Currency.Find().Value.Equals(current.Value).where...where...
I think your main problem is the .Count(x => x.Id > 0), which forces the evaluation of all the conditions before and actually get the total number.
If you can, replace it with Any. In that way, it just has to get one row at most:
bool isBeing = db.Currency
.Where(x => x.ForDate == current.ForDate
&& x.TitleId == current.TitleId
&& x.Value == current.Value
&& x.Id > 0
)
.Any();
You can do all your conditions in just one where, and also you can skip having a bool variable to check your conditions
if(db.Currency.Where(x => x.ForDate == current.ForDate
&& x.TitleId == current.TitleId && x.Value == current.Value && x.Id > 0).Any())
{
db.Currency.AddOrUpdate(current);
}
My Filter is problematic. My goal is to have the datagrid on load show any with a created person where date > Today's day - 1 month.
I have some filters surname, forename. I want to Display any Person that had some activities within last 3 months. Thats found through this query
(ctx.Interactions.Where(z => z.Attendees.Where(w => w.Person_Id == x.Id).Any() && z.ActivityDate >= recent ).Any())
The issue I'm having is when surname or forename is filled, I want the query to ignore the created date prequisite and the Interaction Prerequisite.
Items.AddRange(ctx.People.
Where(x => (
((Surname.Length == 0) && (Forename.Length == 0)) ?
(x.Created > limit) : true &&
(((Surname.Length == 0) || x.Surname.StartsWith(Surname)) &&
((Forename.Length == 0) || x.Forename.StartsWith(Forename)) &&
(ctx.Interactions.Where(z => z.Attendees.Where(w => w.Person_Id == x.Id).Any() && z.ActivityDate >= recent ).Any())
One thing I did try was to move the Interaction query and had it with the x.created but that ruined the runtime. Currently it runs around 15seconds, with that change it takes about 2 mins. Any tips or suggestions would be great.
recent is today's date - 3 months
You can split the filtering expression:
var filteredPeople = ctx.People; // here var should be replaced with IEnumerable<T> where T is the type of items in People
if (string.IsNullOrEmpty(Surname) && string.IsNullOrEmpty(Forename))
filteredPeople = filteredPeople.Where(x => x.Created > limit);
else
{
if (!string.IsNullOrEmpty(Surname))
filteredPeople = filteredPeople.Where(x => x.Surname.StartsWith(Surname));
if (!string.IsNullOrEmpty(Forename))
filteredPeople = filteredPeople.Where(x => x.Forename.StartsWith(Forename));
filteredPeople = filteredPeople.Where(x => ctx.Interactions.Where(z => z.ActivityDate >= recent)
.Any(z => z.Attendees.Any(w => w.Person_id == p.Id)));
}
Items.AddRange(filteredPeople);
I think your issue may be in the use of your conditional operator. The conditional operator takes the ? and : clauses completely independently. So what you're currently telling it is,
If Surname and Forename are length 0
Then return ALL where x.Created > limit
Else
Then apply all other filters.
I've reworked the query a bit. This should improve performance because it will be retrieving fewer (and more accurate) results.
Items.AddRange(ctx.People.
Where(x => (
((Surname.Length == 0 && Forename.Length == 0) ? x.Created > limit : true)
&&
(
(Surname.Length == 0 || x.Surname.StartsWith(Surname))
&& (Forename.Length == 0 || x.Forename.StartsWith(Forename))
&& (ctx.Interactions.Any(z => z.Attendees.Any(w => w.Person_Id == x.Id) && z.ActivityDate >= recent))
)
Here's another version that doesn't hit the ctx twice, and instead traverses from Person > Attendee > Interaction instead of backwards. This might affect the performance as well:
Items.AddRange(ctx.People.
Where(x => (
((Surname.Length == 0 && Forename.Length == 0) ? x.Created > limit : true)
&&
(
(Surname.Length == 0 || x.Surname.StartsWith(Surname))
&& (Forename.Length == 0 || x.Forename.StartsWith(Forename))
&& x.Attendees.Any(attendee => attendee.Interactions.Any(interaction => interaction.ActivityDate >= recent))
)
I have a where clause that looks up child objects on an entity:
var Lookup = row.Offenses.Where(x => x.Desc == co.Desc && x.Action == co.Action && x.AppealYN == co.AppealYN && x.OffDate == co.OffDate).ToList();
Sometimes the co.OffDate can be null, which will cause an exception. Right now, the only way I can think of to get around that, is to use an if statement:
if (co.OffDate.HasValue)
{
var Lookup = row.Offenses.Where(x => x.Desc == co.Desc && x.Action == co.Action && x.AppealYN == co.AppealYN && x.OffDate == co.OffDate).ToList();
}
else
{
var Lookup = row.Offenses.Where(x => x.Desc == co.Desc && x.Action == co.Action && x.AppealYN == co.AppealYN).ToList();
}
Is there anyway I can re-write the linq query to accomplish what the if statement does? I still want to do a lookup, even if the co.OffDate is null.
You could insert a ternary into your Where filter:
var Lookup = row.Offenses
.Where(x =>
x.Desc == co.Desc
&& x.Action == co.Action
&& x.AppealYN == co.AppealYN
&& (co.OffDate.HasValue ? x.OffDate == co.OffDate : true)
).ToList();
I would rewrite it to be more readable (in my opinion):
var query = row.Offenses.Where(x => x.Desc == co.Desc
&& x.Action == co.Action
&& x.AppealYN == co.AppealYN)
if (co.OffenseDate.HasValue)
{
query = query.Where(x.OffDate == co.OffenseDate);
}
var Lookup = query.ToList();
I'm trying to pass a null value from a RenderAction to another view. But in between, at the controller, my linq lambda expression is not loading the right field, despite the null value going through correctly..
SprintManager.cshtml
<div id="Global_Backlog_Board" class="Board_Panel">
#{Html.RenderAction("ListOfSingleCards", new
{
State_ID = 1
});}
</div>
HomeController.cs
public PartialViewResult ListOfSingleCards( int? Sprint_ID,
int State_ID = 1)
{
var Cards = db.Cards.Where(x => x.State_ID == State_ID &&
x.Sprint_ID == Sprint_ID &&
x.Deleted != 1 &&
x.Archive != 1).ToList();
return PartialView(Cards);
}
So Sprint_ID is being passed over and loaded as null here, but I can't get the query to load the rows correctly.
In fact, the following works:
var Cards = db.Cards.Where(x => x.State_ID == State_ID &&
x.Sprint_ID == null &&
x.Deleted != 1 &&
x.Archive != 1).ToList();
So I suppose I could check if Sprint_ID is null and depending on the result run one of the two seperate queries, but I'd like to understand why my original attempt is not working.
Thank you!
I don't know the correct answer but based on your solution you should be able to tidy it up:
var cards = new List<Card>();
var query = db.Cards.Where(x => x.State_ID == State_ID &&
x.Deleted != 1 &&
x.Archive != 1);
if (Sprint_ID.HasValue)
query = query.Where(x => x.Sprint_ID == Sprint_ID);
else
query = query.Where(x => x.Sprint_ID == null);
cards = query.ToList();
A nullable int won't return "null" in the way that you're thinking. You have to check the HasValue property of it to determine if there is a value, and if so then use it otherwise use null:
public PartialViewResult ListOfSingleCards( int? Sprint_ID,
int State_ID = 1)
{
var Cards = db.Cards.Where(x => x.State_ID == State_ID &&
x.Sprint_ID == Sprint_ID.HasValue ? Sprint_ID.Value : null &&
x.Deleted != 1 &&
x.Archive != 1).ToList();
return PartialView(Cards);
}
Until something better comes a long, I'm using this:
var Cards = new List<Card>();
if (Sprint_ID == null)
{
Cards = db.Cards.Where(x => x.State_ID == State_ID &&
x.Sprint_ID == null &&
x.Deleted != 1 &&
x.Archive != 1).ToList();
}
else
{
Cards = db.Cards.Where(x => x.State_ID == State_ID &&
x.Sprint_ID == Sprint_ID &&
x.Deleted != 1 &&
x.Archive != 1).ToList();
}