We Keep Coding

C# & ASP.NET Web API: return Ok(returnData); statement is slow

For an ASP.NET Web API using Linq, I am doing a bunch of operations that complete relatively quickly when I step through them using the debugger once the endpoint is called through Swagger.
However, once I arrive at the return statement, it takes ages to complete - several seconds up to minutes.
I have no idea why this would be happening - I imagine it is some kind of slow serialization from the classes to the JSON fields in the HTTP response?
Here is the relevant code:
View models:
public class RetriesAtGoal
{
public string goal { get; set; }
public int count { get; set; }
public string retries { get; set; }
public int total_retries { get; set; }
}
public class SuccessfulApproaches
{
public string goal { get; set; }
public int count { get; set; }
}
public class ApproachesReturnDataViewModel
{
public IEnumerable<RetriesAtGoal> retriesAtGoal { get; set; }
public IEnumerable<SuccessfulApproaches> successfulApproaches { get; set; }
}
Code in API:
var returnData = new ApproachesReturnDataViewModel { };
var totalRetries = db.message
.Select(x => new
{
type = x.type,
agvId = x.agvId,
createdTimestamp = x.createdTimestamp,
source = x.source,
details = x.details
})
.Where(x => x.type == "ErrorHandling" &&
robotList.Contains(x.agvId) &&
x.createdTimestamp >= start &&
x.createdTimestamp < end)
.ToList();
// <1 second
returnData.retriesAtGoal = totalRetries
.GroupBy(x => new { x.source, x.details })
.Select((x, y) => new RetriesAtGoal
{
goal = x.FirstOrDefault().source,
count = x.Count(),
retries = x.FirstOrDefault().details,
total_retries = x.Count() * (int.Parse(x.FirstOrDefault().details) == 0 ? 1 : int.Parse(x.FirstOrDefault().details + 1))
});
// 4 seconds
var totalSuccessful = db.job
.Include(x => x.jobGoals)
.Select(x => new
{
createdTimestamp = x.createdTimestamp,
assignedAgvId = x.assignedAgvId,
jobGoals = x.jobGoals,
id = x.id
})
.Where(x => x.createdTimestamp >= start &&
x.createdTimestamp < end &&
robotList.Contains(x.assignedAgvId) &&
x.jobGoals.Count() == 2)
.ToList();
// <1 second
returnData.successfulApproaches = totalSuccessful
.SelectMany(x => new[] { new { goal = x.jobGoals.First().goalId, job_id = x.id },
new { goal = x.jobGoals.Last().goalId, job_id = x.id } })
.Where(x => !messageCompare.Any(e => e.affectedJobId == x.job_id && e.source == x.goal))
.GroupBy(x => x.goal)
.Select(x => new SuccessfulApproaches { goal = x.Key, count = x.Count() });
// Apparently takes very, very long!
return Ok(returnData);

Linq group result to list of object

I'm trying group a collection of data by it's State but I'm stuck on the correct way to do this:
FileStateInfoDto
public class FileStateInfoDto : EntityDto<int>
{
public string StateName { get; set; }
public int StateNumber { get; set; }
public int FilesByStateCount { get; set; }
}
FileGroupDto
public class FileGroupDto : EntityDto<int>
{
public int CaseId { get; set; }
public string Name { get; set; }
public string ResourceKey { get; set; }
public bool IsFolder { get; set; }
public int SequenceNumber { get; set; }
public IList<FileStateInfoDto> FileStateInfo { get; set; }
public IList<FileGroupDto> FileGroups { get; set; }
public IList<FileInfoDto> Files { get; set; }
}
Here is the code I have:
return await Context.FileGroups
.Include(g => g.Case).Include(g => g.FileGroups).Include(g => g.Files)
.Where(g => g.Id == fileGroupId &&
g.CaseId == caseId &&
g.Case.CaseState != CaseState.Approved &&
g.Case.CaseState != CaseState.Submitted &&
(g.Case.CaseState != CaseState.Draft || g.Case.CreatorUserId == userId))
.OrderBy(g => g.SequenceNumber)
.Select(g => new FileGroup
{
Id = g.Id,
CaseId = g.CaseId,
Name = g.Name,
ResourceKey = g.ResourceKey,
IsFolder = g.IsFolder,
SequenceNumber = g.SequenceNumber,
FileGroups = g.FileGroups,
FileStateInfo = g.Files.GroupBy(f => f.State), <-- My problem
Files = g.Files.Where(f => f.IsActive && f.State != FileApprovalState.Approved).Select(
f => new File
{
Id = f.Id,
CreationTime = f.CreationTime,
CreatorUserId = f.CreatorUserId,
Title = f.Title,
FileName = f.FileName,
URL = f.URL,
Size = f.Size,
KeepOnPortal = f.KeepOnPortal,
CreatorUserName = Context.Users.FirstOrDefault(u => u.Id == (f.CreatorUserId ?? 0)).UserName,
CreatorUserRole = Context.CasePersons.Where(p => p.CaseId == caseId && p.UserId == f.CreatorUserId).Take(1).Select(p => p.CaseRoleType.Title).FirstOrDefault()
}
).ToList()
}).FirstOrDefaultAsync();
I'm trying to figure out how I should write this line FileStateInfo = g.Files.GroupBy(f => f.State) so it will give the expected result as below.
FileStateInfo = [{"StateName":"Approved","StateNumber":1, "FilesByStateCount":22},
{"StateName":"NotApproved","StateNumber":2, "FilesByStateCount":11}]
The State in g.Files.GroupBy(f => f.State) is an enum that contains Approved and NotApproved
StateName = Name of the State.
StateNumber = The Integer assinged.
FilesByStateCount = The files count by this state.
I hope it's possible because I've been trying to make this for a few days now.
I've tried things like this Post

Cannot implicity convert type 'System.Collections.Generic.List<string>' to 'System.Collections.Generic.IList<LM_WebApi.Controllers.Items>

Having problem on my web api "Cannot implicity convert type IQueryable<> to Generic.List". I'm getting the data from Entity Framework.
When I put ToList() it return this error: System.Collections.Generic.List' to 'System.Collections.Generic.IList<LM_WebApi.Controllers.Items>
public IList<Sales_Data> GetSalesData()
{
using (var db = new LM_ReportEntities())
{
var list = db.OE_Invoiced_BaseProducts.Where(x => x.Year == "2020" && x.Period == 1);
IList<Sales_Data> salesData = new List<Sales_Data>();
salesData.Add(new Sales_Data { Items = list.Select(x => x.Item_Number).ToList() });
salesData.Add(new Sales_Data { Periods = list.Select(x => x.Period) });
return salesData;
}
}
Here are my class:
public class Items
{
public string Item_No { get; set; }
}
public class Periods
{
public string Period { get; set; }
}
My model:
public class Sales_Data
{
public IList<Items> Items { get; set; }
public IList<Periods> Periods { get; set; }
}
I want to return the data from Items and Periods as List.

You have to do this in the following way:
var list = db.OE_Invoiced_BaseProducts.Where(x => x.Year == "2020" && x.Period == 1).ToList();
In the next step, populate the lists of Items and Periods types from list collection.
var ItemsList = new List<Items>();
var PeriodsList = new List<Periods>();
foreach (var i in list)
{
Items item = new Items() { Item_No = i.Item_Number };
Periods period = new Periods() { Period = i.Period };
ItemsList.Add(item);
PeriodsList.Add(period);
}
Finally, populate Sales_Data model by the above lists.
var salesData = new Sales_Data() { Items = ItemsList, Periods = PeriodsList };

change db.OE_Invoiced_BaseProducts.Where(x => x.Year == "2020" && x.Period == 1) to db.OE_Invoiced_BaseProducts.Where(x => x.Year == "2020" && x.Period == 1).ToList()
Edit:
var list = db.OE_Invoiced_BaseProducts.Where(x => x.Year == "2020" && x.Period == 1);
...
salesData.Add(new Sales_Data { Items = list.Select(x => new Items{Item_No = x.Item_Number}).ToList() });
salesData.Add(new Sales_Data { Periods = list.Select(x => new Periods{Periods = x.Period}).ToList() });

First of all, I think, you made a mistake in model, change it this way:
public class Sales_Data
{
public string Item_No { get; set; }
public string Period { get; set; }
}
And then change your method like this:
public IList<Sales_Data> GetSalesData()
{
using (var db = new LM_ReportEntities())
{
var list = db.OE_Invoiced_BaseProducts.Where(x => x.Year == "2020" && x.Period == 1).ToList();
return (from baseProduct in db.OE_Invoiced_BaseProducts
where baseProduct.Year = "2020"
&& baseProduct.Period == 1
select new Sales_Data()
{
Item_No = baseProduct.Item_Number,
Period = baseProduct.Period
}).ToList();
}
}

Linq to entities: finding matches

I have these two EF classes:
class Row
{
public long CategoryId { get; set; }
public virtual Category Category { get; set; }
public long VesselId { get; set; }
public virtual Vessel Vessel { get; set; }
public int TruckType { get; set; }
}
class RowFilter
{
public long? CategoryId { get; set; }
public virtual Category Category { get; set; }
public long? VesselId { get; set; }
public virtual Vessel Vessel { get; set; }
public int? TruckType { get; set; }
public long? PortId { get; set; }
public virtual Port Port { get; set; }
public bool IsMatch(Row row)
{
if (CategoryId == null || CategoryId == row.CategoryId) {
if (VesselId == null || VesselId == row.VesselId) {
if (TruckType == null || TruckType == row.TruckType) {
if (PortId == null || PortId == row.Vessel.PortId) {
return true;
}
}
}
}
return false;
}
}
That is:
A Filter matches a Row if IsMatch() returns true for that row.
I have a list of rows, in an IQueryable manner:
var rows = dbContext.Rows.AsQueryable().Where(...);
...and for each row, I want to select (project) the row itself and the list of filters that match this row. I can do this easily in a Linq-to-Objects way ("in memory"):
// Linq-to-objects
rows.ToList().Select(r => new
{
row = r,
filters = dbContext.RowsFilters.Where(f => f.IsMatch(r))
};
Question is... is it possible to do it with Linq-to-Entities? (sql, not "in memory")
In a static world, I would have these navigation properties:
class Row
{
...
public virtual List<RowFilter> RowFilters { get; set; }
}
class RowFilter
{
...
public virtual List<Rows> Rows { get; set; }
}
but... that means a lot of updating: when creating a new RowFilter, when creating a new Row, etc.

You can do the following steps:
Modify the IsMatch method to return a Expression<Func<Row, bool>> type and implement it like this :
public Expression<Func<Row, bool>> IsMatch()
{
Expression<Func<Row, bool>> filter = r => (CategoryId == null || CategoryId == r.CategoryId)
&& (VesselId == null || VesselId == r.VesselId)
&& (TruckType == null || TruckType == r.TruckType)
&& (PortId == null || PortId == r.PortId);
return filter;
}
Then just use it like this :
var rowFilter = new RowFilter { PortId = 1, CategoryId = 2, TruckType = 3, VesselId = 4 };
var query = context.Rows.Where(rowFilter.IsMatch());
All the linq are translated into SQL then executed on the server side. The generated SQL by EF looks like the following:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[CategoryId] AS [CategoryId],
[Extent1].[VesselId] AS [VesselId],
[Extent1].[TruckType] AS [TruckType],
[Extent1].[PortId] AS [PortId]
FROM [dbo].[Rows] AS [Extent1]
WHERE (#p__linq__0 IS NULL OR #p__linq__1 = [Extent1].[CategoryId]) AND (#p__linq__2 IS NULL OR #p__linq__3 = [Extent1].[VesselId]) AND (#p__linq__4 IS NULL OR #p__linq__5 = [Extent1].[TruckType]) AND (#p__linq__6 IS NULL OR #p__linq__7 = CAST( [Extent1].[PortId] AS bigint))

You can use the following query:
var query = from r in context.Rows
from f in context.RowFilters.Where(f =>
(f.CategoryId == null || f.CategoryId == r.CategoryId) &&
(f.VesselId == null || f.VesselId == r.VesselId) &&
(f.TruckType == null || f.TruckType == r.TruckType) &&
(f.PortId == null || f.PortId == r.Vessel.PortId))
.DefaultIfEmpty()
let x = new {r, f}
group x by x.r
into gr
select new
{
row = gr.Key,
filters = gr.Select(y => y.f).Where(yf => yf != null)
};
var result = query.ToList();
Here is an alternative syntax:
var query = context.Rows
.SelectMany(r =>
context.RowFilters.Where(f =>
(f.CategoryId == null || f.CategoryId == r.CategoryId) &&
(f.VesselId == null || f.VesselId == r.VesselId) &&
(f.TruckType == null || f.TruckType == r.TruckType) &&
(f.PortId == null || f.PortId == r.Vessel.PortId))
.DefaultIfEmpty()
.Select(f => new {r, f}))
.GroupBy(x => x.r)
.Select(x => new
{
row = x.Key,
filters = x.Select(y => y.f).Where(yf => yf != null)
});

LINQ- Max in where condition

I have a class TaskWeekUI with this definition:
public class TaskWeekUI {
public Guid TaskWeekId { get; set; }
public Guid TaskId { get; set; }
public Guid WeekId { get; set; }
public DateTime EndDate { get; set; }
public string PersianEndDate { get; set; }
public double PlanProgress { get; set; }
public double ActualProgress { get; set; } }
and I wrote this query :
TaskWeekUI ti = tis.First( t => t.PlanProgress > 0 && t.EndDate == tis.Where(p => p.PlanProgress != null && p.PlanProgress > 0).Max(w => w.EndDate));
Is this query is true? Can I write my query better than this?

I think you want the one whose PlanProgress > 0 has a most recent EndDate.
TaskWeekUI ti = tis.Where(t => t.PlanProgress > 0)
.OrderByDescending(t => t.EndDate)
.FirstOrDefault();

This query seems to be correct from point of view of result obtained.
But in your inner query tis.Where(p => p.PlanProgress != null && p.PlanProgress > 0).Max(w => w.EndDate) is computed for each element in collection with t.PlanProgress > 0
So its a better way to obtain Max value outside of a query as follows:
var max = tis.Where(p => p.PlanProgress != null && p.PlanProgress > 0).Max(w => w.EndDate);
tis.First( t => t.PlanProgress > 0 && t.EndDate == max);
Going further p.PlanProgress != null is allways true since p.PlanProgress is not of Nullable type. So our code becomes like this:
var max = tis.Where(p => p.PlanProgress > 0).Max(w => w.EndDate);
tis.First( t => t.PlanProgress > 0 && t.EndDate == max);
Or you can change a definition of your class and make p.PlanProgress of Nullable type:
public class TaskWeekUI {
public Guid TaskWeekId { get; set; }
public Guid TaskId { get; set; }
public Guid WeekId { get; set; }
public DateTime EndDate { get; set; }
public string PersianEndDate { get; set; }
public double? PlanProgress { get; set; }
public double ActualProgress { get; set; }
}
var max = tis.Where(p => p.PlanProgress.HasValue && p.PlanProgress.Value > 0).Max(w => w.EndDate);
tis.First( t => t.PlanProgress.HasValue && t.PlanProgress.Value > 0 && t.EndDate == max);

var max_Query =
(from s in db.Songs
join bk in db.Albums on s.BookId equals addAlbumDetailsViewModel.BookId
select s.SongId).Max();
max_Query++;

You don't need compare PlanProgress with null because double is struct type, it can't be null.
If you want TaskWeekUI with Max EndDate and positive PlanProgress You can try this code:
TaskWeekUI ti = tis.Where(t => t.PlanProgress > 0).Max(w => w.EndDate);