We Keep Coding

Linq expression orderBy on child collections

I've got this class with a collection of revisions:
public class Client : BaseEntity
{
public virtual ICollection<Draw> Draws { get; set; }
public virtual ICollection<ClientRevision> ClientRevisions { get; set; }
}
public class ClientRevision : BaseEntity
{
public Guid ClientId { get; set; }
public Client Client { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public double? Increase { get; set; }
public string RevisionCode { get; set; }
public string RevisionNote { get; set; }
}
On controller, I'd like to retrieve the list of clients with the last revision available. I've got a base repository with the following get method:
public async Task<IEnumerable<T>> GetAsync(
Expression<Func<T, bool>> filter = null,
Func<IQueryable<T>, IOrderedQueryable<T>> orderBy = null,
IEnumerable<Expression<Func<T, object>>> includeProperties = null,
int? pageIndex = null,
int? itemsPerPage = null,
bool ignoreQueryFilter = true,
bool ignoreTracking = true)
{
var query = DataContext.GetData<T>(ignoreTracking);
if (filter != null)
{
query = query.Where(filter);
}
if (orderBy != null)
{
query = orderBy(query);
}
if (ignoreQueryFilter)
{
query = query.IgnoreQueryFilters();
}
query = ApplyIncludePropertiesIfNeeded(includeProperties, query);
if (pageIndex.HasValue && itemsPerPage.HasValue)
{
query = query
Skip(pageIndex.Value * itemsPerPage.Value)
Take(itemsPerPage.Value + 1);
}
return await query.ToListAsync();
}
I'd like to order by the list by the Name of the client; the name is in the Client Revision. I try to implement the following filter and orderby, but I cannot create a correct linq expression for retrive the Name of the Client.
public async Task<ActionResult<ClientResponse>> GetClientListAsync()
{
var result = new List<ClientResponse>();
List<Expression<Func<Client, object>>> includes = new() { i => i.Draws };
var sortOn = "ClientRevisions.Name";
var param = Expression.Parameter(typeof(Client), "client");
var parts = sortOn.Split('.');
Expression parent = param;
foreach (var part in parts)
{
parent = Expression.Property(parent, part);
}
var sortExpression = Expression.Lambda<Func<Client, object>>(parent, param);
Func<IQueryable<Client>, IOrderedQueryable<Client>> order = o => o.OrderBy(sortExpression);
}
The error is the following:
Instance property 'Name' is not defined for type 'System.Collections.Generic.ICollection`1[ClientRevision]' (Parameter 'propertyName')
Because is a list. How to retrieve this data from a list? Thanks

LINQ double INNER JOIN on query translation when using selectMany

I have the following LINQ statement:
var repoActivityRowsTest = appManager.GetRepository<ActivityRow>();
var activityRowsTest = await repoActivityRowsTest.Search(f => f.ExcelReport.uploadPhase == RPToolConstants.Phase_Planning, includeProperties: "PlanningInfo")
.Where(f => iso3Alpha3List.Contains(f.ExcelReport.countryOfficeIso3Alpha3))
.SelectMany(sm => sm.PlanningInfo).Select(s => new { s.Year, s.Count, s.ActivityRow.UnitCost })
.GroupBy(g=>new { g.Year }).Select(sg=>new { sg.Key.Year, Total = sg.Sum(sum => sum.UnitCost * sum.Count) })
.ToListAsync();
Which uses the repository pattern. The search function is the one below:
public IQueryable<TEntity> Search(Expression<Func<TEntity, bool>> filter = null,
string includeProperties = "", bool trackChanges = false)
{
IQueryable<TEntity> query = context.Set<TEntity>();
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty.Trim());
}
if (!trackChanges)
{
query = query.AsNoTracking();
}
return query;
}
When I inspect the command that arrives in SQL Server I see that the query is translated in the following SQL:
SELECT [a0].[Year], SUM([a1].[UnitCost] * CAST([a0].[Count] AS decimal(18,2))) AS [Total]
FROM [ActivityRows] AS [a]
INNER JOIN [ExcelReports] AS [e] ON [a].[ExcelReportId] = [e].[Id]
INNER JOIN [ActivityRowPlanningInfo] AS [a0] ON [a].[Id] = [a0].[ActivityRowId]
INNER JOIN [ActivityRows] AS [a1] ON [a0].[ActivityRowId] = [a1].[Id]
WHERE ([e].[uploadPhase] = N'planning')
AND [e].[countryOfficeIso3Alpha3] IN (N'AFG', N'DZA', N'AGO', N'ARM', N'BGD')
GROUP BY [a0].[Year]
It works perfectly, but why there is an inner join duplicated:
INNER JOIN [ActivityRows] AS [a1] ON [a0].[ActivityRowId] = [a1].[Id]
is a non-sense to me!
If I remove it from the SQL it works as before. Is there any issue in my LINQ query that causes this strange SQL?
here is the definition of the entities:
public class ActivityRow : Entity<int>
{
public string Description { get; set; }
public int ExcelReportId { get; set; }
[ForeignKey("ExcelReportId")]
public virtual ExcelReport ExcelReport { get; set; }
public int ActivitySubTypeId { get; set; }
[ForeignKey("ActivitySubTypeId")]
public virtual ActivitySubType ActivitySubType { get; set; }
public int? ActivityCategoryId { get; set; }
[ForeignKey("ActivityCategoryId")]
public virtual ActivityCategory ActivityCategory { get; set; }
public string ResponsibleEntity { get; set; }
[Column(TypeName = "decimal(18,2)")]
public decimal UnitCost { get; set; }
public string Notes { get; set; }
public virtual ICollection<ActivityRowReportingInfo> ReportingInfo { get; set; }
public virtual ICollection<ActivityRowPlanningInfo> PlanningInfo { get; set; }
}
public class ActivityRowPlanningInfo : Entity<int>
{
public int ActivityRowId { get; set; }
[ForeignKey("ActivityRowId")]
public virtual ActivityRow ActivityRow { get; set; }
public int Year { get; set; }
public int Quarter { get; set; }
public int Count { get; set; }
}
and here the definition of the relationships with fluent API:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
//activities
modelBuilder.Entity<ActivityRow>()
.HasMany(b => b.ReportingInfo)
.WithOne(t => t.ActivityRow)
.OnDelete(DeleteBehavior.Cascade);
modelBuilder.Entity<ActivityRow>()
.HasMany(b => b.PlanningInfo)
.WithOne(t => t.ActivityRow)
.OnDelete(DeleteBehavior.Cascade);
...etc.
}

Rewrite query via LINQ Query syntax and you can simplify your query with ease.
The following query do not create non wanted joins:
var repoActivityRowsTest = appManager.GetRepository<ActivityRow>();
var activityRows = repoActivityRowsTest
.Search(f => true);
var resultQuery =
from ar in activityRows
where
ar.ExcelReport.uploadPhase == RPToolConstants.Phase_Planning
&& iso3Alpha3List.Contains(ar.ExcelReport.countryOfficeIso3Alpha3)
from pi in ar.PlanningInfo
group new { ar, pi } by new { pi.Year } into g
select new
{
g.Key.Year,
Total = g.Sum(x => x.ar.UnitCost * x.pi.Count)
};
var result = await resultQuery.ToListAsync();

Elastic search nest dynamic query with object initializer NEST 5.x

Hi I'm a new to elastic nest API and I'm using nest 5.x. I'm currently developing some kind of advanced search page so when user doesn't check a criteria i don't have to include that filter on my query. I'm trying to combine 2 queries under must operator with object initializer approach using nest. How to achieve it? I'm following the example on [https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/bool-queries.html]
var secondSearchResponse = client.Search(new
SearchRequest {
Query = new TermQuery { Field = Field(p => p.Name), Value = "x" } &&
new TermQuery { Field = Field(p => p.Name), Value = "y" } });
But it doesnt work cause Field class doesnt accept type arguments.
I also tried to followed this approach from this topic
[Nest Elastic - Building Dynamic Nested Query
here is my code
public HttpResponseMessage GetSearchResult([FromUri] SearchModels queries)
{
try
{
///
string result = string.Empty;
result += "queryfields + " + queries.queryfields == null ? string.Empty : queries.queryfields;
result += "datefrom + " + queries.datefrom == null ? string.Empty : queries.datefrom;
result += "dateto + " + queries.dateto == null ? string.Empty : queries.dateto;
result += "emitentype + " + queries.emitentype == null ? string.Empty : queries.emitentype;
QueryContainer andQuery = null;
//List<QueryContainer> QueryContainers = new List<QueryContainer>();
IDXNetAnnouncement record = new IDXNetAnnouncement
{
kode_emiten = queries.kodeemiten
};
#region keyword
if (!string.IsNullOrEmpty(queries.queryfields))
{
var val = queries.queryfields;
TermQuery tq = new TermQuery
{
Field = queries.queryfields,
Value = val
};
if (andQuery == null)
andQuery = tq;
else
andQuery &= tq;
//QueryContainers.Add(tq);
}
#endregion keyword
#region kodeemiten
if (!string.IsNullOrEmpty(queries.kodeemiten))
{
var val = queries.kodeemiten;
TermQuery tq = new TermQuery
{
Name = "kode_emiten",
Field = record.kode_emiten,
Value = val
};
if (andQuery == null)
andQuery = tq;
else
andQuery &= tq;
//QueryContainers.Add(tq);
}
#endregion
#region date
if (!string.IsNullOrEmpty(queries.datefrom) && !string.IsNullOrEmpty(queries.dateto))
{
DateRangeQuery dq = new DateRangeQuery();
dq.Name = "tglpengumuman";
dq.LessThanOrEqualTo = DateMath.Anchored(queries.dateto);
dq.GreaterThanOrEqualTo = DateMath.Anchored(queries.datefrom);
dq.Format = "dd/mm/yyyy";
if (andQuery == null)
andQuery = dq;
else
andQuery &= dq;
//QueryContainers.Add(dq);
}
#endregion keyword
var reqs = (ISearchResponse<IDXNetAnnouncement>)null;
if (andQuery != null)
{
reqs = conn.client.Search<IDXNetAnnouncement>(s => s
.AllIndices()
.AllTypes()
.From(queries.indexfrom)
.Size(queries.pagesize)
.Query(q => q.Bool(qb => qb.Must(m => m.MatchAll() && andQuery))));
//var json = conn.client.Serializer.SerializeToString(reqs.ApiCall.ResponseBodyInBytes);
}
else
{
reqs = conn.client.Search<IDXNetAnnouncement>(s => s
.AllIndices()
.AllTypes()
.From(queries.indexfrom)
.Size(queries.pagesize)
.Query(m => m.MatchAll()));
}
//var reqstring = Encoding.UTF8.GetString(conn.client.);
var reslts = this.conn.client.Serializer.SerializeToString(reqs,SerializationFormatting.Indented);
var resp = new HttpResponseMessage()
{
Content = new StringContent(reslts)
};
resp.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
return resp;
}
catch (Exception e)
{
var resp = new HttpResponseMessage()
{
Content = new StringContent(e.ToString())
};
resp.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
return resp;
}
}
But that returns zero result. How to achieve this? Thx anyway.
EDIT :
This is the params variabel definition. Its apoco model of search keywords
public class SearchModels
{
public string queryfields { get; set; }
public string datefrom { get; set; }
public string dateto { get; set; }
public string emitentype { get; set; }
public string kodeemiten { get; set; }
public string issuercode { get; set; }
public int indexfrom { get; set; }
public int pagesize { get; set; }
}
IDXNetAnnouncement is a poco model of search result. Its actualy a document type which is stored on the elastic server
public class IDXNetAnnouncement
{
public string perihalpengumuman { get; set; }
public string attachments { get; set; }
public string createddate { get; set; }
public bool efekemiten_spei { get; set; }
public string jmsxgroupid { get; set; }
public string tglpengumuman { get; set; }
public object errordescription { get; set; }
public string ESversion { get; set; }
public int oldfinalid { get; set; }
public bool efekemiten_etf { get; set; }
public object errorcode { get; set; }
public string jenisemiten { get; set; }
public int pkid { get; set; }
public string judulpengumuman { get; set; }
public string form_id { get; set; }
public bool efekemiten_eba { get; set; }
public string jenispengumuman { get; set; }
public string nopengumuman { get; set; }
public string kode_emiten { get; set; }
public string divisi { get; set; }
public string EStimestamp { get; set; }
public bool efekemiten_obligasi { get; set; }
public long finalid { get; set; }
public bool efekemiten_saham { get; set; }
public string kodedivisi { get; set; }
public string SearchTerms
{
get
{
return string.Format("{0} {1} {2}", judulpengumuman, kode_emiten, nopengumuman);
}
}
}

But it doesnt work cause Field class doesnt accept type arguments.
You need to ensure that you include a using static directive for Nest.Infer i.e.
using static Nest.Infer;
with the rest of the using directives.
.Query(q => q.Bool(qb => qb.Must(m => m.MatchAll() && andQuery))));
No need to wrap in a Must(), just do
.Query(q => q.MatchAll() && andQuery)
which will wrap both queries in a bool query must clause. You also don't need to null check andQuery because NEST is smart enough to not combine the two queries if either or both are null.
if (!string.IsNullOrEmpty(queries.queryfields))
{
var val = queries.queryfields;
TermQuery tq = new TermQuery
{
Field = queries.queryfields,
Value = val
};
if (andQuery == null)
andQuery = tq;
else
andQuery &= tq;
//QueryContainers.Add(tq);
}
NEST has the concept of conditionless queries so you don't need to check it queries.queryfields is null or empty, simply build the query and add it to andQuery. So it would become
var val = queries.queryfields;
andQuery &= new TermQuery
{
Field = queries.queryfields,
Value = val
};
Aside
All of the NEST documentation is generated from source code; you can trace back to the original source file by clicking on any edit link within the documentation. This will take you to a github page, such as this one for bool queries. From here, the document contains an important note that links back to the original source.

Dynamically selecting fields in a linq query

I have a linq query where I’d like to dynamically select only the fields requested by my user.
Currently I’m mapping my Jobs to a data transformation object like this:
var jobs = (from p in jobsDB
select new JobReportDTO()
{
JobID = p.JobID,
EventType = p.EventType,
DateApproved = p.ApprovedDate,
DateEntered = p.EnteredDate,
DateClosed = p.ClosedDate,
StartDate = p.StartDate,
FinishDate = p.FinishDate,
InsuredName = p.InsuredName,
StreetAddress = p.StreetAddress,
Suburb = p.Suburb,
State = p.State,
Postcode = p.Postcode,
.... etc
Within this function I have a number of boolean variables that identify whether that field should be sent to the view, i.e.:
public bool ShowInsuredName { get; set; }
public bool ShowSuburb { get; set; }
public bool ShowICLA { get; set; }
public bool ShowClaimNumber { get; set; }
public bool ShowFileMananger { get; set; }
public bool ShowSupervisor { get; set; }
public bool ShowStatus { get; set; }
... etc
How can I modify my linq query to show selected fields only?
I’ve tried
var jobs = (from p in jobsDB
select new JobReportDTO()
{
JobID = p.JobID,
jobReport.ShowEventType == true ? EventType = p.EventType : "",
... etc
But am getting “invalid initialiser member declarator”

If you can afford LINQ method syntax and use strong naming convention for options like public bool Show{DTOPropertyName} { get; set; }, then you can make your life much easier with the help of the System.Linq.Expressions and the following little helper method
public static class MyExtensions
{
public static IQueryable<TResult> Select<TSource, TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector, object options)
{
var memberInit = (MemberInitExpression)selector.Body;
var bindings = new List<MemberBinding>();
foreach (var binding in memberInit.Bindings)
{
var option = options.GetType().GetProperty("Show" + binding.Member.Name);
if (option == null || (bool)option.GetValue(options)) bindings.Add(binding);
}
var newSelector = Expression.Lambda<Func<TSource, TResult>>(
Expression.MemberInit(memberInit.NewExpression, bindings), selector.Parameters);
return source.Select(newSelector);
}
}
What it does is to remove the assignments which has associated ShowProperty with value set to false.
The usage is simple
var jobs = jobsDB.Select(p => new JobReportDTO
{
JobID = p.JobID,
EventType = p.EventType,
DateApproved = p.ApprovedDate,
DateEntered = p.EnteredDate,
DateClosed = p.ClosedDate,
StartDate = p.StartDate,
FinishDate = p.FinishDate,
InsuredName = p.InsuredName,
StreetAddress = p.StreetAddress,
Suburb = p.Suburb,
State = p.State,
Postcode = p.Postcode,
.... etc
}, jobReport);
If you set a breakpoint in the debugger and examine the newSelector variable, you'll see that only properties that do not have ShowProperty (like JobID) or have ShowProperty = true are included.

Try this way:
EventType = jobReport.ShowEventType == true ? p.EventType : string.Empty,

Send lambda expression from client to server

I have a method that helps to dynamically build a query on a client:
public virtual IList<TEntity> Fill(Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = "")
{
includeProperties = includeProperties ?? "";
var qctx = new TQueryContext
{
QueryType = filter == null ? CommonQueryType.FillAll : CommonQueryType.FillWhere,
Filter = filter,
OrderBy = orderBy
};
qctx.Includes.AddRange(includeProperties.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries));
_detachedServerRepo.Read(qctx);
return qctx.Entities;
}
I want to send qctx to a server repo which might be on another machine. Since a TQueryContext will be typed from QueryContextBase defined in part as below I can't serialize it.
public class QueryContextBase<TEntity, TKey>
where TEntity : StateTrackedObject
where TKey : IEquatable<TKey>
{
public TKey ID { get; set; }
public string Alf { get; set; }
public List<TEntity> Entities { get; set; }
public List<string> Includes { get; set; }
public Expression<Func<TEntity, bool>> Filter { get; set; }
public Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> OrderBy { get; set; }
}
How can I create similar properties to Filter and OrderBy so I can serialize them and then build up the query in the server repo as below:
protected override void FillWhere(TQueryContext qctx)
{
qctx.Entities.AddRange(this.Fill(qctx.Filter, qctx.OrderBy,
qctx.GetIncludesAsString()));
}
protected override void FillAll(TQueryContext qctx)
{
qctx.Entities.AddRange(this.Fill(null, qctx.OrderBy, qctx.GetIncludesAsString()));
}
public virtual IEnumerable<TEntity> Fill(Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = "")
{
includeProperties = includeProperties ?? "";
try
{
IQueryable<TEntity> querySet = DbSet;
if (filter != null)
{
querySet = querySet.Where(filter);
}
foreach (var includeProperty in includeProperties.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
querySet = querySet.Include(includeProperty.Trim());
}
return (orderBy == null) ? querySet.ToList() : orderBy(querySet).ToList();
}
catch (Exception ex)
{
return ex.ThrowDalException<IEnumerable<TEntity>>(OperationType.Read, ex.Message, ex);
}
}

You're right not to want to reinvent the wheel. Check out Serialize.Linq.
You could solve your problem with this library as below:
In your client repo:
public virtual IList<TEntity> Fill(Expression<Func<TEntity, bool>> filter = null,
Expression<Func<IQueryable<TEntity>,
IOrderedQueryable<TEntity>>> orderBy = null,
string includeProperties = "") {
includeProperties = includeProperties ?? "";
try
{
var qctx = new TQueryContext
{
QueryType = filter == null ? CommonQueryType.FillAll : CommonQueryType.FillWhere,
FilterNode = filter == null ? null : filter.ToExpressionNode(),
OrderByNode = orderBy == null ? null : orderBy.ToExpressionNode()
};
And then in your QueryContext just add the extra property and convert:
public ExpressionNode FilterNode { get; set; }
public Expression<Func<TEntity, bool>> Filter
{
get {
return FilterNode == null ? null : FilterNode.ToBooleanExpression<TEntity>();
}
}
public ExpressionNode OrderByNode { get; set; }
public Expression<Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>>> OrderBy
{
get {
return OrderByNode == null ? null : OrderByNode.ToExpression<Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>>>();
}
}

It is not possible to serialize a method... any method (lambda, delegate, ...), thus it is not possible to send a method from client to server.
Best you can do is to send a filter to your service, construct the query there and return the result to the client. That is the usual way.
So in your case instead of passing a Func which filters the data pass the values which the filter uses.
To elaborate that, consider this example of server-side method:
DataType[] GetData(Filter filter, Ordering ordering)
{
var data = GetDataQuerySomeHow(); //for example in EF Context.Table
//filter data according to the filter
if (!string.IsNullOrEmpty(filter.FulltextProperty))
{
data = data.Where(a => a.StringProperty.Contains(filter.FulltextProperty));
}
//do similar thing for ordering
return data.ToArray();
}
Filter and ordering:
public class Filter
{
public string FulltextProperty { get; set; }
//some other filtering properties
}
public class Ordering
{
public string ColumnName { get; set; }
public bool Ascending { get; set; }
}
Client
Filter filter = new Filter()
{
//fill-in whatever you need
};
Ordering ordering = new Ordering(); //also fill in
var data = GetData(filter, ordering);
//display data somewhere