I want to create a reusable query object for my project that uses ElasticSearch. I am already using a similar QueryObject from the Generic Unit of Work/Repositories by LongLe for queries against the database using Entity Framework.
I can't really seem to wrap my head around exactly how to do this - I'm not sure how to "chain" the parts of the lambda expression together. I've tried using Expression/Func but this is a new area for me that I do not fully understand. I feel as though I'm just stabbing in the dark.
Since I don't even know exactly how to word my question, here's an example of what I am currently doing, what I am trying to do, and my progress so far:
What I currently have to do:
ISearchResponse<DemoIndexModel> result = client.Search<DemoIndexModel>(s => s.Query(
q => q.Term(t => t.FirstName, firstName)
&& q.Term(t => t.LastName, lastName)));
What I would like to do:
var query = new DemoIndexQuery();
query = query.ByFirstName(firstName);
query = query.ByLastName(lastName);
result = client.Search<DemoIndexModel>(s => s.Query(query.Compile()));
Code so far:
public abstract class ElasticQueryObject<T> where T : class
{
private Func<QueryDescriptor<T>, QueryContainer> _query;
// tried using Expression, still completely lost
private Expression<Func<QueryDescriptor<T>, QueryContainer>> _expression;
public Func<QueryDescriptor<T>, QueryContainer> Compile()
{
return _query;
}
public Func<QueryDescriptor<T>, QueryContainer> And(Func<QueryDescriptor<T>, QueryContainer> query)
{
if (_query == null)
{
_query = query;
}
else
{
// how do I chain the query??? I only can figure out how to set it.
}
return null;
}
}
public class DemoIndexQuery : ElasticQueryObject<DemoIndexModel>
{
public DemoIndexQuery ByFirstName(string firstName)
{
And(p => p.Term(term => term.FirstName, firstName));
return this;
}
public DemoIndexQuery ByLastName(string lastName)
{
And(p => p.Term(term => term.LastName, lastName));
return this;
}
}
You're breaking the LINQ contract. The query should be immutable - all the methods operating on the query should return a new query instead of modifying the old one. Due to the way the query is built, this is intrinsically composable, so instead of
public DemoIndexQuery ByFirstName(string firstName)
{
And(p => p.Term(term => term.FirstName, firstName));
return this;
}
you can just use this:
public DemoIndexQuery ByFirstName(string firstName)
{
return Where(p => p.Term(term => term.FirstName, firstName));
}
If this is not possible for some reason, you'll need to handle building the expression tree yourself, before you pass it forward. The simplest way would be something like this:
Expression<...> oldQuery = ...;
var newCondition = (Expression<...>)(p => p.Term(...));
return Expression.And(oldQuery, newCondition);
If your query provider doesn't support this, you'll need a bit more work - you can build the whole where predicate yourself separately, and then make sure you fix the lambdas and lambda parameters.
Related
Lets say that I have a Repository with this function
public async Task<IEnumerable<Contacts>> GetAll()
{
return await _context.Contacts.ToListAsync();
}
Where the Contacts Entity is the same one returning the call. But I didn't want to use the same class because there's some fields that I like to keep out of the call. There's any way that I could "mirror" a second model called "ContactsModel" to return the data without using Anonymous calls like :
var result = context.t_validation.Where(a => a.isvalidated == 10).Select(x => new
{
x.date_released,
x.utoken,
x.Images,
x.images_key,
x.Type
});
Of putting into a loop and passing to this new Model :
foreach (var item in list)
{
decp.Add(new ValidationModel
{
uToken = item.utoken,
Date = item.date_released,
Images = bc.Decrypt(item.Images, item.images_key),
Type = item.Type
});
}
Thanks!
Because you are using custom method to decrypt an image, you will not be able to include it in the query, because EF will not be able to translate it into sql query.
Anonymous approach would be the best one
public async Task<IEnumerable<Contacts>> GetAll()
{
var models = await _context
.Contacts
.Select(contact => new
{
contact.date_released,
contact.utoken,
contact.Images,
contact.images_key,
contact.Type
})
.ToListAsync()
return models
.Select(item => new ValidationModel
{
uToken = item.utoken,
Date = item.date_released,
Images = bc.Decrypt(item.Images, item.images_key),
Type = item.Type
}
.ToList();
}
Of course you can wrap it with an extension methods, but if you are using this mapping only in one place you don't need to.
I have this method:
public List<object> GetThings(List<Guid> listOfGuids)
{
var query = serviceContext.Xrm.crmEntity;
bool anyTypeOfSearch = false; // use this to know if we have actually applied any search criteria.
if(listOfGuids != null && listOfGuids.Count > 0)
{
query = query.Where(x => listOfGuids.Contains(x.lgc_muncipalityid.Id));
anyTypeOfSearch = true;
}
var result = new List<object>();
if(anyTypeOfSearch) // instead of a variable here, can i check if there are any whereconditions applied to the query?
result = query
.Select(x => new SupplierSearchResultModel()
{
Id = x.Id,
Name = x.lgc_name,
})
.ToList();
LogMessage("GetThings.Query", <insert code to get query.Where condition tostring()>);
return result;
}
In the real code there are several different if structures with .Where conditions in them and sometimes a call can reach this code without any parameters. In this case I don't want to run the query as the result set would be huge. So I only want to run the query if at least once the .Where() condition has been applied.
Now my question is, can I check a lambda query variable for if it has any .Where() conditions applied without using an external bool like I am?
An alternate interesting usage point would be if there is some way to get some sort of query.Where().ToString() method that would show what conditions will be applied which could be logged in case of errors...
Quick & dirty, if you don't care about having a pretty result:
LogMessage(query.Expression.ToString());
But it will not show you the content of your array parameter, though.
edit Better solutions:
1) What you are looking for is an expression visitor. A template for what you want to do here, which should then be used like:
LogMessage(query.ToPrettyString());
2) Think about an expression query.Where(x=>x.member == GetSomething()) do you want it to be printed like that ? Or do you want GetSomething() result to appear as a string result ? If the second solution, then that's something you can do with this
You can create your own implementation of the ExpressionVisitor to traverse the nodes of the expression. You can do something like this:
public class WhereVisitor : ExpressionVisitor
{
private static bool _filter;
private static WhereVisitor _visitor = new WhereVisitor();
private WhereVisitor() { }
public new static bool Visit(Expression expression)
{
_filter = false;
//Cast to ExpressionVisitor to use the default Visit and not our new one
((ExpressionVisitor)_visitor).Visit(expression);
return _filter;
}
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method.Name == "Where")
_filter = true;
return base.VisitMethodCall(node);
}
}
And use it like this:
bool containsWhere = WhereVisitor.Visit(query.Expression);
If you want you can of course expand the visitor to save the expressions that contain a Where clause, but this one will just tell you if there are is Where or not.
As commonly known in EF-Core there is no Lazy loading. So that kind of means I'm forced to do my queries with some afterthought. So since I have to think, then i might as well try to do it properly.
I have a Fairly standard update query, but I thought hey, I don't always have to include the HeaderImage and PromoImage FK-objects. There should be a way to make that happen. But I can just not find a way to perform a Include at a later point. In-fact I would like to maybe include right before I actually do work on the object. That way i might be able to automate some of the eagerness.
ArticleContent ac = _ctx.ArticleContents
.Include(a=> a.Metadata)
.Include(a=> a.HeaderImage)
.Include(a=> a.PromoImage)
.Single(a => a.Id == model.BaseID);
ac.Title = model.Title;
ac.Ingress = model.Ingress;
ac.Body = model.Body;
ac.Footer = model.Footer;
if (model.HeaderImage != null)
{
ac.HeaderImage.FileURL = await StoreImage(model.HeaderImage, $"Header_{model.Title.Replace(" ", "_")}_{rand.Next()}");
}
if (model.PromoImage != null)
{
ac.PromoImage.FileURL = await StoreImage(model.PromoImage, $"Promo_{model.Title.Replace(" ", "_")}_{rand.Next()}");
}
ac.Metadata.EditedById = uId;
ac.Metadata.LastChangedTimestamp = DateTime.Now;
await _ctx.SaveChangesAsync();
EXTRA
To be clear, this is EF7 (Core), and im after a solution that allows me to add includes on demand, hopefully after the initial _ctx.ArticleContents.Include(a=> a.Metadata).Single(a => a.Id == model.BaseID).
I'm using something similar to Alexander Derck's solution. (Regarding the exception mentioned in the comments: ctx.ArticleContents.AsQueryable() should also work.)
For a couple of CRUD MVC sites I'm using a BaseAdminController. In the derived concrete Controllers I can add Includes dynamically. From the BaseAdminController:
// TModel: e.g. ArticleContent
private List<Expression<Func<TModel, object>>> includeIndexExpressionList = new List<Expression<Func<TModel, object>>>();
protected void AddIncludes(Expression<Func<TModel, object>> includeExpression)
{
includeIndexExpressionList.Add(includeExpression);
}
Later I saw that I need more flexibility, so I added a queryable. E.g. for ThenInclude().
private Func<IQueryable<TModel>, IQueryable<TModel>> IndexAdditionalQuery { get; set; }
protected void SetAdditionalQuery(Func<IQueryable<TModel>, IQueryable<TModel>> queryable)
{
IndexAdditionalQuery = queryable;
}
Here the Index action:
public virtual async Task<IActionResult> Index()
{
// dynamic include:
// dbset is for instance ctx.ArticleContents
var queryable = includeIndexExpressionList
.Aggregate(dbSet.AsQueryable(), (current, include) => current.Include(include));
if(IndexAdditionalQuery != null) queryable = IndexAdditionalQuery(queryable);
var list = await queryable.Take(100).AsNoTracking().ToListAsync();
var viewModelList = list.Map<IList<TModel>, IList<TViewModel>>();
return View(viewModelList);
}
In the concrete Controller I use:
AddIncludes(e => e.EventCategory);
SetAdditionalQuery(q => q
.Include(e => e.Event2Locations)
.ThenInclude(j => j.Location));
I would create a method to fetch the data which takes the properties to include as expressions:
static ArticleContent GetArticleContent(int ID,
params Expression<Func<ArticleContent, object>>[] includes)
{
using(var ctx = new MyContext())
{
var acQuery = _ctx.ArticleContents.Include(a=> a.Metadata);
foreach(var include in includes)
acQuery = acQuery.Include(include);
return acQuery.Single(a => a.Id == model.BaseID);
}
}
And then in main method:
var ac = GetArticleContent(3, a => a.HeaderImage, a => a.PromoImage);
Let's say I have this method to seach my DB for products that fit a certain keyword:
public List<Product> GetByKeyword(string keyword)
{
using(var db = new DataEntities())
{
var query = db.Products.Where(x => x.Description.Contains(keyword);
return query.ToList();
}
}
This works fine, but somewhere else in my project, I want to get the active products only, still by keyword. I would like to do something like :
...
var result = ProductStore.GetByKeyword("Apple", x => x.isActive == 1);
Therefore, I created this method:
public List<Product> GetByKeyword(string keyword, Func<Product, bool> predicate = null)
{
using(var db = new DataEntities())
{
var query = db.Products.Where(x => x.Description.Contains(keyword);
if(predicate != null)
query = query.Where(x => predicate(x));
return query.ToList();
}
}
While this compiles well, the ToList() call generates a NotSupportedException because LINQ does not support the Invoke method.
Of course, I could to it with another method
i.e. GetActiveByKeyword(string keyword) but then I would have to do one for every possible variation, including the ones I didn't think of...
How do I get this to work? Thanks!
Isn't it just this:
if(predicate != null)
query = query.Where(predicate);
it's just as AD.Net says the reason why it works with Expression before is because if you say that the compiler knows it would be a lambda expression
I am using this expression to get all products:
Expression<Func<Product, ProductInCityDto>> GetAllProducts
{
get
{
return product => new ProductInCityDto
{
ProductName = product.ProductName,
CityName = product.Store.City.Name,
CountryName = product.Store.City.Country.Name,
ProductAvgPrice = CalculateAvg(product)
.
.
.
}
}
}
I am using the function CalculateAvg to calculate the product's average price. The calculation is in separated function because I'm using this code in several places.
However, this approach causes multiple calls to the database.
Is it possible to replace the function CalculateAvg with Linq Expression, in order to have only one call to the DB?
Edit:
The function CalculateAvg looks something like this:
public static decimal CalculateAvg(object tempObj)
{
Product obj = tempObj as Product;
return Convert.ToDecimal
(obj.Sales.Where(n => n.type != 1)
.Average(n=>n.Price)
);
}
First of all I would refactor your function into a extension method and remove the conversion to isolate some possibilities. Remember that you have to place an extension method in a static class.
public static double CalculateAvg(this Product product)
{
return product.Sales.Where(n => n.type != 1).Average(n=>n.Price);
}
If this would still cause multiple round trips, please post the query you see in the profiler so we can work from that.
The difficult part of this isn't the conversion to the Linq expression. The difficult part is the invoking of the Expression. Which is hard. But worst of all, it isn't typesafe, and involves something akin to reflection.
The actual magic part is IQueryable.Provider.Execute(Expression)..
You can certainly try this...but it is by no means tested. Not to mention, pretty fugly.
Expression<Func<Product, ProductInCityDto>> GetAllProducts
{
get
{
return product => new ProductInCityDto
{
ProductName = product.ProductName,
CityName = product.Store.City.Name,
CountryName = product.Store.City.Country.Name,
//Bit of a hack to coerce the Provider out...
ProductAvgPrice = product.Sales
.AsQueryable.Provider
.Execute<double>(CalculateAvg(product), Expression.Constant(product, typeof (Product)) ),
.
.
.
}
}
}
public Expression<Func<Product, double>> CalculateAvg()
{
return product => product.Sales.Where(n => n.type != 1).Average(n=>n.Price);
}