I was working this weekend on parallelizing a section of code using Tasks to run all the queries I needed for a dashboard page.
What I have now is many copy/paste methods with almost exactly the same query and a different line at the very end of the method.
Is there a way to write a query against one object context then detach it and pass to a method?
I want to do something like this:
using(DbContext db = new DbContext)
{
var query = db.cars.where(x => x.make == "Ford");
int handleCounts = getDoorHandleCounts(query);
}
public int getDoorHandleCounts(type? query)
{
using(DbContext db = new DbContext())
{
return query.where(x => x.partType == "DoorHandle").Count();
}
}
Any ideas?
Keep in mind all my count() methods are launched from a Task array so they'll be running in parallel. I need a new object context to run each count query on.
I've done some googling and thought about trying to use a pre-compiled query and call it from different object context's, but my real query is kind of complex with allot of if blocks to determine the where condition. Can you compile a query that isn't really simple?
You can't detach and attach a query from/to a context. However, you could reuse the first expression:
Expression<Func<Car,bool>> InitialSelection()
{
return x => x.make == "Ford";
}
public int GetDoorHandleCounts()
{
using(DbContext db = new DbContext())
{
return db.Cars()
.Where(InitialSelection())
.Where(x => x.partType == "DoorHandle").Count();
}
}
And in your task:
int handleCounts = getDoorHandleCounts();
This only works if the initial query is "simple", i.e. does not contain joins and predicates on joined sets that you should repeat over and over in each getX method.
As an alternative, you could initialize a context and feed it to a method that returns a query:
public IQueryable<Car> GetInitialQuery(DbContext db)
{
return db.Cars().Join(....)
.Where(x => x.make == "Ford")
.Where(....);
}
public int GetDoorHandleCounts()
{
using(DbContext db = new DbContext())
{
return GetInitialQuery(db)
.Where(x => x.partType == "DoorHandle").Count();
}
}
Maybe I'm misunderstanding the question, but wouldn't this do what you're looking for?
using(DbContext db = new DbContext)
{
var carsResult = db.cars.where(x => x.make == "Ford");
int handleCounts = getDoorHandleCounts(carsResult);
}
public int getDoorHandleCounts(IEnumerable<Car> result)
{
return result.where(x => x.partType == "DoorHandle").Count();
}
Edit: never mind, I only now saw your mention of the Task array.
Change
public int getDoorHandleCounts(type? query)
to
public int getDoorHandleCounts(IQueryable<cars> query)
And replace cars with the of objects the query will return.
Edit
I would just recommend passing value you're looking to filter on, like this:
{
...
int handleCounts = getDoorHandleCounts("Ford");
}
public int getDoorHandleCounts(string Make)
{
using(DbContext db = new DbContext())
{
return db.cars.where(x => x.make == Make && x.partType == "DoorHandle").Count();
}
}
Related
I'm using Entity Framework Core 6 and I want to find a series of entities in a DbSet. The entities I want to obtain are the ones match some properties in a list of input objects.
I've tried something like this:
public IEnumerable<MyEntity> FindEntities(IEnumerable<MyEntityDtos> entries)
{
return dbContext.MyDbSet.Where(r => entries.Any(e => e.Prop1 == r.Prop1 && e.Prop2 == r.Prop2));
}
But I get the classic EF Core exception saying that my LINQ cannot be translated to a database query (the problem in particular is the entries.Any(...) instruction)
I know I can just loop over the list of entries and obtain the entities one by one from the DbSet, but that is very slow, I was wondering if there was a more efficient way to do this in EF Core that I don't know about.
I think this should work:
public IEnumerable<MyEntity> FindEntities(IEnumerable<MyEntityDtos> entries)
{
var props1=entries.Select(x=>x.Prop1).ToArray();
var props2=entries.Select(x=>x.Prop2).ToArray();
return dbContext.MyDbSet.Where(r => props1.Contains(r.Prop1) && props2.Contains(r.Prop2));
}
In the end, I've done this:
public static IEnumerable<MyEntity> GetRangeByKey(this DbSet<MyEntity> dbSet, IEnumerable<MyEntity> toFind)
{
var keys = new HashSet<string>(toFind.Select(e => e.Id));
IEnumerable<MyEntity> result = null;
for (int i = 0; i < keys.Length; i += 1000)
{
var keyChunk = keys[i..(Math.Min(i + 1000, keys.Length))];
var res = dbSet.Where(x => keyChunk.Any(k => x.ResourceArn == k));
if (result == null)
{
result = res;
}
else
{
result = result.Concat(res);
}
}
return result;
}
Basically I get the keys to find in a HashSet and use it to perform a Where query, which will be translated to a SQL IN clause which is quite fast. I do it in chunks because there's a maximum number of values you can put in a IN clause before the DB engine refuses it.
As commonly known in EF-Core there is no Lazy loading. So that kind of means I'm forced to do my queries with some afterthought. So since I have to think, then i might as well try to do it properly.
I have a Fairly standard update query, but I thought hey, I don't always have to include the HeaderImage and PromoImage FK-objects. There should be a way to make that happen. But I can just not find a way to perform a Include at a later point. In-fact I would like to maybe include right before I actually do work on the object. That way i might be able to automate some of the eagerness.
ArticleContent ac = _ctx.ArticleContents
.Include(a=> a.Metadata)
.Include(a=> a.HeaderImage)
.Include(a=> a.PromoImage)
.Single(a => a.Id == model.BaseID);
ac.Title = model.Title;
ac.Ingress = model.Ingress;
ac.Body = model.Body;
ac.Footer = model.Footer;
if (model.HeaderImage != null)
{
ac.HeaderImage.FileURL = await StoreImage(model.HeaderImage, $"Header_{model.Title.Replace(" ", "_")}_{rand.Next()}");
}
if (model.PromoImage != null)
{
ac.PromoImage.FileURL = await StoreImage(model.PromoImage, $"Promo_{model.Title.Replace(" ", "_")}_{rand.Next()}");
}
ac.Metadata.EditedById = uId;
ac.Metadata.LastChangedTimestamp = DateTime.Now;
await _ctx.SaveChangesAsync();
EXTRA
To be clear, this is EF7 (Core), and im after a solution that allows me to add includes on demand, hopefully after the initial _ctx.ArticleContents.Include(a=> a.Metadata).Single(a => a.Id == model.BaseID).
I'm using something similar to Alexander Derck's solution. (Regarding the exception mentioned in the comments: ctx.ArticleContents.AsQueryable() should also work.)
For a couple of CRUD MVC sites I'm using a BaseAdminController. In the derived concrete Controllers I can add Includes dynamically. From the BaseAdminController:
// TModel: e.g. ArticleContent
private List<Expression<Func<TModel, object>>> includeIndexExpressionList = new List<Expression<Func<TModel, object>>>();
protected void AddIncludes(Expression<Func<TModel, object>> includeExpression)
{
includeIndexExpressionList.Add(includeExpression);
}
Later I saw that I need more flexibility, so I added a queryable. E.g. for ThenInclude().
private Func<IQueryable<TModel>, IQueryable<TModel>> IndexAdditionalQuery { get; set; }
protected void SetAdditionalQuery(Func<IQueryable<TModel>, IQueryable<TModel>> queryable)
{
IndexAdditionalQuery = queryable;
}
Here the Index action:
public virtual async Task<IActionResult> Index()
{
// dynamic include:
// dbset is for instance ctx.ArticleContents
var queryable = includeIndexExpressionList
.Aggregate(dbSet.AsQueryable(), (current, include) => current.Include(include));
if(IndexAdditionalQuery != null) queryable = IndexAdditionalQuery(queryable);
var list = await queryable.Take(100).AsNoTracking().ToListAsync();
var viewModelList = list.Map<IList<TModel>, IList<TViewModel>>();
return View(viewModelList);
}
In the concrete Controller I use:
AddIncludes(e => e.EventCategory);
SetAdditionalQuery(q => q
.Include(e => e.Event2Locations)
.ThenInclude(j => j.Location));
I would create a method to fetch the data which takes the properties to include as expressions:
static ArticleContent GetArticleContent(int ID,
params Expression<Func<ArticleContent, object>>[] includes)
{
using(var ctx = new MyContext())
{
var acQuery = _ctx.ArticleContents.Include(a=> a.Metadata);
foreach(var include in includes)
acQuery = acQuery.Include(include);
return acQuery.Single(a => a.Id == model.BaseID);
}
}
And then in main method:
var ac = GetArticleContent(3, a => a.HeaderImage, a => a.PromoImage);
I need to make a query that return all items with the current price and the current reduction if any.
I tried a few solutions but none seem to work or respect the patterns as i understand them.
The dynamic solution:
I tried to return the data as a dynamic that would be an IQueryable where T would be (Item, CurrentItemPrice, CurrentItemStateIfAny)
public ItemRepository(CoconutEntities context) : base(context){}
public dynamic GetAllCurrentItems(){
var items = (from item in context.Items
select new {
Item = item,
CurrentItemPrice = item.ItemPrices.Where(x => item.ItemPrices.Max(y => y.EffectiveDate) == x.EffectiveDate),
CurrentItemState = item.ItemReductions.Where(x => x.StartDate <= DateTime.Now && DateTime.Now <= x.EndDate)});
return items;
}
But when i try this and i need to add filter, i can't add them the way i was expecting.
public dynamic GetCurrentItems(string filter = "", int categoryId = 1) {
dynamic result;
var categoryServices = new CategoryServices();
IEnumerable<int> categoryIdAndChildCategoriesId = categoryServices.GetCategoryIdAndChildsId(categoryId);
if (!string.IsNullOrWhiteSpace(filter))
{
result = this.GetAllCurrentItems().Where(x => ((string)(x.Item.Name)) == filter);
}
else if(categoryId != 1)
{
result = this.GetAllCurrentItems().Where(x => x.Item.ItemCategories.Any(x => categoryIdAndChildCategoriesId.Contains(x.CategoryId)));
}
return result;
}
Solution 2 : I also tried with Tuple where i should have been able to do somthing like this but i can't create Tuples from Linq to Entities if i understood in an other post. I would need to query all the item first, then use linq to object to create my tuples.
Solution 3 : I can create a viewmodel or a new model that would represent the data i need. I know this would work but i don't understand where it would stand between the two. If it is not a view model, this information won't go to the view it an other way to see an item with only the current information.
In short, there are probably many solutions to this problem, but i need help to understand which solution would be the best and why.
As I understood you you want to do as much as possible on the database - that is good. You might achieve that with tuples like that:
public IEnumerable<Tuple<Item,decimal, decimal>> GetAllCurrentItems(Expression<Func<Item, bool>> filterExpression){
using(MyContext context = new MyContext())
{
var items = context.Items
.Where(filterExpression)
.Select(item => new Tuple<Item,decimal, decimal> (
item,
item.ItemPrices.Where(x => item.ItemPrices.Max(y => y.EffectiveDate) == x.EffectiveDate),
item.ItemReductions.Where(x => x.StartDate <= DateTime.Now && DateTime.Now <= x.EndDate)});
return items;
}
}
And calling it like that:
public IEnumerable<Tuple<Item,decimal, decimal>> GetCurrentItems(string filter = "", int categoryId = 1) {
dynamic result;
var categoryServices = new CategoryServices();
IEnumerable<int> categoryIdAndChildCategoriesId = categoryServices.GetCategoryIdAndChildsId(categoryId);
if (!string.IsNullOrWhiteSpace(filter))
{
result = this.GetAllCurrentItems(x => ((string)(x.Item.Name)) == filter);
}
else if(categoryId != 1)
{
result = this.GetAllCurrentItems(x => x.Item.ItemCategories.Any(x => categoryIdAndChildCategoriesId.Contains(x.CategoryId)));
}
return result;
}
Let's say I have this method to seach my DB for products that fit a certain keyword:
public List<Product> GetByKeyword(string keyword)
{
using(var db = new DataEntities())
{
var query = db.Products.Where(x => x.Description.Contains(keyword);
return query.ToList();
}
}
This works fine, but somewhere else in my project, I want to get the active products only, still by keyword. I would like to do something like :
...
var result = ProductStore.GetByKeyword("Apple", x => x.isActive == 1);
Therefore, I created this method:
public List<Product> GetByKeyword(string keyword, Func<Product, bool> predicate = null)
{
using(var db = new DataEntities())
{
var query = db.Products.Where(x => x.Description.Contains(keyword);
if(predicate != null)
query = query.Where(x => predicate(x));
return query.ToList();
}
}
While this compiles well, the ToList() call generates a NotSupportedException because LINQ does not support the Invoke method.
Of course, I could to it with another method
i.e. GetActiveByKeyword(string keyword) but then I would have to do one for every possible variation, including the ones I didn't think of...
How do I get this to work? Thanks!
Isn't it just this:
if(predicate != null)
query = query.Where(predicate);
it's just as AD.Net says the reason why it works with Expression before is because if you say that the compiler knows it would be a lambda expression
I have not found any good material for querying multiple databases using LINQ. I have been using connection strings to change between databases and display data for user, now i want to implement a search function which query's all databases and returns a list instead of selecting the database before hand.
Here's what I've thrown together so far. which returns one list from one database which obviously is not what i want.
public ActionResult getCustomers(string cust)
{
List<trakman_Entities> teInstances = new List<trakman_Entities>();
IEnumerable<customer> customers = null;
for (var i = 1; i < ConfigurationManager.ConnectionStrings.Count; i++)
{
if (ConfigurationManager.ConnectionStrings[i].ConnectionString.ToLower().Contains("metadata"))
{
string con = ConfigurationManager.ConnectionStrings[i].ConnectionString;
teInstances.Add(new trakman_Entities(con));
}
}
foreach (trakman_Entities entitiy in teInstances)
{
customers = entitiy.customers.Where(c => c.code.StartsWith(cust));
}
foreach(customer c in customers)
{
Response.Write(c.code);
Response.Write(c.name);
}
Response.End();
return View(customers);
}
The problem is that you keep reassigning the customers variable in your foreach loop:
foreach (trakman_Entities entitiy in teInstances)
{
// overwrites on each iteration!
customers = entitiy.customers.Where(c => c.code.StartsWith(cust));
}
Instead, consider:
var customers = teInstances.SelectMany(e => e.customers.Where(c => c.code.StartsWith(cust)))
.ToList();
Or, do do the whole thing using a single LINQ query:
// start with the list of connection string settings cast to IEnumerable<T>
var customers = ConfigurationManager.ConnectionStrings.Cast<ConnectionStringSettings>()
// filter to the relevant connection strings
.Where(s => s.ConnectionString.ToLower().Contains("metadata"))
.SelectMany(s => {
// for each connection string, select a data context
using( var context = new trakman_entities(s.ConnectionString)) {
// for each context, select all relevant customers
return context.customers.Where(c => c.code.StartsWith(cust)).ToArray();
} // and dispose of the context when we're done with it
})
.ToList();