I have an entity in linq to SQL that i query. The query result, of the kind IQueryable is passed through repository and is manipulated in various way.
I noticed that the first query i do, to fetch the first IQueryable is not efficient due to poor ORM translation (and there is nothing more i can do from ORM side), and i would like to use a view instead.
But i don't know how to plug in a view so that it will produce a IQueryable<MyTable> instead of IQueryable<MyView>.
Here an example :
public class MyRepository
{
MyDataContext _dataContext = new MyDataContext();
public IQueryable<MyTable> ComplexRead (..parameters..)
{
return _dataContext.MyTables.Where (..something too complex..); //i would change this query with a view
}
public IQueryable<MyTable> Paging (IQueryable<MyTable> source, int page, int pageSize )
{
return source.Skip(page*pageSize).Take(pageSize);
}
}
public class Program
{
static void Main(string[] args)
{
var repository = new MyRepository();
var query = repository.ComplexRead (..params..);
var result = repository.Paging (query ,0,10);
}
}
As you can see i need to pass IQueryable arround (in a complex project so not easily replaceable), then consider that the Linq table of MyTable will load data with some prefetch (so all the dependant entity are fetched at once).
Because ComplexRead is not optimizable enought with linq (take this fact as granted) i need to replace it with a view. The veiw return the same structure that the un-optimized ORM generated query will (tested with SQL Profiler) so in theory can be replace the table (with all prefetch); but i don't know how to tell to ORM to use that view to fetch MyTable data.
Related
Suppose I have built up, through some conditional logic over many steps, an IQueryable<T> instance we'll call query.
I want to get a count of total records and a page of data, so I want to call query.CountAsync() and query.Skip(0).Take(10).ToListAsync(). I cannot call these in succession, because a race condition occurs where they both try to run a query on the same DbContext at the same time. This is not allowed:
"A second operation started on this context before a previous asynchronous operation completed. Use 'await' to ensure that any asynchronous operations have completed before calling another method on this context. Any instance members are not guaranteed to be thread safe."
I do not want to 'await' the first before even starting the second. I want to fire off both queries as soon as possible. The only way to do this is to run them from separate DbContexts. It seems ridiculous that I might have to build the entire query (or 2, or 3) side-by-side starting with different instances of DbSet. Is there any way to clone or alter an IQueryable<T> (not necessarily that interface, but it's underlying implementation) such that I can have one copy that runs on DbContext "A", and another that will run on DbContext "B", so that both queries can be executing simultaneously? I'm just trying to avoid recomposing the query X times from scratch just to run it on X contexts.
There is no standard way of doing that. The problem is that EF6 query expression trees contain constant nodes holding ObjectQuery instances which are bound to the DbContext (actually the underlying ObjectContext) used when creating the query. Also there is a runtime check before executing the query if there are such expressions bound to a different context than the one executing the query.
The only idea that comes in my mind is to process the query expression tree with ExpressionVisitor and replace these ObjectQuery instances with new ones bound to the new context.
Here is a possible implementation of the aforementioned idea:
using System.Data.Entity.Core.Objects;
using System.Data.Entity.Infrastructure;
using System.Linq;
using System.Linq.Expressions;
namespace System.Data.Entity
{
public static class DbQueryExtensions
{
public static IQueryable<T> BindTo<T>(this IQueryable<T> source, DbContext target)
{
var binder = new DbContextBinder(target);
var expression = binder.Visit(source.Expression);
var provider = binder.TargetProvider;
return provider != null ? provider.CreateQuery<T>(expression) : source;
}
class DbContextBinder : ExpressionVisitor
{
ObjectContext targetObjectContext;
public IQueryProvider TargetProvider { get; private set; }
public DbContextBinder(DbContext target)
{
targetObjectContext = ((IObjectContextAdapter)target).ObjectContext;
}
protected override Expression VisitConstant(ConstantExpression node)
{
if (node.Value is ObjectQuery objectQuery && objectQuery.Context != targetObjectContext)
return Expression.Constant(CreateObjectQuery((dynamic)objectQuery));
return base.VisitConstant(node);
}
ObjectQuery<T> CreateObjectQuery<T>(ObjectQuery<T> source)
{
var parameters = source.Parameters
.Select(p => new ObjectParameter(p.Name, p.ParameterType) { Value = p.Value })
.ToArray();
var query = targetObjectContext.CreateQuery<T>(source.CommandText, parameters);
query.MergeOption = source.MergeOption;
query.Streaming = source.Streaming;
query.EnablePlanCaching = source.EnablePlanCaching;
if (TargetProvider == null)
TargetProvider = ((IQueryable)query).Provider;
return query;
}
}
}
}
One difference with the standard EF6 LINQ queries is that this produces ObjectQuery<T> rather than DbQuery<T>, although except that ToString() does not return the generated SQL, I haven't noticed any difference in the further query building / execution. It seems to work, but use it with care and on your own risk.
You could write a function to build up your query, taking DbContext as a parameter.
public IQueryable<T> MyQuery(DbContext<T> db)
{
return db.Table
.Where(p => p.reallycomplex)
....
...
.OrderBy(p => p.manythings);
}
I've done this many times and it works well.
Now it's easy to make queries with two different contexts:
IQueryable<T> q1 = MyQuery(dbContext1);
IQueryable<T> q2 = MyQuery(dbContext2);
If your concern was the execution time taken to build the IQueryable objects, then my only suggestion is don't worry about it.
So you have an IQueryable<T> that will be performed on DbContext A as soon as the query is executed and you want the same query to run on DbContext B when the query is executed.
For this you'll have to understand the difference between an IEnumerable<T> and an IQueryable<T>.
An IEnumerable<T> holds all code to enumerate over the elements that the enumerable represents. The enumeration starts when GetEnumerator and MoveNext are called. This can be done explicitly. However it is usually done implicitly by functions like foreach, ToList, FirstOrDefault, etc.
An IQueryable does not hold the code to enumerate, it holds an Expression and a Provider. The Provider knows who will execute the query, and it knows how to translate the Expression into the language that is understood by the query executioner.
Due to this separation, it is possible to let the same Expression be executed by different data sources. They don't even have to be of the same type: one data source can be a database management system that understands SQL, the other one could be a comma separated file.
As long as you concatenate Linq statements that return an IQueryable, the query is not executed, only the Expression is changed.
As soon as enumeration starts, either by calling GetEnumerator / MoveNext, or by using foreach or one of the LINQ functions that do not return an IQueryable, the Provider will translate the Expression into the language the the data source understands and communicates with the data source to execute the query. The result of the query is an IEnumerable, which can be enumerated as if all data was in local code.
Some Providers are smart and use some buffering, so that not all data is transferred to local memory, but only part of the data. New data is queried when needed. So if you do a foreach in a database with a zillion elements, only the first few (thousands) elements are queried. More data is queried if your foreach runs out of fetched data.
So you already have one IQueryable<T>, therefore you have an Expression a Provider and an ElementType. You want the same Expression / ElementType to be executed by a differentProvider. You even want to change theExpression` slightly before you execute it.
Therefore you need to be able to create an object that implements IQueryable<T> and you want to be able to set the Expression, ElementType and a Provider
class MyQueryable<T> : IQueryable<T>
{
public type ElementType {get; set;}
public Expression Expression {get; set;}
public Provider Provider {get; set;}
}
IQueryable<T> queryOnDbContextA= dbCotextA ...
IQueryable<T> setInDbContextB = dbContextB.Set<T>();
IQueryable<T> queryOnDbContextB = new MyQueryable<T>()
{
ElementType = queryOnDbContextA.ElementType,
Expression = queryOnDbContextB.Expression,
Provider = setInDbContextB.Provider,
}
If desired you can adjust the query on the other context before executing it:
var getPageOnContextB = queryOnDbContextB
.Skip(...)
.Take(...);
Both queries are still not executed yet. Execute them:
var countA = await queryOnContextA.CountAsync();
var fetchedPageContextB = await getPageOnContextB.ToListAsync();
I need to build about 30 different administration pages to add/edit/delete records from 30 different tables. I could obviously spend the time creating 30 unique pages, to query each table, but I'm curious if there's a way to simply create a single, dynamic page that queries a single, dynamic linq query. This linq query then returns all fields & records from a specified table.
I've seen examples of dynamic linq similar to this one (http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx), but that still requires hardcoding the table name into the query. I'd like to do a select all similar to this, where I pass in the name of the table (i.e. "Products", "Orders", etc), and then somehow query that table:
private List<tableName> MyDynamicQuery(string tableName)
{
IEnumerable<tableName> dynamicList;
using (MyEntities db = _conn.GetContext())
{
dynamicList = (from q in db.<tableName>
select q).ToList();
}
return dynamicList;
}
Is something like this even possible to do?
Thanks
Instead of using table names, why don't you pass in a selector? It would look something like this:
private List<T> GetData<T>(Func<MyEntities, IEnumerable<T>> selector)
{
using (MyEntities db = _conn.GetContext())
{
return selector(db).ToList();
}
}
You'd use it like so:
var orders = GetData(db => db.Orders);
You could use entity framework and do this:
dynamiclist = this.datacontext.Set<T>().ToList(); // where T is the Type, represents the table in EF
I have Fluent NHibernate Linq queries where I check values based on run time arrays. A basic example would be something like:
var array = [1,2,3,4,5,6];
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<MyObject>().Where(x => array.Contains(x.CompareVal)).ToList();
}
I would expect the generated SQL statement to look something like this:
SELECT CompareVal, Column1, Column2
FROM MyObject
WHERE CompareVal IN (1,2,3,4,5,6)
However, what I'm finding instead is that the generated SQL statement simply emits the WHERE clause (proven by watching in Profiler) and selects the entire table, and then seems to run the filter in memory once it gets all the data back.
Something to note - I have a Generic Repository class that all of these calls are funneled through. The Query method is as follows:
public IList<T> Query(Func<T, bool> criteria)
{
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<T>().Where(criteria).ToList();
}
}
Obviously this (lack of a where clause) is not acceptable in a table with a large amount of data. What can I do to force NHibernate to generate the query correctly with the WHERE clause and still keep a generic pattern for repositories?
Does it make a difference if you change your Query method to the following ?
public IList<T> Query(Expression<Func<T, bool>> criteria)
{
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<T>().Where(criteria).ToList();
}
}
This is how I usually proceed with a generic Query :
public List<TOut> GetEntitiesLinq<TIn,TOut>(Expression<Func<IQueryable<TIn>,IQueryable<TOut>>> myFunc)
{
var t = (myFunc.Compile())(_session.Query<TIn>()) ;
return t.ToList();
}
Then how I would use it in your case :
var myObjList = myQueryManager.GetEntitiesLinq<MyObject,MyObject>(x=>x.Where(myObj => array.Contains(myObj.CompareVal)));
Hope this will help
Use Any:
return session.Query<MyObject>().Where(x => array.Any(y => y == x.CompareVal)).ToList();
Your repository pattern (using plain Func) automatically materializes your query to list, if you want something to be deferredly executed, use IQueryable, don't use Func only
Something to note - I have a Generic Repository class that all of
these calls are funneled through. The Query method is as follows:
public IList<T> Query(Func<T, bool> criteria)
{
using (var session = SessionProvider.SessionFactory.OpenSession())
{
return session.Query<T>().Where(criteria).ToList();
}
}
Your repository just mimic what is already provided out of the box by NHibernate
Can you use QueryOver and WhereRestrictionOn instead?
session.QueryOver<MyObject>().WhereRestrictionOn(o => o.CompareVal).IsIn(array).List();
I don't know Linq2Sql so well yet and I was wondering if there is a trick for this probably common MVVM scenario. I have Linq2Sql data context containing Domain models, but I am fetching data for my customized ViewModel object from it.
var query = from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
So i want my ViewModel to inculde both CurrencyId from domain object and the CurrencyText from related table to show it nicely in the View.
This code works great. It generates one DB call with join to fetch the CurrencyText. But the model is simplified, real one has many more fields. I want to make the code reusable because I have many different queries, that returns the same ViewModel. Now every minor change to OrderViewModel requires lots of maintainance.
So I moved the code to OrderViewModel itself as a constructor.
public OrderViewModel(Table_Order ord)
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
}
And call it like this.
var query = from ord in ctx.Table_Orders
select new OrderViewModel(ord);
The Problem: The join is gone DB query is no more optimised. Now I get 1+N calls to database to fetch CurrencyText for every line.
Any comments are welcome. Maybe I have missed different great approach.
This is how far i could get on my own, to get the code reusability. I created a function that does the job and has multiple parameters. Then I need to explicitly pass it everything that has crossed the line of entity.
var query = ctx.Table_Orders.Select(m =>
newOrderViewModel(m, m.Currency.CurrencyText));
The DB call is again optimized. But it still does not feel like I am there yet! What tricks do You know for this case?
EDIT : The final solution
Thanks to a hint by #Muhammad Adeel Zahid I arrived at this solution.
I created an extension for IQueryable
public static class Mappers
{
public static IEnumerable<OrderViewModel> OrderViewModels(this IQueryable<Table_Order> q)
{
return from ord in q
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
}
Now i can do this to get all list
var orders = ctx.Table_Order.OrderViewModels().ToList();
or this to get a single item, or anything in between with Where(x => ..)
var order = ctx.Table_Order
.Where(x => x.OrderId == id).OrderViewModels().SingleOrDefault();
And that completely solves this question. The SQL generated is perfect and the code to translate objects is reusable. Approach like this should work with both LINQ to SQL and LINQ to Entities. (Not tested with the latter) Thank You again #Muhammad Adeel Zahid
Whenever we query the database, we mostly require either enumeration of objects (more than one records in db) or we want a single entity (one record in db). you can write your mapping code in method that returns enumeration for whole table like
public IEnumerable<OrderViewModel> GetAllOrders()
{
return from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
Now you may want to filter these records and return another enumeration for example on currencyID
public IEnumerable<OrderViewModel> GetOrdersByCurrency(int CurrencyID)
{
return GetAllOrders().Where(x=>x.CurrencyId == CurrencyID);
}
Now you may also want to find single record out of all these view models
public OrderViewModel GetOrder(int OrderID)
{
return GetAllOrders().SingleOrDefault(x=>x.OrderId == OrderID);
}
The beauty of IEnumerable is that it keeps adding conditions to query and does not execute it until it is needed. so your whole table will not be loaded unless you really want it and you have kept your code in single place. Now if there are any changes in ViewModel Mapping or in query itself, it has to be done in GetAllOrders() method, rest of code will stay unchanged
You can avoid the N+1 queries problem by having Linq2SQL eagerly load the referenced entites you need to construct your viewmodels. This way you can build one list of objects (and some referenced objects) and use it to construct everything. Have a look at this blog post.
One word of warning though: This technique (setting LoadOptions for the Linq2SQL data context) can only be done once per data context. If you need to perform a second query with a different eager loading configuration, you must re-initalize your data context. I automated this with a simple wrapper class around my context.
I'm running into a common need in my project to return collections of my model objects, plus a count of certain types of children within each, but I don't know if it is possible or how to model a "TotalCount" property in a Model class and populate it as part of on single Entity Framework query, preferably using LINQ queries. Is it possible to do this whilst being able to use the Entity Framework .Include("Object") and .Skip() and .Take()? I'm new to the Entity Framework so I may be missing tons of obvious stuff that can allow this...
I would like to be able to paginate on the dynamically constructed count properties as well. I'm thinking that the most scalable approach would be to store the counts as separate database properties and then simply query the count properties. But for cases where there are small row counts that I'm dealing with, I'd rather do the counts dynamically.
In a model like this:
Table: Class
Table: Professor
Table: Attendee
Table: ClassComment
I'd like to return a list of Class objects in the form of List, but I would also like the counts of Attendees and Class comments to be determined in a single query (LINQ preferred) and set in two Class properties called AttendeeCount and ClassCommentCount.
I have this thus far:
var query = from u in context.Classes
orderby tl.Name
select u;
List<Class> topics = ((ObjectQuery<Class>)query)
.Include("ClassComments")
.Skip(startRecord).Take(recordsToReturn).ToList();
Any suggestions or alternative query approaches that can still allow the use of .Include() and pagination would be much much appreciated, in order to produce a single database query, if at all possible. Thank you for any suggestions!
Try this:
public class ClassViewModel {
public Class Class { get; set; }
public int AttendeeCount { get; set; }
public int ClassCommentCount { get; set; }
}
var viewModel = context.Classes.Select(clas =>
new ClassViewModel {
Class = clas,
AttendeeCount = clas.ClassAttendes.Count,
ClassCommentCount = clas.ClassComments.Count}
).OrderBy(model => model.ClassCommentCount).Skip(startRecord).Take(recordsToReturn).ToList();
You don't have to include comments to get count.
It will not work this way. The easiest approach is to use projection into anonymous (or custom) non entity type. I would try something like this:
var query = context.Classes
.Include("ClassComments") // Only add this if you want eager loading of all realted comments
.OrderBy(c => c.Name)
.Skip(startRecord)
.Take(recordsToReturn)
.Select(c => new
{
Class = c,
AttendeeCount = c.Attendees.Count(),
ClassCommentCount = c.ClassComments.Count() // Not needed because you are loading all Class comments so you can call Count on loaded collection
});
The problem in your requirement are AttendeeCount and ClassCommentCount properties. You can't easily add them to your model because there is no corresponding column in database (unless you define one and in such case you don't need to manually count records). You can define them in partial Class implementation but in such case you can't use them in Linq-to-entities query.
The only way to map this in EF is to use DB view and create special read only entity to represent it in your applicaiton or to use DefiningQuery which is custom SQL command defined in SSDL instead of DB table or view.