Consider a database with multiple tables built using Entity Framework code first. Each table contains a different type of object, but I wish to create a single generic query builder class for extensibility's sake. So far as a framework for this class I have a generic class as so intended to act as a wrapper for Linq to SQL:
public class DBQuerier<T>
where T : class
{
DbSet<T> relation;
public DBQuerier(DbSet<T> table)
{
relation = table;
}
public bool Exists(T toCheck);
public void Add(T toAdd);
public T (Get Dictionary<String, Object> fields);
public bool SubmitChanges();
public void Update(T toUpdate, Dictionary<String, Object> fields);
public void Delete(T toDelete);
}
My problem comes at the first hurdle when trying to check to see if a record exists as I cannot convert between generic type T and an object type that I am trying to work with. If I use base Linq:
public bool Exists(T toCheck)
{
return (from row in relation
where row.Equals(toCheck)
select row).Any();
}
A run-time exception occurs as SQL cannot work with anything but primitive types even if I implement IComparable and designate my own Equals that compares a single field. Lambda Expressions seem to come closer, but then I get problems again with SQL not being able to handle more than primitive types even though my understanding was that Expression.Equal forced it to use the class' comparable function:
public bool Exists(T toCheck)
{
ParameterExpression T1 = Expression.Parameter(typeof(myType), "T1");
ParameterExpression T2 = Expression.Parameter(typeof(myType), "T2");
BinaryExpression compare = Expression.Equal(T1, T2);
Func<T, T, bool> checker =
Expression.Lambda<Func<T, T, bool>>
(compare, new ParameterExpression[] { T1, T2 }).Compile();
return relation.Where(r => checker.Invoke(r, toCheck)).Any();
}
The expression tree was designed in mind so that later I could add a switch statement to build the query according to the type I was trying to look at.
My question is: Is there a much simpler / better way to do this (or fix what I've tried so far) as the only other options I can see are to write a class for each table (not as easy to extend) or check each record application side (potentially horrendously slow if you have to transfer the whole database!)? Apologies if I've made so very basic mistakes as I haven't worked with much of this for very long at all, thanks in advance!
Don't compile it. Func<T,bool> means "run this in memory" while Expression<Func<T,bool>> means "keep the logical idea of what this predicate is" which allows frameworks like entity framework to translate that into the query.
As a side note, I don't think that entity framework lets you do a.Equals(b) for querying, so you'll have to do a.Id == b.Id
Entity framework is unlikely to work with your custom linq, it's quite rigid in the commands that it supports. I am going to ramble a bit and it's pseudocode, but I found two solutions that worked for me.
I first used generics approach, where my generic database searcher would accept a Func<T, string> nameProperty to access the name I was going to query. EF has many overloads for accessing sets and properties so I could make this work, by passing in c => c.CatName and using that to access the property in a generic fashion. It was a bit messy though, so:
I later refactored this to use interfaces.
I have a function that performs a text search on any table/column you pass into the method.
I created an interface called INameSearchable which simply contains a property that will be the name property to search. I then extended my entity objects (they are partial classes) to implement INameSearchable. So I have an entity called Cat which has a CatName property. I used the interface to return CatName; as the Name property of the interface.
I can then create a generic Search method where T : INameSearchable and it will expose the Name property that my interface exposed. I then simply use that in my method to perform the query, eg. (Pseudocode from memory!)
doSearch(myContext.Cats);
and in the method
public IEnumerable<T> DoSearch<T>(IDbSet<T> mySet, string catName)
{
return mySet.Where(c => c.Name == catName);
}
And quite beautifully, it allows me to generically search anything.
I hope this helps.
If you want to use EntityFramework you have to use primitive types. The reasons is that your LINQ-expression is converted to a SQL-statement. SQL doesn't know anything about objects, IComparables, ...
If you don't need it to be in SQL, you first have to execute the query against SQL and then filter it in memory. You can do that with the methods you're currently using
Related
I'm running an EF statement where I need to calculate de deductibles. After long trying, I can't seem to add a custom function in a .Select() statement. Instead I'm trying to add the values after my .Select() statement.
The problem here is, in my CalculateDeductibles() I can't seem to add any values to item.Deductibles.
The GetDeductibles(item.RequestId) is a rather heavy funtion that does several extra queries, so I'm trying to prevent to convert my IQueryable to an IList object.
So there are actually 2 questions:
Can I have the GetDeductibles() function directly in my .Select() statement?
Can I somehow (with keeping an eye on performance) add the value after I did my .Select()
Code:
public IQueryable<ReinsuranceSlip> GetReinsuranceSlipsOverview(int userId, int companyId, string owner, string ownerCompany)
{
IQueryable<ReinsuranceSlip> model = null;
model = _context.Request
.Where(w => w.RequestGroup.ProgramData.MCContactId == userId)
.Select(x => new ReinsuranceSlip()
{
Id = x.Id,
RequestId = x.Id,
LocalPolicyNumber = x.LocalPolicyNumber,
BusinessLine = x.RequestGroup.ProgramData.BusinessLine.DisplayName,
BusinessLineId = x.RequestGroup.ProgramData.BusinessLine.Id,
ParentBroker = x.RequestGroup.ProgramData.Broker.Name,
LocalBroker = x.Broker.Name,
InceptionDate = x.InceptionDate,
RenewDate = x.RenewDate,
//Deductibles = CalculateDeductibles(x)
});
CalculateDeductibles(model);
return model;
}
private void CalculateDeductibles(IQueryable<ReinsuranceSlip> model)
{
//model.ForEach(m => m.Deductibles = GetDeductibles(m.RequestId));
foreach (var item in model)
{
item.Deductibles = GetDeductibles(item.RequestId);
}
}
Updated and Sorry for the first version of this answer. I didn't quite understand.
Answer 1: IQueryable is using to creating a complete SQL statement to call in SQL Server. So If you want to use IQueryable, your methods need to generate statements and return it. Your GetDetuctibles method get request Id argument but your queryable model object didn't collect any data from DB yet, and it didn't know x.Id value. Even more, your GetCarearDetuctiples get an argument so and with that argument generates a queryable object and after some calculations, it returns decimal. I mean yes you can use your methods in select statement but it's really complicated. You can use AsExpendable() LINQ method and re-write your methods return type Expression or Iqueryable.
For detailed info you should check. This:
Entity Navigation Property IQueryable cannot be translated into a store expression and this: http://www.albahari.com/nutshell/predicatebuilder.aspx
And you also should check this article to understand IQueryable interface: https://samueleresca.net/2015/03/the-difference-between-iqueryable-and-ienumerable/
Answer 2: You can use the IEnumerable interface instead IQueryable interface to achieve this. It will be easy to use in this case. You can make performance tests and improve your methods by time.
But if I were you, I'd consider using Stored Procedures for performance gain.
You'll have to understand the differences between an IEnumerable and an IQueryable.
An IEnumerable object holds everything to enumerate over the elements in the sequence that this object represents. You can ask for the first element, and once you've got it, you can repeatedly ask for the next element until there is no more next element.
An IQueryable works differently. An IQueryable holds an Expression and a Provider. The Expression is a generic description of what data should be selected. The Provider knows who has to execute the query (usually a database), and it knows how to translate the Expression into a format that the Provider understands.
There are two types of LINQ functions: the ones that return IQueryable<TResult> and the ones that return TResult. Functions form the first type do not execute the query, they will only change the expression. They use deferred execution. Functions of the second group will execute the query.
When the query must be executed, the Provider takes the Expression and tries to translate it into the format that the process that executes the query understand. If this process is a relational database management system this will usually be SQL.
This translation is the reason that you can't add your own functionality: the Expression must be translatable to SQL, and the only thing that your functions may do is call functions that will change the Expression to something that can be translated into SQL.
In fact, even entity framework does not support all LINQ functionalities. There is a list of Supported and Unsupported LINQ methods
Back to your questions
Can I have GetDeductibles directly in my query?
No you can't, unless you can make it thus simple that it will only change the Expression using only supporte LINQ methods. You'll have to write this in the format of an extension function. See extension methods demystified
Your GetDeductibles should have an IQueryable<TSource> as input, and return an IQueryable<TResult> as output:
static class QueryableExtensions
{
public static IQueryable<TResult> ToDeductibles<TSource, TResult, ...>(
this IQueryable<TSource> source,
... other input parameters, keySelectors, resultSelectors, etc)
{
IQueryable<TResult> result = source... // use only supported LINQ methods
return result;
}
}
If you really need to call other local functions, consider calling AsEnumerable just before calling the local functions. The advantage above ToList is that smart IQueryable providers, like the one in Entity Framework will not fetch all items but the items per page. So if you only need a few ones, you won't have transported all data to your local process. Make sure you throw away all data you don't need anymore before calling AsEnumerable, thus limiting the amount of transported data.
Can I somehow add the value after I did my .Select()
LINQ is meant to query data, not to change it. Before you can change the data you'll have to materialize it before changing it. In case of a database query, this means that you have a copy of the archived data, not the original. So if you make changes, you'll change the copies, not the originals.
When using entity framework, you'll have to fetch every item that you want to update / remove. Make sure you do not select values, but select the original items.
NOT:
var schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.Select(school = new
{
... // you get a copy of the values: fast, but not suitable for updates
})
.FirstOrDefault();
BUT:
School schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.FirstOrDefault()
Now your DbContext has the original School in its ChangeTracker. If you change the SchoolToUpdate, and call SaveChanges, your SchoolToUpdate is compared with the original School, to check if the School must be updated.
If you want, you can bypass this mechanism, by Attaching a new School directly to the ChangeTracker, or call a Stored procedure.
I am storing records in order to link 2 part records together in Orchard (CMS).
The records will have a reference to the 2 items they are linking.
There are several tables of records doing this, and they all implement the same interface.
//One example of a record implementing the common interface
//This relation links a member to a home
public class HomeToMemberRelationRecord : IRelationResultable
{
public virtual int Id { get; set; }
[Aggregate]
public virtual MemberPartRecord MemberPartRecord { get; set; }
[Aggregate]
public virtual HomePartRecord HomePartRecord { get; set; }
//The interface is implemented here
//...
}
I am trying to query these records in a generic way using the interface.
I have a generic method accepting the type and resolving the IRepository from that type.
The problem is the properties I want to use in the query differ for each record.
On the above record I may want to get the Member's from a Home, however another record might be linking a Dog to a DogHouse and I want to find all Dog's in the DogHouse.
The UI element fetching the relations will resolve the repository when needed:
Resolve<IRepository<T>>();
So there is no way to know if the query should be:
.Where(x => x.HomePartRecord.ContentItemRecord.Id == id)
//or
.Where(x => x.DogHousePartRecord.ContentItemRecord.Id == id)
So the interface implemented by the relation records must define how they themselves are to be queried.
I tried using a method to return the property I needed to query, but NHibernate did not like that and served me a NotSupportedException.
// Method in the interface
int GetId();
// Attempt to call above method from a query.
var repo = _services.WorkContext.Resolve<IRepository<T>>(); //Constraint: where T : IMemberSearchResultable, new()
var relations = repo.Table.Where(x => x.GetRelevantId() == id);
I tried using a property in my interface instead, but NHibernate would simply look for the property on the record in the database (It only exists in the model).
// Attempt to use a property in the interface instead of a method.
int RelevantId { get; set; } //Usage: .Where(x => x.RelevantId == id)
So I tried building an expression:
(Spoiler: This also failed.)
public static Expression<Func<T, bool>> GetExpression<T>(string propertyName, int filterValue)
{
var parameter = Expression.Parameter(typeof(T));
var property = Expression.Property(parameter, propertyName);
var method = typeof(int).GetMethod("Equals", new[] { typeof(int) });
var body = Expression.Call(property, method, Expression.Constant(filterValue));
return (Expression<Func<T, bool>>)Expression.Lambda(body, parameter);
}
public Expression<Func<T, bool>> TestGet<T>(int id)
{
return GetExpression<T>("MemberPartRecord.ContentItemRecord.Id", id);
}
I have tried a number of approaches to naming the property, but can not find one that works.
I can of course build the specific query I need in each case without using an interface. Which leads me to believe it should be possible to build the same query using the interface.
Am I building the expression wrong, is it the wrong approach or is the entire idea far fetched?
Changing mappings
If changing the mapping (and probably the interface altogether) is an option, you should try mapping your entities as inheriting from your common interface, as illustrated in inheritance mapping documentation. It should works without a base class.
But this would mean your interface would define some HomeBase property (typed as a base class or another common interface) which would the mapped one, and which would exists as such in your entities.
Then you would add some specialized ConcreteHome or DogHouse property on your entities, not mapped, and casting the HomeBase to the concrete home.
Probable issues
Beware of proxies, such setting would probably force you to use lazy="no-proxy" or lazy="false" on HomeBase property mapping.
Moreover, a post writes this is not supported in Fluent mapping (I use .hbm files).
And on top of that, if you need querying the related Home specific properties (not belonging to HomeBase), you would then have a new trouble.
Going farer into runtime type inspection
You may instead go farer with your current approach. But this will require quite some tinkering.
The aim would be to get a more capable helper:
public static Expression<Func<T, bool>> GetExpression<T, U>(
Expression<Func<I, BaseHome>> interfaceHomePropertySelector, U filterValue)
The helper implementation should get the interface property memberInfo, and then infer the corresponding member in T entity. From that, it would construct the adequate expression.
For performances reasons, the resulting inferring should then be cached for reducing runtime cost of subsequent uses.
Dirty inferring
You may infer the concrete property in T by checking all its properties and take the first which is compatible with the interface property, while not having the same name as the interface property. This requires having only a single Home property in each of your entities.
Harder inferring
You may go a quite more elaborated way to get the right property: runtime evaluation of what get actually called when accessing the interface property on an instance of T. For that, you would need to use the same proxyfying approach than NHibernate uses for handling lazy-loading.
Instantiate a new dummy instance of a T proxy, instrumented for calling a callback of yours at each property access. Access the interface property. Your callback should fires at least twice : at interface property access, then at the concrete property access the interface implementation should do.
So in your callback you would then have to inspect the call-stack to check in which case you are, and infer the concrete property of T.
I am posting what I have so far as an answer, but I hope someone out there has a better solution:
I have managed to get a working solution for my specific problem, but it's not a perfect solution as it does not allow you to use references in your query.
I can basically only refer to properties that are present on the C# record mapped to the database.
This is the method I am currently using:
public static Expression<Func<T, bool>> GetExpression<T, U>(string propertyName, U filterValue)
{
var parameter = Expression.Parameter(typeof(T));
var predicate = Expression.Lambda<Func<T, bool>>(
Expression.Equal(Expression.Property(parameter, propertyName),
Expression.Constant(filterValue)), parameter);
return predicate;
}
And I call it this way:
GetExpression<T, U>("MemberPartRecord", rec);
The significant difference here is that I am now querying the MemberPartRecord rather than trying to access MemberPartRecord.ContentItemRecord.Id
For my current needs this is enough, but only because I am in a position to provide the record I need for the query.
I will not accept this answer in the hopes that someone can provide a complete answer that allows the use of referenced records in the query.
I'm using a Generic Repository like pattern to fetch data. There are 100+ entities so creating a separate repository for each is not really an option. here are a few functions from the same class:
public int Count(Func<TEntity, bool> x=null)
{
return x == null ?
mgr.CTX.GetAll<TEntity>().Count() :
mgr.CTX.GetAll<TEntity>().Where(x).Count();
}
public TEntity One(Func<TEntity, bool> x)
{
return mgr.CTX.GetAll<TEntity>().Where(x).Take(1).FirstOrDefault();
}
public IQueryable<TEntity> All(Func<TEntity, bool> x=null)
{
return x == null ?
mgr.CTX.GetAll<TEntity>() :
mgr.CTX.GetAll<TEntity>().Where(x).AsQueryable<TEntity>();
}
The problem is no matter which function is call, the Sql profiler shows the same
Select [columns] from [table]
I suppose when using Take(1) or Count() or Where() the query should be made accordingly using Count(), Top or Where clauses of Select but these functions have absolutely no effects on query generation. Apparently, every operation seems to be performed in memory after fetching all the data from server.
Guide me if there is something wrong with the way I'm accessing it or this is the normal behavior of telerik?
I believe you are victim of a subtle difference between definitions of LINQ extension method - in-memory ones use Func<> while SQL-bound use Expression<> as parameter type.
My suggestion is to change All(Func<TEntity, bool> x=null) to All(Expression<Func<TEntity, bool>> x=null)
I'll let the code speak for itself:
public interface ISoftDeletable {
bool IsDeleted {get; set;}
}
public static class Extensions {
public IQueryable<T> Active<T>(this IQueryable<T> q) where T : ISoftDeletable {
return q.Where(t => !t.IsDeleted);
}
}
public partial class Thing : ISoftDeletable {
...
}
...
var query = from tc in db.ThingContainers
where tc.Things.Active().Any(t => t.SatisfiesOtherCondition)
select new { ... }; // throws System.NotSupportedException
The error is:
LINQ to Entities does not recognize the method 'System.Collections.Generic.IQueryable`1[Thing] ActiveThing' method, and this method cannot be translated into a store expression.
You get the idea: I'd like a fluent kind of way of expressing this so that for any ISoftDeletable I can add on a 'where' clause with a simple, reusable piece of code. The example here doesn't work because Linq2Entities doesn't know what to do with my Active() method.
The example I gave here is a simple one, but in my real code, the Active() extension contains a much more intricate set of conditions, and I don't want to be copying and pasting that all over my code.
Any suggestions?
You have two unrelated problems in the code. The first one is that Entity Framework cannot handle casts in expressions, which your extension method does.
The solution to this problem is to add a class restriction to your extension method. If you do not add that restriction, the expression is compiled to include a cast:
.Where (t => !((ISoftDeletable)t.IsDeleted))
The cast above confuses Entity Framework, so that is why you get a runtime error.
When the restriction is added, the expression becomes a simple property access:
.Where (t => !(t.IsDeleted))
This expression can be parsed just fine with entity framework.
The second problem is that you cannot apply user-defined extension methods in query syntax, but you can use them in the Fluent syntax:
db.ThingContainers.SelectMany(tc => tc.Things).Active()
.Any(t => t.SatisfiesOtherCondition); // this works
To see the problem we have to look at what the actual generated query will be:
db.ThingContainers
.Where(tc => tc.Things.Active().Any(t => t.StatisfiesOtherCondition))
.Select(tc => new { ... });
The Active() call is never executed, but is generated as an expression for EF to parse. Sure enough, EF does not know what to do with such a function, so it bails out.
An obvious workaround (although not always possible) is to start the query at the Things instead of the ThingContainers:
db.Things.Active().SelectMany(t => t.Container);
Another possible workaround is to use Model Defined Functions, but that is a more involved process. See this, this and this MSDN articles for more information.
While #felipe has earned the answer credit, I thought I would also post my own answer as an alternative, similar though it is:
var query = from tc in db.ThingContainers.Active() // ThingContainer is also ISoftDeletable!
join t in db.Things.Active() on tc.ID equals t.ThingContainerID into things
where things.Any(t => t.SatisfiesOtherCondition)
select new { ... };
This has the advantage of keeping the structure of the query more or less the same, though you do lose the fluency of the implicit relationship between ThingContainer and Thing. In my case, the trade of works out that it's better to specify the relationship explicitly rather than having to specify the Active() criteria explicitly.
I'm trying to implement a generic class that will interact with a generic repository, and all is fine except for when I have to deal with getting objects out of the repository.
I'm going to have a virtual method in the generic class which will receive an int and I want to use that int to form a query to the repository that gets objects by their primary key. I have a feeling I need to work with the EntityKey property in EF, but not too sure how.
Anyway, here's what I'm trying to do in code, I hope someone will have suggestions on how to accomplish what I want:
public virtual T Get(int PrimaryKey) {
this.Repository.Select(
t =>
(t.PRIMARYKEY == PrimaryKey)).Single();
}
I want to extend this class with more specialized classes, but since most of them only get their objects by querying the PK, it makes since to me to have a base method that can do it.
Thanks in advance for any suggestions!
UPDATE
So, here's where I've gotten with reflection, and I doubt its the proper way, but it somewhat works... I'm getting a NotSupportedException with the message LINQ to Entities does not recognize the method 'System.Object GetValue(System.Object, System.Object[])' method, and this method cannot be translated into a store expression.. Although I understand what it says and why it's saying, I'm not sure how to overcome it when my code looks like this:
private readonly string TEntityName = typeof(T).Name;
public virtual T Get(
int PrimaryKey) {
return this.Repository.Select(
t =>
(((int)t.GetType().GetProperties().Single(
p =>
(p.Name == (this.TEntityName + "Id"))).GetValue(t, null)) == PrimaryKey)).Single();
}
Hoping that someone who knows how to use reflection, unlike me, can point me in the right direction. Thanks!
Retrieving an entity by a PK using EF requires an expression/predicate, like this:
Expression<Func<Order,bool>> predicate = x => x.OrderId == 1;
return ctx.Orders.Single(predicate);
There is no easy way (short of reflection or expression tree creation) to be able to dynamically create this predicate.
What you could do is accept the predicate as a parameter:
public virtual T Get(Expression<Func<T,bool>> predicate) {
this.Repository.Select(predicate).Single();
}
Also make sure you put some generic constraints on T (either at the class/method level).
http://msdn.microsoft.com/en-us/library/bb738961.aspx is way to go. Then use http://msdn.microsoft.com/en-us/library/bb738607.aspx, spend some time in debugger and your misson is completed.