In my C# code, I have 2 WHERE queries both of which I can call on an IQueryable and have the whole thing compiled down to SQL, and both of which have a great deal of common logic.
I believe this is NOT a duplication of this similar question:
Using Function in Select Clause of Entity Framework Query because in my scenario the function in question CAN be converted into SQL - EF just isn't realising that it can do so.
The queries are approximately:
public static IQueryable<Template> WhereIsOwnedByUser(this IQueryable<Template> set, User user)
{
return set.Where(temp =>
temp.Requests
.Where(req => req.WasSent)
.OrderByDescending(req => req.DueDate)
.Take(2)
.SelectMany(req => req.RequestRecipients.Select(reqRecip => reqRecip.Recipient.Id))
.Contains(user.Id));
}
AND
public static IQueryable<Template> WhereIsOwnedByUser(this IQueryable<DataReturn> set, User user)
{
return set.Where(ret=>
ret.Entity.Id == user.Entity.Id
&&
ret.Request.Template.Requests
.Where(req => req.WasSent)
.OrderByDescending(req => req.DueDate)
.Take(2)
.SelectMany(req => req.RequestRecipients.Select(reqRecip => reqRecip.Recipient.Id))
.Contains(user.Id));
}
So a basic BusinessLogic rule for "owns a template" and then a corollary of that for "owns DataReturn if company matches AND owns template"
As you can see, thinking only about the C#, these could easily be refactored as:
private static bool UserOwnsTemplate(User user, Template temp)
{
return temp.Requests
.Where(req => req.WasSent)
.OrderByDescending(req => req.DueDate)
.Take(2)
.SelectMany(req => req.RequestRecipients.Select(reqRecip => reqRecip.Recipient.Id))
.Contains(user.Id);
}
public static IQueryable<Template> WhereIsOwnedByUser(this IQueryable<Template> set, User user)
{
return set.Where(temp => UserOwnsTemplate(user, temp));
}
public static IQueryable<DataReturn> WhereIsOwnedByUser(this IQueryable<DataReturn> set, User user)
{
return set.Where(
ret =>
ret.Entity.Id == user.Entity.Id
&&
UserOwnsTemplate(user, ret.Request.Template)
);
}
Thus reducing the duplication (Yay!)
But then EF will complain that it doesn't know what to do with UserOwnsTemplate, despite the fact that it can handle the logic in SQL perfectly well.
AFAICT there is no nice way to solve this. I think my options are:
Turn UserOwnsTemplate into a UDF, a SQL function defined in the database.
But I can't create a UDF from a C# lamda, I have to define the SQL, which would be more hassle.
Assign the Expression<Func<Template,bool>> that UserOwnsTemplate defines as a variable, and then build relevant Expression<Func<DataReturn ,bool>> for the DataReturn version by hand using Expression.AndAlso to glue the two "clauses" together.
Meta-programming. Ughhh. I've done this before in another project and it was vile to do, and a nightmare to maintain.
Live with the duplication.
Likely what will happen unless SO can advise otherwise. ;)
Can anyone see any other options available?
Can I do anything to force EF into parsing the function into SQL? (the phrase "inling" comes to mind but I don't 100% know what I think I mean by that?)
Can anyone see a way to convert ret.Request.Template into an IQueryable so that I can just call the other WhereIsOwnedBy extension method on it?
Any other suggestions AT ALL?
You can keep your syntax and make it work but you will need to call an additional method on the outer IQueryable<>.
The trick is to manually replace the IQueryable<>.Expression with a copy in which you replace the function call by the corresponding Expression>.
So the idea is to do something like that:
public static class MyLinqExtensions
{
public static IQueryable<T> InlineFunctions<T>(this IQueryable<T> queryable)
{
var expression = TransformExpression(queryable.Expression);
return (IQueryable<T>)queryable.Provider.CreateQuery(expression);
}
private static Expression TransformExpression(System.Linq.Expressions.Expression expression)
{
var visitor = new InlineFunctionsExpressionVisitor();
return visitor.Visit(expression);
}
private class InlineFunctionsExpressionVisitor : System.Linq.Expressions.ExpressionVisitor
{
protected override System.Linq.Expressions.Expression VisitMethodCall(System.Linq.Expressions.MethodCallExpression methodCallExpression)
{
if (methodCallExpression.Method.IsStatic
&& methodCallExpression.Method.DeclaringType == typeof(MyDeclaringType)
&& methodCallExpression.Method.Name == "WhereIsOwnedByUser")
{
var setArgumentExpression = methodCallExpression.Arguments[0];
var userArgumentExpression = methodCallExpression.Arguments[1];
var methodInfo = ... // Get typeof(IQueryable<Template>).MethodInfo
var whereConditionExpression = ...// Build where condition and use userArgumentExpression
return Expression.MethodCallExpression(methodInfo, setArgumentExpression, whereConditionExpression);
}
return base.VisitMethodCall(methodCallExpression);
// Some ideas to make this more flexible:
// 1. Use an attribute to mark the functions that can be inlined [InlinableAttribute]
// 2. Define an Expression<Func<>> first to be able to get the Expression and substritute the function call with it:
// Expression<Func<IQueryable<Template>, User, IQueryable<Template>>> _whereIsOwnedByUser = (set, user) =>
// {
// return set.Where(temp => UserOwnsTemplate(user, temp));
// };
//
// public static IQueryable<Template> WhereIsOwnedByUser(this IQueryable<Template> set, User user)
// {
// // You should cache the compiled expression
// return _whereIsOwnedByUser.Compile().Invoke(set, user);
// }
//
}
}
}
And then you can do this:
public static IQueryable<DataReturn> WhereIsOwnedByUser(this IQueryable<DataReturn> set, User user)
{
return set.Where(
ret =>
ret.Entity.Id == user.Entity.Id
&&
UserOwnsTemplate(user, ret.Request.Template)
)
.InlineFunctions();
}
The problem is that your method becomes part of the expression tree and that EF cannot evaluate it. In principle, it is possible to evaluate parts of the expression tree before triggering the query. Have a look at Re-Linq: https://relinq.codeplex.com/ It has a class PartialEvaluatingExpressionTreeVisitor which can evaluate all partial expression trees, i.e. it will find your method, evaluate it, and inject the actual expression tree. This will come at a certain performance cost, but it may not be significant and you will have to gauge clean design versus performance.
Related
What methods can I use Find, FirstOrDefault, Any ....
public virtual void Add(T entity)
{
using (MovieAppContext _context = new MovieAppContext())
{
var record = _context.Set<T>().Find(entity); // does not work I got e
if (record == null)
{
_context.Set<T>().Add(entity);
_context.SaveChanges();
}
}
}
Given your scope here, DbSet<T>.Find<T>(params object[]) is probably not the method you should be using.
The following solution uses IQueryable<T>.Any<T>(Expression<Func<T, bool>>), a Linq Extension Method that accepts a Predicate\Function.
Code Sample using it
using (MovieAppContext _context = new MovieAppContext())
{
bool queryResult = _context.Set<T>().Any((e) => e.Name.Equals(entity.Name, StringComparison.OrdinalIgnoreCase));
if (queryResult)
{
_context.Set<T>().Add(entity);
_context.SaveChanges();
}
}
Here, we are defining an anonymous function delegate, Func<T, bool>, that is in turn being used to define a Linq Expression, Expression<Func<T, bool>>.
Note
I defined this function predicate expression to evaluate on the entity names (a made-up property of this type; that in this example portrays a unique constraint.)
Also, an alternative to using anonymous functions declaration for this expression is to declare it, then pass it.
That looks like the following:
...
Expression<Func<T, bool>> WhereEntityNameMatches = (e) => e.Name.Equals(entity.Name, StringComparison.OrdinalIgnoreCase));
bool queryResult = _context.Set<T>().Any(WhereEntityNameMatches);
...
Which may help out with reusability and scope considerations.
For consideration, An alternative here would be to not attempt the query lookup and check at all.
Instead, you would do a try-catch for DbUpdateException in the attempt to save changes after\following this Add operation.
try
{
_context.Set<T>().Add(entity);
_context.SaveChanges();
}
catch (DbUpdateException)
{
var isRaisedByUniqueIndexOrConstraint = new[] { "2601", "2627" }.Contains(ex.GetBaseException().Data["HelpLink.EvtID"]) ?? false;
if (!isRaisedByUniqueIndexOrConstraint)
throw;
// Could suppress and\or handle attempted duplicate record adding
}
This relies on those index\constraints existing, so you would first define a Unique Index on the column(s) (as well as your entity type using Data Annotations: IndexAttribute).
Note
Part of this answer\solution purposes getting the Error Number from ADO.NET SqlClient Base Exception to further scope and qualify the cause of exception. The Error Numbers, "2601", "2627", are hard-coded in this example.
I am trying to use a custom Function inside a Linq Select that is used with EF.
I want to project each item of tblMitarbeiter onto one tblMitarbeiterPersonalkostenstelleHistories that is valid ad the given date.
This should be done with an extension method so that I do not repeat myself ;)
I can only get it to work when used directly on the DbSet, but not inside a Select.
How can I teach EF to recognize my Method (3.) as if I would be writing it out (1.)?
void Main()
{
var date = DateTime.Now;
// 1. works, returns IEnumerable<tblMitarbeiterPersonalkostenstelleHistories>
tblMitarbeiters
.Select(m => m.tblMitarbeiterPersonalkostenstelleHistories.Where(p => p.ZuordnungGültigAb <= date).OrderByDescending(p => p.ZuordnungGültigAb).FirstOrDefault())
.Dump();
// 2. works, returns one tblMitarbeiterPersonalkostenstelleHistories
tblMitarbeiterPersonalkostenstelleHistories
.GetValidItemForDate(p => p.ZuordnungGültigAb, date)
.Dump();
// 3. throws NotSupportedException
tblMitarbeiters
.Select(m => m.tblMitarbeiterPersonalkostenstelleHistories.GetValidItemForDate(p => p.ZuordnungGültigAb, date))
.Dump();
// 4. throws NotSupportedException
tblMitarbeiters
.Select(m => m.tblMitarbeiterPersonalkostenstelleHistories.AsQueryable().GetValidItemForDate(p => p.ZuordnungGültigAb, date))
.Dump();
}
public static class QueryableExtensions
{
public static T GetValidItemForDate<T>(this IQueryable<T> source, Expression<Func<T, DateTime>> selector, DateTime date)
{
var dateAccessor = Expression.Lambda<Func<T, DateTime>>(Expression.Constant(date), selector.Parameters);
var lessThanOrEqual = Expression.LessThanOrEqual(selector.Body, dateAccessor.Body);
var lambda = Expression.Lambda<Func<T, bool>>(lessThanOrEqual, selector.Parameters);
return source.Where(lambda).OrderByDescending(selector).FirstOrDefault();
}
public static T GetValidItemForDate<T>(this IEnumerable<T> source, Func<T, DateTime> selector, DateTime date) =>
source.Where(i => selector(i) <= date).OrderByDescending(selector).FirstOrDefault();
}
You can, to some extent, split up complex LINQ expressions using LINQKit. If you'll excuse me, I'll use an example model that's less germanic:
public class Employee
{
public long Id { get; set; }
public virtual ICollection<EmployeeHistoryRecord> HistoryRecords { get; set; }
}
public class EmployeeHistoryRecord
{
public long Id { get; set; }
public DateTime ValidFrom { get; set; }
public long EmployeeId { get; set; }
public Employee Employee { get; set; }
}
If I understood your question correctly, it should be identical to yours where it matters.
When using LINQKit, and LINQ in general, you must understand that the only tool you have at your disposal when reusing query code, without using stored procedures, is breaking apart and stitching together expressions.
Your utility method would translate to something like this:
private static Expression<Func<IEnumerable<TItem>, TItem>> GetValidItemForDate<TItem>(
Expression<Func<TItem, DateTime>> dateSelector,
DateTime date)
{
return Linq.Expr((IEnumerable<TItem> items) =>
items.Where(it => dateSelector.Invoke(it) <= date)
.OrderByDescending(it => dateSelector.Invoke(it))
.FirstOrDefault())
.Expand();
}
What this method does is dynamically create an expression whose input is an IEnumerable<TItem> that returns a TITem. You can see it's pretty similar to the code you're extracting. A few things to note:
The source collection is not a parameter of the utility method, but of the expression returned.
You have to call the Invoke() extension method from LinqKit on any expressions you're "plugging into" this one.
You should call Expand() on the result if you used any Invoke()s inside it. This will make LINQKit replace the calls to Invoke() in the expression tree with the expression being invoked. (This isn't 100% necessary, but it makes it easier to fix errors when expansion fails for some reason. If you don't Expand() in every helper method, any error that happens during expansion will manifest in the method that does the expansion, and not in the method that actually contains the offending code.)
You then use this similarly, again using Invoke():
var db = new EmployeeHistoryContext();
var getValidItemForDate = GetValidItemForDate((EmployeeHistoryRecord cab) => cab.ValidFrom, DateTime.Now);
var historyRecords = db.Employees.AsExpandable().Select(emp => getValidItemForDate.Invoke(emp.HistoryRecords));
(I've only tested this code against an empty database, insofar that it doesn't make EntityFramework throw a NotSupportedException.)
Here, you should note:
The subexpression you're plugging into the one you're passing into Select() needs to be saved in a local variable, LINQKit doesn't support method calls during expansion.
You need to call AsExpandable() on the first IQueryable in the chain, so LINQKit gets to work its magic.
You're probably not going to be able to use extension method call syntax inside the expression like in your question.
All the subexpressions have to be determined before expansion occurs.
These limitations stem from the fact that what you're doing isn't really calling methods. You're building one ginormous expression from a bunch of smaller ones, but the resulting expression itself still has to be something that LINQ-to-Entities will understand. On the other hand, the input has to be something LINQKit will understand, and it only handles expressions of the form localVariable.Invoke(). Any dynamism has to be in the code outside this expression tree. Basically, it's doing the same as your solution 2, just using syntax more intuitive than building the expression tree programmatically.
Last, but not least: when doing this, do not go overboard. Complex EF queries are already really hard to debug when anything goes wrong, because you're not told where in your code the problem is. If the query was assembled dynamically from bits and pieces all over your codebase, debugging some errors (like the delightful "Unable to cast the type X to type Y") will easily become a nightmare.
(For future questions: I think it's usually a good idea when if you make a code sample from scratch, instead of using bits from your actual codebase. They might be overly domain-specific, and understanding the names might require some context you take for granted. Identifiers should ideally be simple English names everyone can understand. I can maybe speak enough German to interview for a job in it, but "Mitarbeiterpersonalkostenstellehistorie" is just hard to keep in my head and reason about when I haven't actually worked on your project long enough to be familiar with what it's supposed to mean.)
I'm currently attempting to run some unit tests on a query that is running through the Entity Framework. The query itself runs without any issues on the live version, but the unit tests are always failing.
I've narrowed this down to my usage of DbFunctions.TruncateTime, but I don't know of a way around this to get the unit tests to reflect what is happening on the live server.
Here is the method that I am using:
public System.Data.DataTable GetLinkedUsers(int parentUserId)
{
var today = DateTime.Now.Date;
var query = from up in DB.par_UserPlacement
where up.MentorId == mentorUserId
&& DbFunctions.TruncateTime(today) >= DbFunctions.TruncateTime(up.StartDate)
&& DbFunctions.TruncateTime(today) <= DbFunctions.TruncateTime(up.EndDate)
select new
{
up.UserPlacementId,
up.Users.UserId,
up.Users.FirstName,
up.Users.LastName,
up.Placements.PlacementId,
up.Placements.PlacementName,
up.StartDate,
up.EndDate,
};
query = query.OrderBy(up => up.EndDate);
return this.RunQueryToDataTable(query);
}
If I comment out the lines with DbFunctions in, the tests all pass (except for the ones that are checking that only valid results for a given date are run).
Is there a way I can provide a mocked version of DbFunctions.TruncateTime to use in these tests? Essentially it should just be returning Datetime.Date, but that isn't available in EF queries.
Edit: Here's the test that's failing that uses the date check:
[TestMethod]
public void CanOnlyGetCurrentLinkedUsers()
{
var up = new List<par_UserPlacement>
{
this.UserPlacementFactory(1, 2, 1), // Create a user placement that is current
this.UserPlacementFactory(1, 3, 2, false) // Create a user placement that is not current
}.AsQueryable();
var set = DLTestHelper.GetMockSet<par_UserPlacement>(up);
var context = DLTestHelper.Context;
context.Setup(c => c.par_UserPlacement).Returns(set.Object);
var getter = DLTestHelper.New<LinqUserGetLinkedUsersForParentUser>(context.Object);
var output = getter.GetLinkedUsers(1);
var users = new List<User>();
output.ProcessDataTable((DataRow row) => students.Add(new UserStudent(row)));
Assert.AreEqual(1, users.Count);
Assert.AreEqual(2, users[0].UserId);
}
Edit 2: This is the message and debug trace from the test in question:
Test Result: Failed
Message: Assert.AreEqual failed. Expected:<1>. Actual:<0>
Debug Trace: This function can only be invoked from LINQ to Entities
From what I've read, this is because there isn't a LINQ to Entities implementation of this method that could be used in this place for the Unit Test, although there is on the live version (as it's querying an SQL server).
I know I'm late to the game, but a very simple fix is to write your own method which uses the DbFunction attribute. Then use that function instead of DbFunctions.TruncateTime.
[DbFunction("Edm", "TruncateTime")]
public static DateTime? TruncateTime(DateTime? dateValue)
{
return dateValue?.Date;
}
Using this function will execute the EDM TruncateTime method when used by Linq to Entities and will run the provided code otherwise.
Thanks for all of the help everyone, I managed to track down a solution that worked for me after reading up on shims that qujck mentioned. After adding a fake assembly of EntityFramework, I was able to fix these tests by changing them to the following:
[TestMethod]
public void CanOnlyGetCurrentLinkedUsers()
{
using (ShimsContext.Create())
{
System.Data.Entity.Fakes.ShimDbFunctions.TruncateTimeNullableOfDateTime =
(DateTime? input) =>
{
return input.HasValue ? (DateTime?)input.Value.Date : null;
};
var up = new List<par_UserPlacement>
{
this.UserPlacementFactory(1, 2, 1), // Create a user placement that is current
this.UserPlacementFactory(1, 3, 2, false) // Create a user placement that is not current
}.AsQueryable();
var set = DLTestHelper.GetMockSet<par_UserPlacement>(up);
var context = DLTestHelper.Context;
context.Setup(c => c.par_UserPlacement).Returns(set.Object);
var getter = DLTestHelper.New<LinqUserGetLinkedUsersForParentUser>(context.Object);
var output = getter.GetLinkedUsers(1);
}
var users = new List<User>();
output.ProcessDataTable((DataRow row) => users.Add(new User(row)));
Assert.AreEqual(1, users.Count);
Assert.AreEqual(2, users[0].UserId);
}
There is a way to do it. Since unit testing of business logic is generally encouraged, and since it is perfectly OK for business logic to issue LINQ queries against application data, then it must be perfectly OK to unit test those LINQ queries.
Unfortunately, DbFunctions feature of Entity Framework kills our ability to unit test code that contains LINQ queries. Moreover, it is architecturally wrong to use DbFunctions in business logic, because it couples business logic layer to a specific persistence technology (which is a separate discussion).
Having said that, our goal is the ability to run LINQ query like this:
var orderIdsByDate = (
from o in repo.Orders
group o by o.PlacedAt.Date
// here we used DateTime.Date
// and **NOT** DbFunctions.TruncateTime
into g
orderby g.Key
select new { Date = g.Key, OrderIds = g.Select(x => x.Id) });
In unit test, this will boil down to LINQ-to-Objects running against a plain array of entities arranged in advance (for example). In a real run, it must work against a real ObjectContext of Entity Framework.
Here is a recipe of achieving it - although, it requires a few steps of yours. I'm cutting down a real working example:
Step 1. Wrap ObjectSet<T> inside our own implementation of IQueryable<T> in order to provide our own intercepting wrapper of IQueryProvider.
public class EntityRepository<T> : IQueryable<T> where T : class
{
private readonly ObjectSet<T> _objectSet;
private InterceptingQueryProvider _queryProvider = null;
public EntityRepository<T>(ObjectSet<T> objectSet)
{
_objectSet = objectSet;
}
IEnumerator<T> IEnumerable<T>.GetEnumerator()
{
return _objectSet.AsEnumerable().GetEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return _objectSet.AsEnumerable().GetEnumerator();
}
Type IQueryable.ElementType
{
get { return _objectSet.AsQueryable().ElementType; }
}
System.Linq.Expressions.Expression IQueryable.Expression
{
get { return _objectSet.AsQueryable().Expression; }
}
IQueryProvider IQueryable.Provider
{
get
{
if ( _queryProvider == null )
{
_queryProvider = new InterceptingQueryProvider(_objectSet.AsQueryable().Provider);
}
return _queryProvider;
}
}
// . . . . . you may want to include Insert(), Update(), and Delete() methods
}
Step 2. Implement the intercepting query provider, in my example it is a nested class inside EntityRepository<T>:
private class InterceptingQueryProvider : IQueryProvider
{
private readonly IQueryProvider _actualQueryProvider;
public InterceptingQueryProvider(IQueryProvider actualQueryProvider)
{
_actualQueryProvider = actualQueryProvider;
}
public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
{
var specializedExpression = QueryExpressionSpecializer.Specialize(expression);
return _actualQueryProvider.CreateQuery<TElement>(specializedExpression);
}
public IQueryable CreateQuery(Expression expression)
{
var specializedExpression = QueryExpressionSpecializer.Specialize(expression);
return _actualQueryProvider.CreateQuery(specializedExpression);
}
public TResult Execute<TResult>(Expression expression)
{
return _actualQueryProvider.Execute<TResult>(expression);
}
public object Execute(Expression expression)
{
return _actualQueryProvider.Execute(expression);
}
}
Step 3. Finally, implement a helper class named QueryExpressionSpecializer, which would replace DateTime.Date with DbFunctions.TruncateTime.
public static class QueryExpressionSpecializer
{
private static readonly MethodInfo _s_dbFunctions_TruncateTime_NullableOfDateTime =
GetMethodInfo<Expression<Func<DateTime?, DateTime?>>>(d => DbFunctions.TruncateTime(d));
private static readonly PropertyInfo _s_nullableOfDateTime_Value =
GetPropertyInfo<Expression<Func<DateTime?, DateTime>>>(d => d.Value);
public static Expression Specialize(Expression general)
{
var visitor = new SpecializingVisitor();
return visitor.Visit(general);
}
private static MethodInfo GetMethodInfo<TLambda>(TLambda lambda) where TLambda : LambdaExpression
{
return ((MethodCallExpression)lambda.Body).Method;
}
public static PropertyInfo GetPropertyInfo<TLambda>(TLambda lambda) where TLambda : LambdaExpression
{
return (PropertyInfo)((MemberExpression)lambda.Body).Member;
}
private class SpecializingVisitor : ExpressionVisitor
{
protected override Expression VisitMember(MemberExpression node)
{
if ( node.Expression.Type == typeof(DateTime?) && node.Member.Name == "Date" )
{
return Expression.Call(_s_dbFunctions_TruncateTime_NullableOfDateTime, node.Expression);
}
if ( node.Expression.Type == typeof(DateTime) && node.Member.Name == "Date" )
{
return Expression.Property(
Expression.Call(
_s_dbFunctions_TruncateTime_NullableOfDateTime,
Expression.Convert(
node.Expression,
typeof(DateTime?)
)
),
_s_nullableOfDateTime_Value
);
}
return base.VisitMember(node);
}
}
}
Of course, the above implementation of QueryExpressionSpecializer can be generalized to allow plugging in any number of additional conversions, allowing members of custom types to be used in LINQ queries, even though they are not known to Entity Framework.
Check out this answer: https://stackoverflow.com/a/14975425/1509728
To be honest, upon thinking about it I totally agree with the answer and generally follow the principle that my EF queries are tested against the database and only my application code is tested with Moq.
It looks like there is no elegant solution to using Moq for testing EF queries with your query above, while there are some hacky ideas out there. For example this one and the answer that follows it. Both seem like they could work for you.
Another approach to testing your queries would be one implemented on another project I worked on: Using VS out of box unit tests, each query (again refactored into its own method) test would be wrapped in a transaction scope. Then the project's test framework would take care of manually entering phony data into the db and the query would try to filter this phony data. At the end, the transaction is never completed so it is rolled back. Due to the nature of transaction scopes, this might not be an ideal scenario for a lot of projects. Most probably not on prod environments.
Otherwise if you must continue mocking functionality, you might want to consider other mocking frameworks.
Hmm, not sure but couldn't you do something like this?
context.Setup(s => DbFunctions.TruncateTime(It.IsAny<DateTime>()))
.Returns<DateTime?>(new Func<DateTime?,DateTime?>(
(x) => {
/* whatever modification is required here */
return x; //or return modified;
}));
since i hit the same problem recently, and opted for a simpler solution, wanted to post it here.. this solution requires no Shims, Mocking, nothing expansive etc.
Pass a 'useDbFunctions' boolean flag to your method with default value as true.
When your live code executes, your query will use DbFunctions and everything will work. Due to the default value, callers need not worry about it.
When your unit tests invoke the method to test, they can pass useDbFunctions: false.
In your method, you can make use the flag to compose your IQueryable..
if useDbFunctions is true, use the DbFunctions to add the predicate to the queryable.
if useDbFunctions is false, then skip the DbFunctions method call, and do an explicit C# equivalent solution.
This way, your unit tests will check almost 95% of your method in parity with live code. You still have the delta of "DbFunctions" vs. your equivalent code, but be diligent about it and the 95% will look like a lot of gain.
public System.Data.DataTable GetLinkedUsers(int parentUserId, bool useDbFunctions = true)
{
var today = DateTime.Now.Date;
var queryable = from up in DB.par_UserPlacement
where up.MentorId == mentorUserId;
if (useDbFunctions) // use the DbFunctions
{
queryable = queryable.Where(up =>
DbFunctions.TruncateTime(today) >= DbFunctions.TruncateTime(up.StartDate)
&& DbFunctions.TruncateTime(today) <= DbFunctions.TruncateTime(up.EndDate));
}
else
{
// do db-functions equivalent here using C# logic
// this is what the unit test path will invoke
queryable = queryable.Where(up => up.StartDate < today);
}
var query = from up in queryable
select new
{
up.UserPlacementId,
up.Users.UserId,
up.Users.FirstName,
up.Users.LastName,
up.Placements.PlacementId,
up.Placements.PlacementName,
up.StartDate,
up.EndDate,
};
query = query.OrderBy(up => up.EndDate);
return this.RunQueryToDataTable(query);
}
Unit tests will invoke the mthod as:
GetLinkedUsers(parentUserId: 10, useDbFunctions: false);
Because unit tests would have setup local DbContext entities, the C# logic/DateTime functions would work.
Use of Mocks ended sometime ago. Do not Mock, just connect to real DB. Regenerate/Seed DB on start of test.
If you still want to go ahead with mocks then create your own method as given below. IT changes behaviour runtime. When using real DB it uses DB functions, else this method. Replace DBfunctions method in code with this method
public static class CanTestDbFunctions
{
[System.Data.Entity.DbFunction("Edm", "TruncateTime")]
public static DateTime? TruncateTime(DateTime? dateValue)
{
...
}
}
This is the real function that is called. And remember, time cannot be removed from DateTime object, live with midnight or create a string equivalent.
I am using PredicateBuilder to build reusable expressions as return values of objects. For example:
public interface ISurveyEligibilityCriteria
{
Expression<Func<Client, bool>> GetEligibilityExpression();
}
I want to have automated tests that determine whether a particular expression is translateable into T-SQL by Entity Framework (ie that it doesn't throw a NotSupportedException while "executing"). I can't find anything on the internet - is this possible (seems like it should be)?
You can create a LINQ statement containing the expression and then check whether it can be translated without actually executing it:
var connString = #"server=x;database=x";
using(var db = new MyContext(connString))
{
// ToString() shows the generated SQL string.
var sql = db.Entities.Where(generatedExpression).ToString();
Assert.IsTrue(sql.StartsWith("SELECT");
}
In the Assert you can test anything you'd expect to be part of the generated SQL string, but of course if the expression can't be translated, the test will fail because e.g. a NotSupportedException is thrown.
You can wrap this up into a handy extension method:
public static class EntityFrameworkExtensions
{
public static void CompilePredicate<T>(this DbContext context, Expression<Func<T, bool>> predicate)
where T : class
{
context.Set<T>().Where(predicate).ToString();
}
}
Then in your test:
// act
Action act = () => context.CompilePredicate(predicate);
// assert
act.ShouldNotThrow();
A very simple solution is executing it:
using (var context = ...)
{
// The query will return null, but will be executed.
context.Clients.Where(GetEligibilityExpression())
.Where(() => false)
.SingleOrDefault();
}
In older versions of EF (or using ObjectContext) you could have tried "manually" compiling the query with CompiledQuery.Compile, but this isn't supported with DbContext.
I'm currently attempting to run some unit tests on a query that is running through the Entity Framework. The query itself runs without any issues on the live version, but the unit tests are always failing.
I've narrowed this down to my usage of DbFunctions.TruncateTime, but I don't know of a way around this to get the unit tests to reflect what is happening on the live server.
Here is the method that I am using:
public System.Data.DataTable GetLinkedUsers(int parentUserId)
{
var today = DateTime.Now.Date;
var query = from up in DB.par_UserPlacement
where up.MentorId == mentorUserId
&& DbFunctions.TruncateTime(today) >= DbFunctions.TruncateTime(up.StartDate)
&& DbFunctions.TruncateTime(today) <= DbFunctions.TruncateTime(up.EndDate)
select new
{
up.UserPlacementId,
up.Users.UserId,
up.Users.FirstName,
up.Users.LastName,
up.Placements.PlacementId,
up.Placements.PlacementName,
up.StartDate,
up.EndDate,
};
query = query.OrderBy(up => up.EndDate);
return this.RunQueryToDataTable(query);
}
If I comment out the lines with DbFunctions in, the tests all pass (except for the ones that are checking that only valid results for a given date are run).
Is there a way I can provide a mocked version of DbFunctions.TruncateTime to use in these tests? Essentially it should just be returning Datetime.Date, but that isn't available in EF queries.
Edit: Here's the test that's failing that uses the date check:
[TestMethod]
public void CanOnlyGetCurrentLinkedUsers()
{
var up = new List<par_UserPlacement>
{
this.UserPlacementFactory(1, 2, 1), // Create a user placement that is current
this.UserPlacementFactory(1, 3, 2, false) // Create a user placement that is not current
}.AsQueryable();
var set = DLTestHelper.GetMockSet<par_UserPlacement>(up);
var context = DLTestHelper.Context;
context.Setup(c => c.par_UserPlacement).Returns(set.Object);
var getter = DLTestHelper.New<LinqUserGetLinkedUsersForParentUser>(context.Object);
var output = getter.GetLinkedUsers(1);
var users = new List<User>();
output.ProcessDataTable((DataRow row) => students.Add(new UserStudent(row)));
Assert.AreEqual(1, users.Count);
Assert.AreEqual(2, users[0].UserId);
}
Edit 2: This is the message and debug trace from the test in question:
Test Result: Failed
Message: Assert.AreEqual failed. Expected:<1>. Actual:<0>
Debug Trace: This function can only be invoked from LINQ to Entities
From what I've read, this is because there isn't a LINQ to Entities implementation of this method that could be used in this place for the Unit Test, although there is on the live version (as it's querying an SQL server).
I know I'm late to the game, but a very simple fix is to write your own method which uses the DbFunction attribute. Then use that function instead of DbFunctions.TruncateTime.
[DbFunction("Edm", "TruncateTime")]
public static DateTime? TruncateTime(DateTime? dateValue)
{
return dateValue?.Date;
}
Using this function will execute the EDM TruncateTime method when used by Linq to Entities and will run the provided code otherwise.
Thanks for all of the help everyone, I managed to track down a solution that worked for me after reading up on shims that qujck mentioned. After adding a fake assembly of EntityFramework, I was able to fix these tests by changing them to the following:
[TestMethod]
public void CanOnlyGetCurrentLinkedUsers()
{
using (ShimsContext.Create())
{
System.Data.Entity.Fakes.ShimDbFunctions.TruncateTimeNullableOfDateTime =
(DateTime? input) =>
{
return input.HasValue ? (DateTime?)input.Value.Date : null;
};
var up = new List<par_UserPlacement>
{
this.UserPlacementFactory(1, 2, 1), // Create a user placement that is current
this.UserPlacementFactory(1, 3, 2, false) // Create a user placement that is not current
}.AsQueryable();
var set = DLTestHelper.GetMockSet<par_UserPlacement>(up);
var context = DLTestHelper.Context;
context.Setup(c => c.par_UserPlacement).Returns(set.Object);
var getter = DLTestHelper.New<LinqUserGetLinkedUsersForParentUser>(context.Object);
var output = getter.GetLinkedUsers(1);
}
var users = new List<User>();
output.ProcessDataTable((DataRow row) => users.Add(new User(row)));
Assert.AreEqual(1, users.Count);
Assert.AreEqual(2, users[0].UserId);
}
There is a way to do it. Since unit testing of business logic is generally encouraged, and since it is perfectly OK for business logic to issue LINQ queries against application data, then it must be perfectly OK to unit test those LINQ queries.
Unfortunately, DbFunctions feature of Entity Framework kills our ability to unit test code that contains LINQ queries. Moreover, it is architecturally wrong to use DbFunctions in business logic, because it couples business logic layer to a specific persistence technology (which is a separate discussion).
Having said that, our goal is the ability to run LINQ query like this:
var orderIdsByDate = (
from o in repo.Orders
group o by o.PlacedAt.Date
// here we used DateTime.Date
// and **NOT** DbFunctions.TruncateTime
into g
orderby g.Key
select new { Date = g.Key, OrderIds = g.Select(x => x.Id) });
In unit test, this will boil down to LINQ-to-Objects running against a plain array of entities arranged in advance (for example). In a real run, it must work against a real ObjectContext of Entity Framework.
Here is a recipe of achieving it - although, it requires a few steps of yours. I'm cutting down a real working example:
Step 1. Wrap ObjectSet<T> inside our own implementation of IQueryable<T> in order to provide our own intercepting wrapper of IQueryProvider.
public class EntityRepository<T> : IQueryable<T> where T : class
{
private readonly ObjectSet<T> _objectSet;
private InterceptingQueryProvider _queryProvider = null;
public EntityRepository<T>(ObjectSet<T> objectSet)
{
_objectSet = objectSet;
}
IEnumerator<T> IEnumerable<T>.GetEnumerator()
{
return _objectSet.AsEnumerable().GetEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return _objectSet.AsEnumerable().GetEnumerator();
}
Type IQueryable.ElementType
{
get { return _objectSet.AsQueryable().ElementType; }
}
System.Linq.Expressions.Expression IQueryable.Expression
{
get { return _objectSet.AsQueryable().Expression; }
}
IQueryProvider IQueryable.Provider
{
get
{
if ( _queryProvider == null )
{
_queryProvider = new InterceptingQueryProvider(_objectSet.AsQueryable().Provider);
}
return _queryProvider;
}
}
// . . . . . you may want to include Insert(), Update(), and Delete() methods
}
Step 2. Implement the intercepting query provider, in my example it is a nested class inside EntityRepository<T>:
private class InterceptingQueryProvider : IQueryProvider
{
private readonly IQueryProvider _actualQueryProvider;
public InterceptingQueryProvider(IQueryProvider actualQueryProvider)
{
_actualQueryProvider = actualQueryProvider;
}
public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
{
var specializedExpression = QueryExpressionSpecializer.Specialize(expression);
return _actualQueryProvider.CreateQuery<TElement>(specializedExpression);
}
public IQueryable CreateQuery(Expression expression)
{
var specializedExpression = QueryExpressionSpecializer.Specialize(expression);
return _actualQueryProvider.CreateQuery(specializedExpression);
}
public TResult Execute<TResult>(Expression expression)
{
return _actualQueryProvider.Execute<TResult>(expression);
}
public object Execute(Expression expression)
{
return _actualQueryProvider.Execute(expression);
}
}
Step 3. Finally, implement a helper class named QueryExpressionSpecializer, which would replace DateTime.Date with DbFunctions.TruncateTime.
public static class QueryExpressionSpecializer
{
private static readonly MethodInfo _s_dbFunctions_TruncateTime_NullableOfDateTime =
GetMethodInfo<Expression<Func<DateTime?, DateTime?>>>(d => DbFunctions.TruncateTime(d));
private static readonly PropertyInfo _s_nullableOfDateTime_Value =
GetPropertyInfo<Expression<Func<DateTime?, DateTime>>>(d => d.Value);
public static Expression Specialize(Expression general)
{
var visitor = new SpecializingVisitor();
return visitor.Visit(general);
}
private static MethodInfo GetMethodInfo<TLambda>(TLambda lambda) where TLambda : LambdaExpression
{
return ((MethodCallExpression)lambda.Body).Method;
}
public static PropertyInfo GetPropertyInfo<TLambda>(TLambda lambda) where TLambda : LambdaExpression
{
return (PropertyInfo)((MemberExpression)lambda.Body).Member;
}
private class SpecializingVisitor : ExpressionVisitor
{
protected override Expression VisitMember(MemberExpression node)
{
if ( node.Expression.Type == typeof(DateTime?) && node.Member.Name == "Date" )
{
return Expression.Call(_s_dbFunctions_TruncateTime_NullableOfDateTime, node.Expression);
}
if ( node.Expression.Type == typeof(DateTime) && node.Member.Name == "Date" )
{
return Expression.Property(
Expression.Call(
_s_dbFunctions_TruncateTime_NullableOfDateTime,
Expression.Convert(
node.Expression,
typeof(DateTime?)
)
),
_s_nullableOfDateTime_Value
);
}
return base.VisitMember(node);
}
}
}
Of course, the above implementation of QueryExpressionSpecializer can be generalized to allow plugging in any number of additional conversions, allowing members of custom types to be used in LINQ queries, even though they are not known to Entity Framework.
Check out this answer: https://stackoverflow.com/a/14975425/1509728
To be honest, upon thinking about it I totally agree with the answer and generally follow the principle that my EF queries are tested against the database and only my application code is tested with Moq.
It looks like there is no elegant solution to using Moq for testing EF queries with your query above, while there are some hacky ideas out there. For example this one and the answer that follows it. Both seem like they could work for you.
Another approach to testing your queries would be one implemented on another project I worked on: Using VS out of box unit tests, each query (again refactored into its own method) test would be wrapped in a transaction scope. Then the project's test framework would take care of manually entering phony data into the db and the query would try to filter this phony data. At the end, the transaction is never completed so it is rolled back. Due to the nature of transaction scopes, this might not be an ideal scenario for a lot of projects. Most probably not on prod environments.
Otherwise if you must continue mocking functionality, you might want to consider other mocking frameworks.
Hmm, not sure but couldn't you do something like this?
context.Setup(s => DbFunctions.TruncateTime(It.IsAny<DateTime>()))
.Returns<DateTime?>(new Func<DateTime?,DateTime?>(
(x) => {
/* whatever modification is required here */
return x; //or return modified;
}));
since i hit the same problem recently, and opted for a simpler solution, wanted to post it here.. this solution requires no Shims, Mocking, nothing expansive etc.
Pass a 'useDbFunctions' boolean flag to your method with default value as true.
When your live code executes, your query will use DbFunctions and everything will work. Due to the default value, callers need not worry about it.
When your unit tests invoke the method to test, they can pass useDbFunctions: false.
In your method, you can make use the flag to compose your IQueryable..
if useDbFunctions is true, use the DbFunctions to add the predicate to the queryable.
if useDbFunctions is false, then skip the DbFunctions method call, and do an explicit C# equivalent solution.
This way, your unit tests will check almost 95% of your method in parity with live code. You still have the delta of "DbFunctions" vs. your equivalent code, but be diligent about it and the 95% will look like a lot of gain.
public System.Data.DataTable GetLinkedUsers(int parentUserId, bool useDbFunctions = true)
{
var today = DateTime.Now.Date;
var queryable = from up in DB.par_UserPlacement
where up.MentorId == mentorUserId;
if (useDbFunctions) // use the DbFunctions
{
queryable = queryable.Where(up =>
DbFunctions.TruncateTime(today) >= DbFunctions.TruncateTime(up.StartDate)
&& DbFunctions.TruncateTime(today) <= DbFunctions.TruncateTime(up.EndDate));
}
else
{
// do db-functions equivalent here using C# logic
// this is what the unit test path will invoke
queryable = queryable.Where(up => up.StartDate < today);
}
var query = from up in queryable
select new
{
up.UserPlacementId,
up.Users.UserId,
up.Users.FirstName,
up.Users.LastName,
up.Placements.PlacementId,
up.Placements.PlacementName,
up.StartDate,
up.EndDate,
};
query = query.OrderBy(up => up.EndDate);
return this.RunQueryToDataTable(query);
}
Unit tests will invoke the mthod as:
GetLinkedUsers(parentUserId: 10, useDbFunctions: false);
Because unit tests would have setup local DbContext entities, the C# logic/DateTime functions would work.
Use of Mocks ended sometime ago. Do not Mock, just connect to real DB. Regenerate/Seed DB on start of test.
If you still want to go ahead with mocks then create your own method as given below. IT changes behaviour runtime. When using real DB it uses DB functions, else this method. Replace DBfunctions method in code with this method
public static class CanTestDbFunctions
{
[System.Data.Entity.DbFunction("Edm", "TruncateTime")]
public static DateTime? TruncateTime(DateTime? dateValue)
{
...
}
}
This is the real function that is called. And remember, time cannot be removed from DateTime object, live with midnight or create a string equivalent.