I have an interface called ICatalog as shown below where each ICatalog has a name and a method that will return items based on a Predicate<Item> function.
public interface ICatalog
{
string Name { get; }
IEnumerable<Item> GetItems(Predicate<Item> predicate);
}
A specific implementation of a catalog may be linked to catalogs in various format such as XML, or a SQL database.
With an XML catalog I end up deserializing the entire XML file into memory, so testing each item with the predicate function does does not add a whole lot more overhead as it's already in memory.
Yet with the SQL implementation I'd rather not retrieve the entire contents of the database into memory, and then filter the items with the predicate function. Instead I'd want to find a way to somehow pass the predicate to the SQL server, or somehow convert it to a SQL query.
This seems like a problem that can be solved with Linq, but I'm pretty new to it. Should my interface return IQueryable instead? I'm not concerned right now with how to actually implement a SQL version of my ICatalog. I just want to make sure my interface will allow for it in the future.
Rob has indicated how you might do this (although a more classic LINQ approach might take Expression<Func<Item,bool>>, and possbily return IQueryable<IFamily>).
The good news is that if you want to use the predicate with LINQ-to-Objects (for your xml scenario) you can then just use:
Predicate<Item> func = predicate.Compile();
or (for the other signature):
Func<Item,bool> func = predicate.Compile();
and you have a delegate (func) to test your objects with.
The problem though, is that this is a nightmare to unit test - you can only really integration test it.
The problem is that you can't reliably mock (with LINQ-to-Objects) anything involving complex data-stores; for example, the following will work fine in your unit tests but won't work "for real" against a database:
var foo = GetItems(x => SomeMagicFunction(x.Name));
static bool SomeMagicFunction(string name) { return name.Length > 3; } // why not
The problem is that only some operations can be translated to TSQL. You get the same problem with IQueryable<T> - for example, EF and LINQ-to-SQL support different operations on a query; even just First() behaves differently (EF demands you explicitly order it first, LINQ-to-SQL doesn't).
So in summary:
it can work
but think carefully whether you want to do this; a more classic black box repository / service interface may be more testable
You don't need to go all the way and create an IQueryable implementation
If you declare your GetItems method as:
IEnumerable<IFamily> GetItems(Expression<Predicate<Item>> predicate);
Then your implementing class can inspect the Expression to determine what is being asked.
Have a read of the IQueryable article though, because it explains how to build a expression tree visitor, which you'll need to build a simple version of.
Related
The goal here is to get some WCF Data Services query scenarios accurately reflected in code under test.
Following the same pattern for mocking EF functionality detailed here, I'm mocking a DataServiceQuery wrapper class and delegating to the IQueryable returned by calling .AsQueryable() on my test data.
I want to override the LINQ to objects behavior for a specific equality expression (if it matters, when comparing two byte arrays). I want to compare the values in the arrays instead of the references (e.g. using SequenceEqual).
The way I can see accomplishing this is to create a provider where I visit the expression, pull out the appropriate values, compose a new expression, and pass that along to the LINQ to objects provider. That's a lot of work. Short of doing this, is there an easier way to get this done or will I have to resort to parsing expressions?
You can try to create your own extension methods that take a more specific data type. Something along this line :
static IEnumerable<byte[]> Where(this IEnumerable<byte[]> source, Func<byte[], bool> predicate)
{
}
I'm using Code First to map classes to an existing database. I need a way to unit test these mappings, which are a mix of convention-based, attribute-based, and fluent-api.
To unit test, I need to confirm that properties of the classes map to the correct table and column names in the database. This test needs to be performed against the context, and should cover all configuration options for code first.
At a very high level, I'd be looking to assert something like (pseudo-code):
Assert.IsTrue(context.TableFor<Widget>().IsNamed("tbl_Widget"));
Assert.IsTrue(context.ColumnFor<Widget>(w => w.Property).IsNamed("WidgetProperty"));
Another idea to consider is using Linq and ToString().
For eaxample this :
context.Widget.Select(c => c.Property).ToString()
Will result in this for SQL Server Provider :
"SELECT [Var_3].[WidgetProperty] AS [WidgetProperty] FROM [dbo].[Widget]..."
Now we could hide it all in some Extension method that and parses resulting SQL it would look almost like Your pseudo-code :
Assert.IsTrue(context.Widgets.GetSqlColumnNameFor(w => w.Property).IsNamed("WidgetProperty"));
Draft for extension :
public string GetSqlColumnNameFor<TSource>(this DbSet<T> source, Expression<Func<TSource, TResult>> selector)
{
var sql = source.Select(selector).ToString();
var columnName = sql... // TODO : Some regex parsing
return
columnName;
}
Similary we could create GetSqlTableNameFor().
UPDATE : I decided to look for some dedicates SQL Parsers, so this solution is more generic, obviously there is such a thing for .NET :
http://www.dpriver.com/blog/list-of-demos-illustrate-how-to-use-general-sql-parser/generate-internal-query-parse-tree-in-xml-for-further-processing/
The only way I can think of to cover every possible option would be to use the Entity Framework Power Tools to pre-compile the views of your DbContext, and probably use a combination of reflection on that generated type and RegEx on the generated code itself to verify everything maps the way you want it to. Sounds pretty painful to me.
Another thing that comes to mind is creating a facade around DbModelBuilder to intercept and check everything that passes through it, but I don't know if that would handle the convention-based stuff. Also sounds painful.
As a less-complete, but much easier alternative, you can probably knock out a large portion of this by switching to attribute-based mapping wherever possible. This would allow you to create a base test class, say, ModelTesting<TEntity>, which includes a few test methods that use reflection to verify that TEntity has:
A single TableAttribute.
Each property has a single ColumnAttribute or NotMappedAttribute.
At least one property with a KeyAttribute.
Each property type maps to a compatible database type.
You could even go so far as to enforce a naming convention based on the names of the properties and class (with a caveat for table-per-hierarchy types). It would also be possible to check the foreign key mappings as well. That's a write-once base class you can derive from once for each of your model types and catch the majority of your mistakes (well, it catches the majority of mine, anyway).
Anything that can't be represented by attributes, like TPH inheritance and such, becomes a little harder. An integration test that fires up the DbContext and does a FirstOrDefault on Set<TEntity>() would probably cover most of those bases, assuming your DbContext isn't generating your database for you.
If you wrote a method
public static string ToMappingString(this Widget obj)
Then you could easily testing this via approval tests ( www.approvaltests.com or nuget)
There's a video here: http://www.youtube.com/watch?v=vKLUycNLhgc
However, if you are looking to test "My objects save and retrive themselves"
Then this is a perfect place of "Theory Based Testing"
Theory based testing
Most unit test take the form of
Given A,B expect C
Theory based testing is
Given A,B expect Theory
The beauty of this is there is no need to worry about which particular form A & B take since you don't need to know C, so any random generator will work.
Example 1: Testing Add and Subtract methods
Normally you would have stuff like
Assert.AreEqual(5, Add(2,3));
Assert.AreEqual(9, Add(10,-1));
Assert.AreEqual(10, Add(5,5));
Assert.AreEqual(7, Subtract(10,3));
However if you wrote a Theory Test it would look like
for(int i = 1; i < 100; i++)
{
int a = random.Next();
int b = random.Next();
Assert.AreEqual(a, Subtract(Add(a,b),b, string.Format("Failed for [a,b] = [{0},{1}], a,b));
}
Now that you understand Theory based testing, the theory you are trying to test is
Given Model A
When A is stored to the database, and retrieved the resulting object is equal to A
I have a Class / API that uses an IQueryable<FirstClass> data source however I wish to expose an IQueryable<SecondClass>, where SecondClass is a wrapper class for FirstClass that exposes nearly identical properties, however for various reasons needs to inherit from an unrelated base class. For example:
// My API
void IQueryable<SecondClass> GetCurrentRecords()
{
return from row in dataSource
/* Linq query */
select new SecondClass(row);
}
// User of my API
var results = GetCurrentRecords().Where(row => row.Owner = "Mike");
Now I can make the above compile simply by using AsQueryable however I want to expose a "true" IQueryable that efficiently queries the database based on the API users query.
I know that this isn't trivial (my wrapper IQueryable implementation needs to understand the relationship between the properties of SecondClass and FirstClass), and that it has nothing to do with the Select function, but it seems like it should be possible.
How do I do this?
Note: I know that instead my API could just expose FirstClass along with a helper method to convert FirstClass to SecondClass for when the API user is "done" creating their query, but it feels messy and I don't like the idea of exposing my generated classes in this way. Also I'd like to know how to do the above anyway just from a purely academic standpoint.
Probably, you should return not an IQueriable, but Expression. Then you will be able to modify expression and let LINQ generate a query from a final Expression object. Example is here: http://msdn.microsoft.com/en-us/library/bb882637.aspx
I have a generic method that exists in EntityRepository that gets entities by Name, which is defined as follows:
public IEnumerable<T> GetEntitiesByName<T>(string searchExpression)
where T : class, ISearchableEntity, new()
{
return _session.CreateCriteria<T>()
.Add(LambdaSubquery.Property<Fund>(x => x.Id)
.AddNameSearchCriteria<T>(searchExpression)
.List<T>();
}
There are a number of services that use this method, and everything was fine.. but I now have a requirement that some services require more specific search capabilites than is available in this method.
So I'm wonder if it's possible for me to pass in a DetachedCriteria as an Expression (or something) that I could plug into the query? Each service that needs different functionality could pass in the expression it needs to.
I'm not hugely familiar with NHibernate or Expressions, so I might be out of my depth here. I'm mostly trying to get a feel for what's possible. Otherwise I'll have to abandon the generic method in favour of more specific Repository methods for each service. Thanks.
I posted a code sample in my answer to a somewhat similar question which might help you. However, it is only a rough beginning. If you really want to translate full LINQ expressions into NHibernate and have some semblance of completeness, I would recommend to search for existing solutions first. A quick Google search found me this:
Implementing LINQ for NHibernate
I'm working with a client who wants to mix LINQ to SQL with their in-house DAL. Ultimately they want to be able to query their layer using typical LINQ syntax. The point where this gets tricky is that they build their queries dynamically. So ultimately what I want is to be able to take a LINQ query, pull it apart and be able to inspect the pieces to pull the correct objects out, but I don't really want to build a piece to translate the 'where' expression into SQL. Is this something I can just generate using Microsoft code? Or is there an easier way to do this?
(you mean just LINQ, not really LINQ-to-SQL)
Sure, you can do it - but it is massive amounts of work. Here's how; I recommend "don't". You could also look at the source code for DbLinq - see how they do it.
If you just want Where, it is a bit easier - but as soon as you start getting joins, groupings, etc - it will be very hard to do.
Here's just Where support on a custom LINQ implemention (not a fully queryable provider, but enough to get LINQ with Where working):
using System;
using System.Collections.Generic;
using System.Linq.Expressions;
using System.Reflection;
namespace YourLibrary
{
public static class MyLinq
{
public static IEnumerable<T> Where<T>(
this IMyDal<T> dal,
Expression<Func<T, bool>> predicate)
{
BinaryExpression be = predicate.Body as BinaryExpression;
var me = be.Left as MemberExpression;
if(me == null) throw new InvalidOperationException("don't be silly");
if(me.Expression != predicate.Parameters[0]) throw new InvalidOperationException("direct properties only, please!");
string member = me.Member.Name;
object value;
switch (be.Right.NodeType)
{
case ExpressionType.Constant:
value = ((ConstantExpression)be.Right).Value;
break;
case ExpressionType.MemberAccess:
var constMemberAccess = ((MemberExpression)be.Right);
var capture = ((ConstantExpression)constMemberAccess.Expression).Value;
switch (constMemberAccess.Member.MemberType)
{
case MemberTypes.Field:
value = ((FieldInfo)constMemberAccess.Member).GetValue(capture);
break;
case MemberTypes.Property:
value = ((PropertyInfo)constMemberAccess.Member).GetValue(capture, null);
break;
default:
throw new InvalidOperationException("simple captures only, please");
}
break;
default:
throw new InvalidOperationException("more complexity");
}
return dal.Find(member, value);
}
}
public interface IMyDal<T>
{
IEnumerable<T> Find(string member, object value);
}
}
namespace MyCode
{
using YourLibrary;
static class Program
{
class Customer {
public string Name { get; set; }
public int Id { get; set; }
}
class CustomerDal : IMyDal<Customer>
{
public IEnumerable<Customer> Find(string member, object value)
{
Console.WriteLine("Your code here: " + member + " = " + value);
return new Customer[0];
}
}
static void Main()
{
var dal = new CustomerDal();
var qry = from cust in dal
where cust.Name == "abc"
select cust;
int id = int.Parse("123");
var qry2 = from cust in dal
where cust.Id == id // capture
select cust;
}
}
}
Technically if your DAL exposes IQueryable<T> instead of IEnumerable<T> you can also implement a IQueryProvider and do exactly what you describe. However, this is not for the faint of heart.
But if you expose the LINQ to SQL tables themselves in the DAL, they will do exactly this for you. There is a (big) risk though since you'll be handling the client code total control over how to express SQL queries, and the usual result is some complex query that joins everything and slaps pagination a top of it with less than spectacular run time performance.
I think you should consider carefully what is actually needed from the DAL and expose only that.
I just read an interesting article on Expression Trees, LINQ to SQL uses these to translate the query into SQL and send it over the wire.
Maybe that's something you could use?
Just some though. I know some language support building a string that can be execute in the code itself. I never tried it with .Net, but this is common in functional languages like LISP. Since .Net support lambdas, maybe this is possible.
Since F# is coming to .Net soon, maybe it will possible if it is not right now.
What I am trying to say is if you can do this then maybe you can build that string that will be use as the LINQ statement and then execute it. Since it is a string, it will be possible to analyse the string and get the information you want.
Try Dynamic Linq
To anyone else with the same question out there. Pulling out the where clause from LINQ-to-SQL isn’t quite as straightforward, as one would’ve hoped for. Additionally, doing that by itself is probably meaningless. There are a couple of options, depending on the requirements – either grab it from the generated string, but then it would contain parameter references and object property mappings that would also have to be resolved, so those would also have to be pulled out of the original provider somehow, otherwise this would be pointless. Another – would be to find a modular provider that can do that, as well as make member mappings easily accessible, but once again, without the rest of the query, I see little utility in doing that, because the where clause would reference table/column aliases from the select statement.
I had a similar task to write a full blown provider for a custom ORM/DAL a couple of years ago. While it qualifies as the most complex thing I’ve worked on, being an experience developer, I can say it’s not as bad, as some people claim once you wrap your head around the concepts that lie at the foundation of such a component. Some solutions that I’ve seen go the wrong way about it, add redundant functionality and have extra code addressing problems introduced by underlying logic. E.g. the “optimization” stage/module that attempts to re-factor bloated, nested SQL produced by the main parser. If the latter was designed in such a way that would output clean SQL from the start, then no clean-up phase would be needed. I’ve seen providers that create a new level of nesting for each where and join call. That’s a bad strategy. By breaking down a query into three/four main parts – select, from, where and orderby, which are built individually as the tree is being visited, this problem is avoided altogether. I’ve developed an object-to-data (aka LINQ-to-SQL) provided based on these principles for a custom ORM/DAL and it produces nice, clean SQL, with an excellent performance, as each statement is compiled to IL and cached.
For anyone that is looking to do something similar, please see my posts that include a sample project with a tutorial/barebones implementation that makes it easy to see how it works. Included is also the full solution:
How to write a LINQ to SQL provider in C# Part 1 - Introduction
How to write a LINQ to SQL provider in C# Part 2 - Expression Visitor
How to write a LINQ to SQL provider in C# Part 3 - Where Clause Visitor
How to write a LINQ to SQL provider in C# Part 4 - Compiling Expression Trees