We Keep Coding

Intercepting and overriding a comparison expression in an object that otherwise delegates to Linq to Objects provider

The goal here is to get some WCF Data Services query scenarios accurately reflected in code under test.
Following the same pattern for mocking EF functionality detailed here, I'm mocking a DataServiceQuery wrapper class and delegating to the IQueryable returned by calling .AsQueryable() on my test data.
I want to override the LINQ to objects behavior for a specific equality expression (if it matters, when comparing two byte arrays). I want to compare the values in the arrays instead of the references (e.g. using SequenceEqual).
The way I can see accomplishing this is to create a provider where I visit the expression, pull out the appropriate values, compose a new expression, and pass that along to the LINQ to objects provider. That's a lot of work. Short of doing this, is there an easier way to get this done or will I have to resort to parsing expressions?

You can try to create your own extension methods that take a more specific data type. Something along this line :
static IEnumerable<byte[]> Where(this IEnumerable<byte[]> source, Func<byte[], bool> predicate)
{
}

Access a property of a DbSet by name

I have a DbSet<T>
I want to access one of the properties on it by name.
Is this possible?
I'm basically making a generic function, so that for any DbSet I have, I can specify a column and have it list the values of that column.
You can get the DbSet itself from the context by doing context.Set(T) but I am not sure about the fields.

I did something similar a while back using reflection.
T item = context.Set(T).First();
string propName = "MyProperty";
object value = item.GetType().GetProperty(propName).GetValue(item, null);
Of course note that you'll either need to cast the values to a specific type manually, or use ToString, which should work quite well on all basic types.
This assumes you already have the data from the server, and now need to process it.
EDIT:
If you want to create a query, then I found this!
Apparently, what you're looking for is available as part of Entity Framework these days.
An extension method is available which allows you to use .Select("propertyName") which returns IQueriable. Remember to add System.Linq.Dynamic to your using section.
You can then create select queries by specifying the name of the parameter.
List<object> data = (db.Set<SetType>()
.Where("propertyName == #0 && someOtherProperty == #1", propertyValue, someOtherPropertyValue)
.Select("propertyName") as IEnumerable<object>).ToList();

Check out this article on Dynamic LINQ.
Using the provided code, I was able to write a LINQ to Entities query like this:
var query = context.Set(Type.GetType("Person")).Select("Name");

Using Dynamic LINQ (or Generics) to query/filter Azure tables

So here's my dilemma. I'm trying to utilize Dynamic LINQ to parse a search filter for retrieving a set of records from an Azure table. Currently, I'm able to get all records by using a GenericEntity object defined as below:
public class GenericEntity
{
public string PartitionKey { get; set; }
public string RowKey { get; set; }
Dictionary<string, object> properties = new Dictionary<string, object>();
/* "Property" property and indexer property omitted here */
}
I'm able to get this completely populated by utilizing the ReadingEntity event of the TableServiceContext object (called OnReadingGenericEvent). The following code is what actually pulls all the records and hopefully filter (once I get it working).
public IEnumerable<T> GetTableRecords(string tableName, int numRecords, string filter)
{
ServiceContext.IgnoreMissingProperties = true;
ServiceContext.ReadingEntity -= LogType.GenericEntity.OnReadingGenericEntity;
ServiceContext.ReadingEntity += LogType.GenericEntity.OnReadingGenericEntity;
var result = ServiceContext.CreateQuery<GenericEntity>(tableName).Select(c => c);
if (!string.IsNullOrEmpty(filter))
{
result = result.Where(filter);
}
var query = result.Take(numRecords).AsTableServiceQuery<GenericEntity>();
IEnumerable<GenericEntity> res = query.Execute().ToList();
return res;
}
I have TableServiceEntity derived types for all the tables that I have defined, so I can get all properties/types using Reflection. The problem with using the GenericEntity class in the Dynamic LINQ Query for filtering is that the GenericEntity object does NOT have any of the properties that I'm trying to filter by, as they're really just dictionary entries (dynamic query errors out). I can parse out the filter for all the property names of that particular type and wrap
"Property[" + propName + "]"
around each property (found by using a type resolver function and reflection). However, that seems a little... overkill. I'm trying to find a more elegant solution, but since I actually have to provide a type in ServiceContext.CreateQuery<>, it makes it somewhat difficult.
So I guess my ultimate question is this: How can I use dynamic classes or generic types with this construct to be able to utilize dynamic queries for filtering? That way I can just take in the filter from a textbox (such as "item_ID > 1023000") and just have the TableServiceEntity types dynamically generated.
There ARE other ways around this that I can utilize, but I figured since I started using Dynamic LINQ, might as well try Dynamic Classes as well.
Edit: So I've got the dynamic class being generated by the initial select using some reflection, but I'm hitting a roadblock in mapping the types of GenericEntity.Properties into the various associated table record classes (TableServiceEntity derived classes) and their property types. The primary issue is still that I have to initially use a specific datatype to even create the query, so I'm using the GenericEntity type which only contains KV pairs. This is ultimately preventing me from filtering, as I'm not able to do comparison operators (>, <, =, etc.) with object types.
Here's the code I have now to do the mapping into the dynamic class:
var properties = newType./* omitted */.GetProperties(
System.Reflection.BindingFlags.Instance |
System.Reflection.BindingFlags.Public);
string newSelect = "new(" + properties.Aggregate("", (seed, reflected) => seed += string.Format(", Properties[\"{0}\"] as {0}", reflected.Name)).Substring(2) + ")";
var result = ServiceContext.CreateQuery<GenericEntity>(tableName).Select(newSelect);
Maybe I should just modify the properties.Aggregate method to prefix the "Properties[...]" section with the reflected.PropertyType? So the new select string will be made like:
string newSelect = "new(" + properties.Aggregate("", (seed, reflected) => seed += string.Format(", ({1})Properties[\"{0}\"] as {0}", reflected.Name, reflected.PropertyType)).Substring(2) + ")";
Edit 2: So now I've hit quite the roadblock. I can generate the anonymous types for all tables to pull all values I need, but LINQ craps out on my no matter what I do for the filter. I've stated the reason above (no comparison operators on objects), but the issue I've been battling with now is trying to specify a type parameter to the Dynamic LINQ extension method to accept the schema of the new object type. Not much luck there, either... I'll keep you all posted.

I've created a simple System.Refection.Emit based solution to create the class you need at runtime.
http://blog.kloud.com.au/2012/09/30/a-better-dynamic-tableserviceentity/

I have run into exactly the same problem (with almost the same code :-)). I have a suspicion that the ADO.NET classes underneath somehow do not cooperate with dynamic types but haven't found exactly where yet.

So I've found a way to do this, but it's not very pretty...
Since I can't really do what I want within the framework itself, I utilized a concept used within the AzureTableQuery project. I pretty much just have a large C# code string that gets compiled on the fly with the exact object I need. If you look at the code of the AzureTableQuery project, you'll see that a separate library is compiled on the fly for whatever table we have, that goes through and builds all the properties and stuff we need as we query the table. Not the most elegant or lightweight solution, but it works, nevertheless.
Seriously wish there was a better way to do this, but unfortunately it's not as easy as I had hoped. Hopefully someone will be able to learn from this experience and possibly find a better solution, but I have what I need already so I'm done working on it (for now).

Linq based generic alternate to Predicate<T>?

I have an interface called ICatalog as shown below where each ICatalog has a name and a method that will return items based on a Predicate<Item> function.
public interface ICatalog
{
string Name { get; }
IEnumerable<Item> GetItems(Predicate<Item> predicate);
}
A specific implementation of a catalog may be linked to catalogs in various format such as XML, or a SQL database.
With an XML catalog I end up deserializing the entire XML file into memory, so testing each item with the predicate function does does not add a whole lot more overhead as it's already in memory.
Yet with the SQL implementation I'd rather not retrieve the entire contents of the database into memory, and then filter the items with the predicate function. Instead I'd want to find a way to somehow pass the predicate to the SQL server, or somehow convert it to a SQL query.
This seems like a problem that can be solved with Linq, but I'm pretty new to it. Should my interface return IQueryable instead? I'm not concerned right now with how to actually implement a SQL version of my ICatalog. I just want to make sure my interface will allow for it in the future.

Rob has indicated how you might do this (although a more classic LINQ approach might take Expression<Func<Item,bool>>, and possbily return IQueryable<IFamily>).
The good news is that if you want to use the predicate with LINQ-to-Objects (for your xml scenario) you can then just use:
Predicate<Item> func = predicate.Compile();
or (for the other signature):
Func<Item,bool> func = predicate.Compile();
and you have a delegate (func) to test your objects with.
The problem though, is that this is a nightmare to unit test - you can only really integration test it.
The problem is that you can't reliably mock (with LINQ-to-Objects) anything involving complex data-stores; for example, the following will work fine in your unit tests but won't work "for real" against a database:
var foo = GetItems(x => SomeMagicFunction(x.Name));
static bool SomeMagicFunction(string name) { return name.Length > 3; } // why not
The problem is that only some operations can be translated to TSQL. You get the same problem with IQueryable<T> - for example, EF and LINQ-to-SQL support different operations on a query; even just First() behaves differently (EF demands you explicitly order it first, LINQ-to-SQL doesn't).
So in summary:
it can work
but think carefully whether you want to do this; a more classic black box repository / service interface may be more testable

You don't need to go all the way and create an IQueryable implementation
If you declare your GetItems method as:
IEnumerable<IFamily> GetItems(Expression<Predicate<Item>> predicate);
Then your implementing class can inspect the Expression to determine what is being asked.
Have a read of the IQueryable article though, because it explains how to build a expression tree visitor, which you'll need to build a simple version of.

LINQ to SQL Custom Property query on where clause

I am using LINQ to SQL classes, and have extended one (with a partial class) and added an extra property. I want to query on this property, so:
(from p in db.TableName where p.CustomProperty=="value" select p)
However this doesn't work. I get an error: The member 'TableName.CustomProperty' has no supported translation to SQL. At the moment I have no code in the 'set' section of my Custom Property as I'm unsure how.
So basically, Custom Property which can be queried on with LINQ to SQL, how?
As a follow up: this CustomProperty is NOT a column in the table. It is a separate value, but I need to both fetch it (easy enough) but also query on it!

As you can understand, there can't be any magic, so essentially there will be two queries: first one is a SQL query with database criteria and on its result there should be applied your custom criteria as a second query.
So the workaround you could use is to split two parts explicitly like this:
var dbFetch = (from p in db.TableName where p.RealProperty ==" value" select p).ToArray();
var result = from p in dbFetch where p.CustomProperty == "value" select p;
But of course you'll run into several limitations. For example if you fetching results page-by-page, the second criterion will break paging since it performs additional filtering.
HTH

It's called LINQ to SQL. Just to avoid misunderstandings.
About your problem: have you added that property using the designer? And have you re-created the database after that?
If you did it by hand, make sure you have a private storage field (like _CustomProperty), and your property (CustomProperty) is marked with the ColumnAttribute, e.g.
private string _CustomProperty;
[Column(Storage="_CustomProperty", CanBeNull=true)]
public string CustomProperty
{
get { return _CustomProperty; }
}
Hope this helps.

Aren't you missing an equality sign there? In C#, equality is expressed with double equal signs, as in "a == b", while single equal sign signifies assignment, as in "obj.SomeProp = 5;"

I have implemented a system where you can query manually added properties that represent enumerated wrappers around integer properties from database columns. So I know it's possible, and it looks like you might be wanting to do something similar. The way I did it was not easy, though, and unless you are building a framework that you want to use properly for many cases, you might be better off using the solution suggested by archimed7592. I don't have the code handy at the moment so I can't provide all the details, but briefly my solution works like this. I created a custom LINQ provider, replacing the LINQ-to-SQL provider. I did this by implementing a custom IQueryable interface that returned my LINQ provider instead of that provided by LINQ-to-SQL. Then, in the functions that take expression objects, I pre-processed the expression before returning the result. I replaced all comparisons between enum-type properties and enum values with comparisons between integer properties and integer values, then passed that expression to the normal LINQ-to-SQL implementation in order to return the result. Since expressions are read-only, I had to make a (recursive) function that re-built the entire expression with the customized parts replaced.