We Keep Coding

How can i select generic columns from a table but make sure my keycolumn is selected too?

I have a table in my database with a lot of columns. I want to have a class where i load columns specified in the constructor into a list. I dont want to load all columns because that takes too long. Additionaly i may want to apply functions on specific columns becuase some data needs to be sanitized. Later i want to be able to return rows from this list by a keycolumn that is fixed (no need to specify it in the constructor).
This is kinda what i want:
public class DataHolder<TType> where TType:class
{
private List<TType> _data;
public DataHolder(DataContext context,Expression<Func<MyTable, TType>> select)
{
_data = context.MyTable.Select(select).DoSanitation().ToList();
//do sanitation on a column if it is in _data here
}
public TType Get(int id)
{
return _data.Single(d => d.Id == id);
}
}
And then i want to use it kinda like this:
var datHolder = new DataHolder(context, x=> new{x.Column1,x.Column2});
var row= datHolder.Get(123);
And row should have the fields "Column1" and "Column2" and "Id".
So i tried it by using anonymous types but because anonymous types cant use interfaces i am not able to make sure the type has the field "Id". Also the whole sanitation thing doesnt make sense on a anonymous type.
I have the sense that i am doing something i should not do or am not seeing a simple solution. I also had a look into Ado.Net which seems like it solve my problems because i can assemble columns adhoc. But all my other code runs with ef core so i am not sure if i should proceed in that direction.

You can't do this with anonymous types, but with types, known at compile time, you can do something like this:
public interface IEntity
{
public int Id { get; }
}
public class DataHolder<TType>
where TType : class, IEntity
{
private static readonly Lazy<IEnumerable<PropertyInfo>> MyTableProperties = new Lazy<IEnumerable<PropertyInfo>>(() => GetPublicInstanceProperties<MyTable>());
private static readonly Lazy<Expression<Func<MyTable, TType>>> Selector = new Lazy<Expression<Func<MyTable, TType>>>(GetSelector);
private readonly IReadOnlyDictionary<int, TType> data;
public DataHolder(MyContext context, Action<TType> doSanitation)
{
var entities = context.MyTable
.Select(Selector.Value)
.ToList();
foreach (var entity in entities)
{
doSanitation(entity);
}
data = entities.ToDictionary(_ => _.Id);
}
public TType Get(int id) => data[id];
private static Expression<Func<MyTable, TType>> GetSelector()
{
var lambdaParameter = Expression.Parameter(typeof(MyTable));
var memberBindings = GetPublicInstanceProperties<TType>()
.Select(propertyInfo => Expression.Bind(propertyInfo, Expression.MakeMemberAccess(lambdaParameter, MyTableProperties.Value.FirstOrDefault(p => p.Name == propertyInfo.Name))));
var memberInit = Expression.MemberInit(Expression.New(typeof(TType)), memberBindings);
return Expression.Lambda<Func<MyTable, TType>>(memberInit, lambdaParameter);
}
private static IEnumerable<PropertyInfo> GetPublicInstanceProperties<T>() => typeof(T).GetProperties(BindingFlags.Instance | BindingFlags.Public);
}
GetSelector method is just a simple mapper: it assigns property of TType object from the property of MyTable object with the same name.
Usage:
using (var context = new MyContext())
{
var dataHolder = new DataHolder<EntityA>(context, entity =>
{
// TODO:
});
var row = dataHolder.Get(1);
}
where EntityA is:
public class EntityA : IEntity
{
public int Id { get; set; }
public int A { get; set; }
}

LINQ to Entities and polymorphism

Let's say I have a simple model:
public class Movie
{
public int ID { get; set; }
public string Name { get; set; }
}
And a DbContext:
public class MoviesContext : DbContext
{
...
public DbSet<Movie> Movies { get; set; }
}
Also I have a method in MoviesContext class that filters Movies by substring like this:
return Movies.Where(m => m.Name.Contains(filterString)).Select(m => m);
Now suppose I'd like to add a new model, say:
public class Person
{
public int ID { get; set; }
public string FirstName { get; set; }
public string MiddleName { get; set; }
public string LastName { get; set; }
public string FullName { get { return FirstName + (MiddleName?.Length > 0 ? $" {MiddleName}" : "") + $" {LastName}"; } }
}
I also want to filter persons (DbSet Persons) by name (i.e. FullName). I'd like DRY, so it is preferrable to generalize a filter method of MoviesContext. And, what is important, I'd like to do filtering on the database level. So I have to deal with LINQ for Entities.
If not for this, the task is pretty simple. I could use an abstract class and add a virtual method that do the "contains substring" logic. Alternatively, I could use an interface. Unfortunately, because of LINQ for Entities, I can't use a FullName property (which is not convenient but bearable) and I can't write something like this:
return dbset.Where(ent => ent.NameContains(filterString)).Select(ent => ent);
So, how to solve this problem? I've found some solution (almost have my head broken), but I am not very happy with it. I'll post my solution separately, but I hope there is a more elegant one.

Reading your code a little more closely, instead of your NameFilterable abstract class, could you not do something like this:
public interface IHasPredicateGetter<T> {
[NotNull] Expression<Func<T, bool>> GetPredicateFromString([NotNull] string pValue);
}
public class Movie : IHasPredicateGetter<Movie> {
public int ID { get; set; }
public string Name { get; set; }
public Expression<Func<Movie, bool>> GetPredicateFromString(string pValue) {
return m => m.Name.Contains(pValue);
}
}
This prevents you from needing a cast, for example. It's so hard to grasp just what you're trying to do here, so I'm not sure this is the whole thing or not. You're still stuck with an instance method that should probably be a static method, but it couldn't implement an interface otherwise.

My solution looks like this.
[1] The base class:
public abstract class NameFilterable
{
protected static Expression<Func<T, bool>> False<T>() { return f => false; }
public virtual Expression<Func<T, bool>> GetNameContainsPredicate<T>(string filterString)
{
return False<T>();
}
}
[2] The Person class (I'll omit the Movie class, it is more simple):
public class Person : NameFilterable
{
...
public override Expression<Func<T, bool>> GetNameContainsPredicate<T>(string filterString)
{
return entity =>
String.IsNullOrEmpty(filterString) ||
(entity as Person).LastName.Contains(filterString) ||
(entity as Person).FirstName.Contains(filterString) ||
(((entity as Person).MiddleName != null) && (entity as Person).MiddleName.Contains(filterString))
;
}
}
[3] The filter methods in MoviesContext:
private static IQueryable<T> _filterDbSet<T>(DbSet<T> set, Expression<Func<T, bool>> filterPredicate) where T : class
{
return set
.Where(filterPredicate)
.Select(ent => ent);
}
private static IQueryable<T> _filterDbSet<T>(DbSet<T> set, string search = null) where T : NameFilterable, new()
{
T ent = new T();
return _filterDbSet<T>(set, (ent as NameFilterable).GetNameContainsPredicate<T>(search));
}
public static ICollection<T> Filter<T>(DbSet<T> set, string search = null) where T : NameFilterable, new()
{
return _filterDbSet(set, search).ToList();
}
And it seems that all this works pretty well. But I can't say it is very elegant.
[1] I have to use a generic T, though on the Person level I always work with Person objects (or descendants). So I have to convert T to Person (as Person).
[2] In GetNameContainsPredicate method, I can't write (because of LINQ for Entities):
return entity =>
{
Person p = entity as Person;
String.IsNullOrEmpty(filterString) ||
p.LastName.Contains(filterString) ||
p.FirstName.Contains(filterString) ||
((p.MiddleName != null) && p.MiddleName.Contains(filterString))
};
[3] I can't use static methods (statics couldn't be overridden), so I have to create a dummy T object (T ent = new T();).
[4] I still can't use a FullName.Contains(filterString)
So, the question remains: Maybe I miss something and there is a more elegant solution to the problem?

You could create a method that is responsible for search if a type has a particular property and filter for that property,if the object has not the property simply return null. Whith this you can create an expression that filters for this property
//gets the property info of the property with the giving name
public static PropertyInfo GetPropetyInfo<T>(string name)
{
var type = typeof(T);
var property = type.GetProperty(name);
return property;
}
//Creates an expression thats represents the query
public static Func<T, bool> GetFilterExpression<T>( string propertyName, object propertyValue)
{
var prop = GetPropetyInfo<T>(propertyName);
if(prop==null)return t=>false;
var parameter = Expression.Parameter(typeof(T), "t");
Expression expression = parameter;
var left = Expression.Property(expression, prop);
if (prop.PropertyType == typeof(string))
{
var toLower = typeof(string).GetMethods().FirstOrDefault(t => t.Name.Equals("ToLower"));
var tlCall = Expression.Call(left, toLower);
var right = Expression.Constant(propertyValue.ToString().ToLower());
var contains = Expression.Call(tlCall, typeof(string).GetMethod("Contains"), right);
var containsCall = Expression.IsTrue(contains);
expression = Expression.AndAlso(Expression.NotEqual(left, Expression.Constant(null)), containsCall);
}
else
{
if (prop.PropertyType.ToString().ToLower().Contains("nullable"))
{
var getValue = prop.PropertyType.GetMethods().FirstOrDefault(t => t.Name.Equals("GetValueOrDefault"));
var getValueCall = Expression.Call(left, getValue);
var right = Expression.Constant(propertyValue);
expression = Expression.Equal(getValueCall, right);
}
else
{
var value = Convert.ChangeType(propertyValue,prop.PropertyType);
var right = Expression.Constant(value);
expression = Expression.Equal(left, right);
}
}
return Expression.Lambda<Func<T, bool>>(expression, new ParameterExpression[] { parameter }).Compile();
}
The you can use it as follow
var expression = YOURCLASS.GetFilterExpression<Person>("LastName", "Jhon");
var result=dbset.Where(expression);

There are a few things I've done to get polymorphism with EF, but in your specific case of wanting reusable filters, I'm not sure it's worth the trouble. I've basically tried to do the same exact thing, but every time I end up realizing that there's no point. Ask yourself: what exactly are the benefits of doing this, and how is it any more flexible than what a Where clause already offers?
There are two issues. One is that it's hard or nigh impossible to get a filter to be used between two separate classes by using a shared interface (INamedObject for example). This is because you need a strongly typed expression. You can make a function that returns a strongly typed expression, but why would you not have just wrote the expression in the first place? The other issue is that you need a new filter expression for every search value, which is pretty close to where we are already.
If you perfected this, what would you have? The ability to infer type, specify a search value, and get an expression you could use? Isn't that essentially what we already have? The way Where clauses already are, they already have strong typing, and the ability to use dynamic search values. While it might feel a tiny bit redundant to say x => x.Name == value in more than one place, really the ability to specify such a concise and powerful filter statement is already a pretty amazing place to be.

generic class for linq / lambda expression

C# Entity framework 4.0
I have a database with 10's of table with 2 common columns 'id' and 'modstamp'
to access modstamp in a table I have a function
protected internal override string GetModStampinChild(int sid)
{
DBContext sq = new DBContext();
return sq.xxxx.Where(s => s.id == sid)
.Select(s => s.modstamp).SingleOrDefault().ToModStampString();
}
where xxxx change for every table.
I am presently overriding this function for every table.
Is there a way to use some kind of generic "class" which I could use where "xxxx" would be any table?

First, you would need to have all of your Entities implement either an interface or an abstract class that contains both the ID and ModStamp properties in it, let's call it Stampable:
public abstract class Stampable
{
[Key]
public int ID { get; set; }
[Required]
public string ModStamp { get; set; }
}
At that point, all you need to do for your method is to have it implement generic typing:
protected internal override string GetModStampInChild<T>(int sid) where T : Stampable
{
using (var sq = new DbContext())
{
return sq.Set<T>.Where(s => s.id == sid)
.Select(s => s.modstamp)
.SingleOrDefault()
.ToModStampString();
}
}

If I understand you correctly, you need a property Set<T> of DbContext class:
First, create base class of all your entity classes with id and modstamp properties. Then:
protected internal override string GetModStampInChild<T>(int sid) where T : BaseEntity
{
using (var sq = new DbContext())
{
return sq.Set<T>.Where(s => s.id == sid)
.Select(s => s.modstamp)
.SingleOrDefault()
.ToModStampString();
}
}
But you must use code-first paradigm for this method.

Another option would be add a new Property to your entity class via the partial class feature of c#.
So the entity definition generated might look like this, note I have no idea what the actual DataType of your ModStamp column is:
public partial class Company
{
public int Id { get; set; }
public byte[] ModStamp { get; set; }
public string Name { get; set; }
public string City { get; set; }
public string State { get; set; }
}
Note the ModStamp column that you want to convert.
Then add to the Partial.cs file that EF creates code like this, note I have no idea what you actually want to do with the ModStamp value:
public static class ModConverter
{
public static string ToModStampString(byte[] modStamp)
{
return BitConverter.ToString(modStamp);
}
}
public partial class Company
{
public string ModStampString
{
get
{
return ModConverter.ToModStampString(this.ModStamp);
}
}
}
You would then have to manually add a new ModStampString Get Property for every Entity with a ModStamp Column like I did for the Company Entity.

Here is a solution that uses the Set method on the DbContext and expression trees to dynamically query that object.
private Expression<Func<TArg, bool>> CreatePredicate<TArg, TPredicateField>(string fieldName, TPredicateField value)
{
ParameterExpression parameter = Expression.Parameter(typeof(TArg), "o");
MemberExpression memberExpression = Expression.Property(parameter, fieldName);
var condition = Expression.Equal(memberExpression, Expression.Constant(value));
var lambda = Expression.Lambda<Func<TArg, bool>>(condition, parameter);
return lambda;
}
private Expression<Func<TArg, TPredicateField>> CreateSelector<TArg, TPredicateField>(string fieldName)
{
ParameterExpression parameter = Expression.Parameter(typeof(TArg), "o");
Expression propertyExpr = Expression.Property(parameter, fieldName);
var lambda = Expression.Lambda<Func<TArg, TPredicateField>>(propertyExpr, parameter);
return lambda;
}
public TSelectorField GetModStamp<TEntity, TPredicateField, TSelectorField>(TPredicateField id) where TEntity : class
{
using (var ctx = new OnTheFlyEntities("Data Source=(local);Initial Catalog=AscensionBO;Integrated Security=True;MultipleActiveResultSets=True"))
{
var predicate = CreatePredicate<TEntity, TPredicateField>("Id", id);
var selector = CreateSelector<TEntity, TSelectorField>("ModStamp");
TSelectorField item = ctx.Set<TEntity>().Where(predicate).Select(selector).SingleOrDefault();
return item;
}
}
You can then call it like this:
GetModStamp<Entity2, int, string>(1)
If you were willing to just return the found entity, you could eliminate the TSelectorField and then grab the ModStamp from the item after it is retrieved. That will drop one of the expression tree methods and a generic input on the main method.
As someone else suggested, you could go the interface route and use that example, it will be much simpler.

How to make expression treat value type as a reference type?

I wanted to store a collection of expressions accessing object's properties. For example:
class Entity
{
public int Id { get; set; }
public Entity Parent { get; set; }
public string Name { get; set; }
public DateTime Date { get; set; }
public decimal Value { get; set; }
public bool Active { get; set; }
}
static void Main(string[] args)
{
var list = new List<Expression<Func<Entity, object>>>();
list.Add(e => e.Id);
list.Add(e => e.Name);
list.Add(e => e.Parent);
list.Add(e => e.Date);
list.Add(e => e.Value);
list.Add(e => e.Active);
StringBuilder b = new StringBuilder();
list.ForEach(f => b.AppendLine(f.ToString()));
Console.WriteLine(b.ToString());
Console.ReadLine();
}
This code outputs:
e => Convert(e.Id)
e => e.Name
e => e.Parent
e => Convert(e.Date)
e => Convert(e.Value)
e => Convert(e.Active)
It does add Convert to value types.
As far as in the end I wanted to use those expressions with LINQ to SQL, I need not to have that Convert in expressions, for them to be successfully translated to SQL.
How can I achieve this?
P.S.: expressions from this collection are later used as arguments to OrderBy and ThenBy methods.

If you create a function generic in the proeprty type you can avoid the Convert:
private static LambdaExpression GetExpression<TProp>
(Expression<Func<Entity, TProp>> expr)
{
return expr;
}
then you can change the type of list:
var list = new List<LambdaExpression>();
list.Add(GetExpression(e => e.Id));
list.Add(GetExpression(e => e.Name));
This will require you to create your OrderBy and ThenBy expressions using reflection e.g.
LambdaExpression idExpr = list[0];
Type keyType = idExpr.ReturnType;
var orderByMethod = typeof(Queryable).GetMethods()
.Single(m => m.Name == "OrderBy" && m.GetParameters().Length == 2)
.MakeGenericMethod(typeof(Entity), keyType);
var ordered = (IQueryable<Entity>)
orderByMethod.Invoke(null, new object[] { source, idExpr });

I patched up a EF code first attempt at using your code like this
public class Entity
{
public int Id { get; set; }
public Entity Parent { get; set; }
public string Name { get; set; }
public DateTime Date { get; set; }
public decimal Value { get; set; }
public bool Active { get; set; }
}
public class EntityContext : DbContext
{
public EntityContext()
: base(new SqlCeConnection("Data Source=Database.sdf;Persist Security Info=False;"),
contextOwnsConnection: true)
{
// Using a SQL Compact database as backend
}
public DbSet<Entity> Entities { get; set; }
}
and attempted some linq on the context
static void Main(string[] args)
{
var list = new List<Expression<Func<Entity, object>>>();
list.Add(e => e.Date);
list.Add(e => e.Name);
using (var c = new EntityContext())
{
//each time a new record is added
var data = new Entity
{
Name = string.Format("Data{0}", c.Entities.Count()),
Date = DateTime.Now
};
c.Entities.Add(data);
c.SaveChanges();
// sort by date
foreach (var e in c.Entities.OrderBy(list.First().Compile()))
Console.WriteLine(string.Format("{0} - {1}", e.Name, e.Date));
// sort by name .. in reverse
foreach (var e in c.Entities.OrderByDescending(list.Last().Compile()))
Console.WriteLine(string.Format("{0} - {1}", e.Name, e.Date));
}
Console.ReadLine();
}
There were no issues running the code.
UPDATE The same holds true for LINQ to SQL: I built a table in a local SQL Server with the same structure as the class, and tried to OrderBy it : no problem.
My answer is "You don't need to worry about that".

Thank's to the answer by Alex I found out for myself that, when ordering the data I can use two different methods, depending on the specified argument:
Queryable.OrderBy Method with Expression<Func<TSource, TKey>>
Enumerable.OrderBy Method with Func<TSource, TKey>
When Queryable.OrderBy is used, LINQ compiles the OrderBy clause into the SQL statement, executed over the database. So when I try to give it a Expression<Func<TEntity, object>> that looks like e => Convert(e.Field), LINQ throws an InvalidOperationException, saying Cannot order by type 'System.Object'.
When Enumerable.OrderBy is used, LINQ does not compile the OrderBy clause into the SQL query, but executes the current query and applies sorting on the enumerable of entities, returned by the query, in the program's memory. Here no problem with ordering by Func<TEntity, object>.
So I found two alternatives here:
Query the database without sorting and order the returned result set
Provide better expressions to LINQ, that it could compile the SQL query, and then apply sorting in the database layer; here the answer by Lee suggests one way..
In my exact case sorting is the last operation to execute, and I don't see much harm, if I order the result set in the programm's memory...I'm not going to expect huge amounts of data to be returned...
Though in a more common case, probably it's still better to do all possible operations in the database layer...
P.S.: SO: Order a linq query - a close discussion...

Changing selected objects inside a query

I have a class that needs a property set inside a LINQ-to-SQL query. My first attempt was to have a "setter" method that would return the object instance and could be used in my select, like this:
public partial class Foo
{
public DateTime RetrievalTime { get; set; }
public Foo SetRetrievalTimeAndReturnSelf ( DateTime value )
{
RetrievalTime = value;
return this;
}
}
....
from foo in DataContext.GetTable<Foo> select foo.SetRetrievalTimeAndReturnSelf();
Unfortunately, such a query throws an exception like this: "System.NotSupportedException: Method 'Foo.SetRetrievalTime(System.DateTime)' has no supported translation to SQL".
Is there any alternative to converting the result to a list and iterating over it? The query is used in a custom "Get" method that wraps the DataContext.GetTable method, so will be used as the base for many other queries. Immediately converting a potentially-large result set to a list would not be optimal.
UPDATE
Here's a better example of what I'm trying to do, updated with Jason's proposed solution:
protected IQueryable<T> Get<T>() where T : class, ISecurable
{
// retrieve all T records and associated security records
var query = from entity in DataContext.GetTable<T> ()
from userEntityAccess in DataContext.GetTable<UserEntityAccess> ()
where userEntityAccess.SysUserId == CurrentUser.Id
&& entity.Id == userEntityAccess.EntityId
&& userEntityAccess.EntityClassName == typeof ( T ).Name
select new { entity, userEntityAccess };
return query.AsEnumerable ()
.Select ( item =>
{
item.entity.CanRead = item.userEntityAccess.CanRead;
item.entity.CanWrite = item.userEntityAccess.CanWrite;
item.entity.CanDelete = item.userEntityAccess.CanDelete;
return item.entity;
} ).AsQueryable ();
}
public interface ISecurable
{
int Id { get; set; }
bool CanRead { get; set; }
bool CanWrite { get; set; }
bool CanDelete { get; set; }
}
UserEntityAccess is a cross-reference table between a user and a business object record (i.e. an entity). Each record contains fields like "CanRead", "CanWrite", and "CanDelete", and determines what a specific user can do with a specific record.
ISecurable is a marker interface that must be implemented by any LINQ-to-SQL domain class that needs to use this secured Get method.

var projection = DataContext.GetTable<Foo>
.AsEnumerable()
.Select(f => f.SetRetrievalTimeAndReturnSelf());
This will then perform the invocation of SetRetrievalTimeAndReturnSelf for each instance of Foo in DataContext.GetTable<Foo> when the IEnumerable<Foo> projection is iterated over.
What do you need to know the time that object was yanked of the database for? That's potentially smelly.