Specifying deferred action before IQueryable has been evaluated - c#

I would like to perform some post-processing on an entity that is returned from an IQueryable, but I would like to specify that post-processing before evaluating the query. So for example:
IQueryable<Client> GetClients()
{
return _context.Set<Client>()
.PostProcess(c => c.MobileNumber = c.MobileNumber.Replace(" ", ""));
}
Client GetClient(int id)
{
return GetClients()
.FirstOrDefault(c => c.Id == id);
}
In the above example I would want every Client returned from the above two methods to have all spaces removed from their mobile number. Is this possible?

Edit: This is a bad idea
It works in the simple case stated in the OP, but any further IQueryable extensions (such as a .Where() clause) will discard the post-processing behaviour. I'm not sure it's technically possible to do what I want to do, and now I come to think of it I'm not even sure what the semantics would be beyond the simple case. Still, the simple case works and is quite handy to keep things fluent...
Original:
I thought I would share my approach for this in case someone finds it useful (or has a comment on why it's a bad idea). The solution uses two decorators to defer the post-processing action until the query is executed, and an extension method to set them up:
public static class QueryableExtensions
{
public static IQueryable<T> PostProcess<T>(this IQueryable<T> source, Action<T> postProcessor) where T : class
{
return new QueryableWrapper<T>(source, postProcessor);
}
// wraps IQueryProvider.Execute methods with post-processing action
class QueryProviderWrapper<T> : IQueryProvider where T : class
{
private readonly IQueryProvider _wrapped;
private readonly Action<T> _postProcessor;
public QueryProviderWrapper(IQueryProvider wrapped, Action<T> postProcessor)
{
_wrapped = wrapped;
_postProcessor = postProcessor;
}
public IQueryable CreateQuery(Expression expression)
{
return _wrapped.CreateQuery(expression);
}
public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
{
return _wrapped.CreateQuery<TElement>(expression);
}
public object Execute(Expression expression)
{
var result = _wrapped.Execute(expression);
var asT = result as T;
if (asT != null)
_postProcessor(asT);
return result;
}
public TResult Execute<TResult>(Expression expression)
{
var result = _wrapped.Execute<TResult>(expression);
var asT = result as T;
if (asT != null)
_postProcessor(asT);
return result;
}
}
// wraps IQueryable.GetEnumerator() with post-processing action
class QueryableWrapper<T> : IQueryable<T> where T : class
{
private readonly IQueryable<T> _wrapped;
private readonly Action<T> _postProcessor;
private readonly IQueryProvider _provider;
public QueryableWrapper(IQueryable<T> wrapped, Action<T> postProcessor)
{
_wrapped = wrapped;
_postProcessor = postProcessor;
_provider = new QueryProviderWrapper<T>(_wrapped.Provider, postProcessor);
}
public Expression Expression
{
get { return _wrapped.Expression; }
}
public Type ElementType
{
get { return _wrapped.ElementType; }
}
public IQueryProvider Provider
{
get { return _provider; }
}
public IEnumerator<T> GetEnumerator()
{
return _wrapped
.AsEnumerable()
.Do(_postProcessor)
.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
}
The GetEnumerator() decorator uses the .Do() extension method from Interactive Extensions which is like a cross between Select and ForEach: it is lazily invoked, but takes an Action<T> and returns T

Related

EF Core LINQ Where with extension methods are not translated to SQL

Given the following query:
context.ToolingOrders
.Include(r => r.ToolingOrderDetails)
.Include(r => r.PurchaseOrder)
.Where(r => r.VendorId.EqualsOrNull(filter.VendorId) &&
r.PoNumber.ContainsOrEmpty(filter.PoNumber))
I use these extension methods to save some code:
public static class FilterExtensions
{
public static bool ContainsOrEmpty(this string source, string toCheck)
{
return string.IsNullOrWhiteSpace(toCheck) || source?.IndexOf(toCheck, StringComparison.OrdinalIgnoreCase) >= 0;
}
public static bool EqualsOrNull(this int source, int? toCheck)
{
return !toCheck.HasValue || source == toCheck;
}
}
The problem is, due to these extension methods, the Where part is not translated into SQL.
I use .Net Core 2.2, which has this client side evaluation feature, which totally hides this issue, but the SQL profiler shows it anyway.
Is there any way to make this work, or I have to write every piece of the where part explicitly?
You could extend the IQueryable for your specific object.
Take the following object for instance:
public class MyObject
{
public string MyProperty;
}
You could write an extension like this:
public static class MyQueryExtension
{
public static IQueryable<MyObject> WhereMyPropertyNull(this IQueryable<MyObject> queryable)
{
return queryable.Where(obj => obj.MyProperty == null);
}
}
And use it like this:
var queryable = new List<MyObject>().AsQueryable();
var result = queryable.WhereMyPropertyNull().ToList();
EDIT
Based on some feedback i updated my answer to handle generics.
public static class Extensions
{
public static IQueryable<TEntity> EqualOrNull<TEntity, TProperty>(this IQueryable<TEntity> source, Func<TEntity, TProperty> selector, TProperty match)
{
return source.Where(entity => Match(selector.Invoke(entity), match));
}
private static bool Match<TEntity, TProperty>(TEntity entity, TProperty match)
{
if (entity == null) {
return true;
} else {
return entity.Equals(match);
}
}
}
It can be used to pass the value of a property to the where statement:
var list = new List<MyObject>();
list.Add(new MyObject {MyProperty = "Test"});
list.Add(new MyObject {MyProperty = "NoMatch"});
list.Add(new MyObject {MyProperty = null});
var result = list.AsQueryable()
.EqualOrNull(o => o.MyProperty, "Test")
.ToList();

How can I abstract IRepository<Invoice> and IRepository<Estimate> as IRepository<SalesTransaction>?

I have the following class structure
public abstract class SalesTransaction
{
}
public class Invoice : SalesTransaction
{
}
public class Estimate : SalesTransaction
{
}
public interface IRepostory<T>
{
T First(Expression<Func<T, bool>> predicate);
void Add(T item);
T Single(Expression<Func<T, bool>> predicate, bool disableTracking = false);
}
Can I somehow abstract IRepository<Invoice> and IRepository<Estimate> by casting it to IRepository<SalesTransaction>?
IRepository<Invoice> invoiceRepository = /* Create instance */;
IRepository<SalesTransaction> salesTransaction = invoiceRepository;
The code above doesn't compile. Is there a workaround to get it to compile and work?
You could create a SalesTransaction repository as wrapper around repositories of more specific types. (Shown here for invoices)
public class InvoiceAsSalesTransactionRepository : IRepostory<SalesTransaction>
{
private readonly IRepostory<Invoice> _invoiceRepository = new InvoiceRepository();
public void Add(SalesTransaction item)
{
if (item is Invoice invoice) {
_invoiceRepository.Add(invoice);
} else {
throw new ArgumentException("Item must be Invoice");
}
}
public SalesTransaction First(Expression<Func<SalesTransaction, bool>> predicate)
{
return _invoiceRepository.First(ToInvoicePredicate(predicate));
}
public SalesTransaction Single(Expression<Func<SalesTransaction, bool>> predicate,
bool disableTracking = false)
{
return _invoiceRepository.Single(ToInvoicePredicate(predicate), disableTracking);
}
private static Expression<Func<Invoice, bool>> ToInvoicePredicate(
Expression<Func<SalesTransaction, bool>> predicate)
{
var param = Expression.Parameter(typeof(Invoice), predicate.Parameters[0].Name);
return Expression.Lambda<Func<Invoice, bool>>(predicate.Body, param);
}
}
As you can see, you need some Reflection magic to convert the predicate type. Therefore, you cannot simply cast your repositories. Note that we don't need to cast anything here, since a valid SalesTansaction predicate is always a valid Invoice predicate, as SalesTansactions contain only properties also available in Invoices.

Override IEnumerable<T> Where

I've written a class that implements IEnumerable :
public class MyEnumerable : IEnumerable<MyClass>
{
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
public IEnumerator<MyClass> GetEnumerator()
{
//Enumerate
}
}
I'd like to "override" the Where method. What I want to do is :
MyEnumerable myEnumerable = new MyEnumerable();
MyEnumerable myEnumerable2 = myEnumerable.Where(/*some predicate*/);
This is not possible for the moment because myEnumerable.Where() returns an IEnumerable.
What I want is that myEnumerable.Where() returns a MyEnumerable.
Is that possible to do this ?
Thank you
Sure - just add a Where method to MyEnumerable. The Linq Where method is an extension method, so it's not technically an override. you're "hiding" the linq method.
public class MyEnumerable : IEnumerable<MyClass>
{
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
public IEnumerator<MyClass> GetEnumerator()
{
//Enumerate
}
public MyEnumerable Where()
{
// implement `Where`
}
}
There are some caveats, though:
Your Where method will only be called if the declared type is MyEnumerable - it will not be called on variables of type IEnumerable<MyClass> (or any collection that implements it, like List<MyClass>
There are several overloads of Where that will need to be implemented as well if you want to maintain consistently with Linq.
Update
From your comment your enumerator is a lazy file enumerator and you want to be able to select items from it based on a predicate and still have the laziness.
You could create another class inheriting that class or an interface to help with this.
Here is an example
public class FileItem
{
//Some properties
}
public interface IFileEnumerator : IEnumerable<FileItem>
{
IFileEnumerator Where(Func<FileItem, bool> predicate);
}
public class FileEnumerator : IFileEnumerator
{
private readonly string fileName;
public FileEnumerator(string fileName)
{
this.fileName = fileName;
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
public IEnumerator<FileItem> GetEnumerator()
{
var items = new List<FileItem>();
//Read from file and add lines to items
return items.GetEnumerator();
}
public IFileEnumerator Where(Func<FileItem, bool> predicate)
{
return new MemoryEnumerator(ToEnumerable(GetEnumerator()).Where(predicate));
}
private static IEnumerable<T> ToEnumerable<T>(IEnumerator<T> enumerator)
{
while (enumerator.MoveNext())
{
yield return enumerator.Current;
}
}
}
public class MemoryEnumerator : IFileEnumerator
{
private readonly IEnumerable<FileItem> items;
public MemoryEnumerator(IEnumerable<FileItem> items)
{
this.items = items;
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
public IEnumerator<FileItem> GetEnumerator()
{
return items.GetEnumerator();
}
public IFileEnumerator Where(Func<FileItem, bool> predicate)
{
return new MemoryEnumerator(items.Where(predicate));
}
}

Instantiating Immutable Objects With Reflection

I created a base class to help me reduce boilerplate code of the initialization of the immutable Objects in C#,
I'm using lazy initialization in order to try not to impact performance a lot ,
I was wondering how much am I affecting the performance by doing this?
This is my base class:
public class ImmutableObject<T>
{
private readonly Func<IEnumerable<KeyValuePair<string, object>>> initContainer;
protected ImmutableObject() {}
protected ImmutableObject(IEnumerable<KeyValuePair<string,object>> properties)
{
var fields = GetType().GetFields().Where(f=> f.IsPublic);
var fieldsAndValues =
from fieldInfo in fields
join keyValuePair in properties on fieldInfo.Name.ToLower() equals keyValuePair.Key.ToLower()
select new {fieldInfo, keyValuePair.Value};
fieldsAndValues.ToList().ForEach(fv=> fv.fieldInfo.SetValue(this,fv.Value));
}
protected ImmutableObject(Func<IEnumerable<KeyValuePair<string,object>>> init)
{
initContainer = init;
}
protected T setProperty(string propertyName, object propertyValue, bool lazy = true)
{
Func<IEnumerable<KeyValuePair<string, object>>> mergeFunc = delegate
{
var propertyDict = initContainer == null ? ObjectToDictonary () : initContainer();
return propertyDict.Select(p => p.Key == propertyName? new KeyValuePair<string, object>(propertyName, propertyValue) : p).ToList();
};
var containerConstructor = typeof(T).GetConstructors()
.First( ce => ce.GetParameters().Count() == 1 && ce.GetParameters()[0].ParameterType.Name == "Func`1");
return (T) (lazy ? containerConstructor.Invoke(new[] {mergeFunc}) : DictonaryToObject<T>(mergeFunc()));
}
private IEnumerable<KeyValuePair<string,object>> ObjectToDictonary()
{
var fields = GetType().GetFields().Where(f=> f.IsPublic);
return fields.Select(f=> new KeyValuePair<string,object>(f.Name, f.GetValue(this))).ToList();
}
private static object DictonaryToObject<T>(IEnumerable<KeyValuePair<string,object>> objectProperties)
{
var mainConstructor = typeof (T).GetConstructors()
.First(c => c.GetParameters().Count()== 1 && c.GetParameters().Any(p => p.ParameterType.Name == "IEnumerable`1") );
return mainConstructor.Invoke(new[]{objectProperties});
}
public T ToObject()
{
var properties = initContainer == null ? ObjectToDictonary() : initContainer();
return (T) DictonaryToObject<T>(properties);
}
}
Can be implemented like so:
public class State:ImmutableObject<State>
{
public State(){}
public State(IEnumerable<KeyValuePair<string,object>> properties):base(properties) {}
public State(Func<IEnumerable<KeyValuePair<string, object>>> func):base(func) {}
public readonly int SomeInt;
public State someInt(int someInt)
{
return setProperty("SomeInt", someInt);
}
public readonly string SomeString;
public State someString(string someString)
{
return setProperty("SomeString", someString);
}
}
and can be used like this:
//creating new empty object
var state = new State();
// Set fields, will return an empty object with the "chained methods".
var s2 = state.someInt(3).someString("a string");
// Resolves all the "chained methods" and initialize the object setting all the fields by reflection.
var s3 = s2.ToObject();
As was already mentioned in the comments, it would make more sense, not to "conflate" the immutable instance implementation or interface with the behavior of what is essentially a builder for new instances.
You could make a much cleaner and quite type safe solution that way. So we could define some marker interfaces and type safe versions thereof:
public interface IImmutable : ICloneable { }
public interface IImmutableBuilder { }
public interface IImmutableOf<T> : IImmutable where T : class, IImmutable
{
IImmutableBuilderFor<T> Mutate();
}
public interface IImmutableBuilderFor<T> : IImmutableBuilder where T : class, IImmutable
{
T Source { get; }
IImmutableBuilderFor<T> Set<TFieldType>(string fieldName, TFieldType value);
IImmutableBuilderFor<T> Set<TFieldType>(string fieldName, Func<T, TFieldType> valueProvider);
IImmutableBuilderFor<T> Set<TFieldType>(Expression<Func<T, TFieldType>> fieldExpression, TFieldType value);
IImmutableBuilderFor<T> Set<TFieldType>(Expression<Func<T, TFieldType>> fieldExpression, Func<TFieldType, TFieldType> valueProvider);
T Build();
}
And provide all the required basic builder behavior in a class like below. Note that most error checking/compiled delegate creation is omitted for the sake of brevity/simplicity. A cleaner, performance optimized version with a reasonable level of error checking can be found in this gist.
public class DefaultBuilderFor<T> : IImmutableBuilderFor<T> where T : class, IImmutableOf<T>
{
private static readonly IDictionary<string, Tuple<Type, Action<T, object>>> _setters;
private List<Action<T>> _mutations = new List<Action<T>>();
static DefaultBuilderFor()
{
_setters = GetFieldSetters();
}
public DefaultBuilderFor(T instance)
{
Source = instance;
}
public T Source { get; private set; }
public IImmutableBuilderFor<T> Set<TFieldType>(string fieldName, TFieldType value)
{
// Notes: error checking omitted & add what to do if `TFieldType` is not "correct".
_mutations.Add(inst => _setters[fieldName].Item2(inst, value));
return this;
}
public IImmutableBuilderFor<T> Set<TFieldType>(string fieldName, Func<T, TFieldType> valueProvider)
{
// Notes: error checking omitted & add what to do if `TFieldType` is not "correct".
_mutations.Add(inst => _setters[fieldName].Item2(inst, valueProvider(inst)));
return this;
}
public IImmutableBuilderFor<T> Set<TFieldType>(Expression<Func<T, TFieldType>> fieldExpression, TFieldType value)
{
// Error checking omitted.
var memberExpression = fieldExpression.Body as MemberExpression;
return Set<TFieldType>(memberExpression.Member.Name, value);
}
public IImmutableBuilderFor<T> Set<TFieldType>(Expression<Func<T, TFieldType>> fieldExpression, Func<TFieldType, TFieldType> valueProvider)
{
// Error checking omitted.
var memberExpression = fieldExpression.Body as MemberExpression;
var getter = fieldExpression.Compile();
return Set<TFieldType>(memberExpression.Member.Name, inst => valueProvider(getter(inst)));
}
public T Build()
{
var result = (T)Source.Clone();
_mutations.ForEach(x => x(result));
return result;
}
private static IDictionary<string, Tuple<Type, Action<T, object>>> GetFieldSetters()
{
// Note: can be optimized using delegate setter creation (IL).
return typeof(T).GetFields(BindingFlags.Public | BindingFlags.Instance)
.Where(x => !x.IsLiteral)
.ToDictionary(
x => x.Name,
x => SetterEntry(x.FieldType, (inst, val) => x.SetValue(inst, val)));
}
private static Tuple<Type, Action<T, object>> SetterEntry(Type type, Action<T, object> setter)
{
return Tuple.Create(type, setter);
}
}
Example usage
This could then be used like this, using your example class of State:
public static class Example
{
public class State : IImmutableOf<State>
{
public State(int someInt, string someString)
{
SomeInt = someInt;
SomeString = someString;
}
public readonly int SomeInt;
public readonly string SomeString;
public IImmutableBuilderFor<State> Mutate()
{
return new DefaultBuilderFor<State>(this);
}
public object Clone()
{
return base.MemberwiseClone();
}
public override string ToString()
{
return string.Format("{0}, {1}", SomeInt, SomeString);
}
}
public static void Run()
{
var original = new State(10, "initial");
var mutatedInstance = original.Mutate()
.Set("SomeInt", 45)
.Set(x => x.SomeString, "Hello SO")
.Build();
Console.WriteLine(mutatedInstance);
mutatedInstance = original.Mutate()
.Set(x => x.SomeInt, val => val + 10)
.Build();
Console.WriteLine(mutatedInstance);
}
}
With the following output:
45, Hello SO
20, initial
Well to answer your question about performance, reflection is very expensive (relatively speaking). I would not use your design if it's in performance critical code.
When it comes to generics and reflection the performance hit can often be surprisingly large. Consider even something as simple as this:
public class Builder<T> where T : new()
{
public T Build()
{
return new T();
}
}
What this is actually doing is calling Activator.CreateInstance which uses reflection and it's extremely expensive.
If I wanted to optimize code like the above case I would use dynamic methods. And the performance difference between the two would be drastic.
Of course, keep in mind we're entering the zone of advanced code that's more complex and harder to read for the sake of performance. You could consider this overly optimized and overkill in code that isn't performance critical.
But in code that I write I avoid reflection like the plague.
My favourite way to things like that is to use expression trees. You can manually construct your expression tree to just create a new instance of your type and compile this expression tree into a delegate. The beauty of this approach is that you only need reflection and dynamic code generation for once and afterwards you work with the generated delegate. Also, the expression compiler does its best to work even on partial trusted environments, where dynamic methods are problematic. On the other hand, you have an abstraction layer much higher than writing pure IL code in an ILGenerator, which would be the way to go in a dynamic method.

Quotes escape in Where clause in LINQ to Entites

I wonder how to escape quotes in LINQ to Entities.
This is my environment : Entity Framework 5 with Silverlight 5 and WCF RIA Services, MySQL 5.6 and MySQLConnector 6.5.6.
I've got the following query:
DomainContext.Load<Product>(DomainContext.GetProductQuery()
.Where<Product>(p => p.name.Contains(parameter))
.Take<Product>(30));
If the parameter variable contains a quote ' it raises a MySQL syntax error exception.
Whatever the method (StartWith, Contains) it always raises an exception.
This does the same using FilterDescriptor with a DomainDataSource.
Important note : It does not raise any exception with characters like % or double quote ". Also it does not raise any exception with a simple quote if the operator is equal strict like the following.
DomainDataSource.FilterDescriptors.Add(new FilterDescriptor("productName", FilterOperator.IsEqualTo, SelectedProductName));
or
DomainContext.Load<Product>(DomainContext.GetProductQuery()
.Where<Product>(p == parameter)
.Take<Product>(30));
I don't have any difficulties to insert data.
Any help will be much appreciated.
Thank you.
Update: I forgot to mention some things.
This is my method on service side.
public IQueryable<Product> GetProduct()
{
return this.ObjectContext.product;
}
How am I supposed to secure this against SQL injections ? Do I have to write dozens lines of code to manage filters ?
EDIT : Issue is solved in the last version of the MySQL provider for EF.
For a quick solution for your exact problem:
string cleanParameter = parameter.Replace("'", "\'")
OR
Take a look here for a more general solution that describes the C# equivalent of mysql_real_escape_string.
string cleanParameter = MySQLEscape(parameter)
MySQLEscape as described in the mentioned article:
private static string MySQLEscape(string str)
{
return Regex.Replace(str, #"[\x00'""\b\n\r\t\cZ\\%_]",
delegate(Match match)
{
string v = match.Value;
switch (v)
{
case "\x00": // ASCII NUL (0x00) character
return "\\0";
case "\b": // BACKSPACE character
return "\\b";
case "\n": // NEWLINE (linefeed) character
return "\\n";
case "\r": // CARRIAGE RETURN character
return "\\r";
case "\t": // TAB
return "\\t";
case "\u001A": // Ctrl-Z
return "\\Z";
default:
return "\\" + v;
}
});
}
Side note: your code sounds like it may be prone to SQL injection attacks. (I can't tell without more context). This article describes what sql injection attacks are and how to prevent them.
I've found a workaround for this issue.
First, I would like to thank tomlev from www.developpez.net for his solution as chamamo for his help too.
This is the direct link to the discussion in French
http://www.developpez.net/forums/d1349604/services-web/wcf-ria-services-injection-sql/
This is the source code of a wrapper to fix this issue.
class MySqlQueryableWrapper<T> : IQueryable<T>
{
private readonly IQueryable<T> _queryable;
private readonly IQueryProvider _provider;
public MySqlQueryableWrapper(IQueryable<T> queryable)
{
_queryable = queryable;
_provider = new MySqlQueryProviderWrapper(queryable.Provider);
}
public Type ElementType
{
get { return _queryable.ElementType; }
}
public Expression Expression
{
get { return _queryable.Expression; }
}
public IQueryProvider Provider
{
get { return _provider; }
}
public IEnumerator<T> GetEnumerator()
{
return _queryable.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
}
class MySqlQueryProviderWrapper : IQueryProvider
{
private readonly MySqlExpressionFixer _visitor = new MySqlExpressionFixer();
private readonly IQueryProvider _provider;
public MySqlQueryProviderWrapper(IQueryProvider provider)
{
_provider = provider;
}
public IQueryable CreateQuery(Expression expression)
{
return _provider.CreateQuery(_visitor.Visit(expression));
}
public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
{
return _provider.CreateQuery<TElement>(_visitor.Visit(expression));
}
public object Execute(Expression expression)
{
return _provider.Execute(_visitor.Visit(expression));
}
public TResult Execute<TResult>(Expression expression)
{
return _provider.Execute<TResult>(_visitor.Visit(expression));
}
}
class MySqlExpressionFixer : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if ((node.Method.Name == "Contains" || node.Method.Name == "StartsWith") &&
node.Method.DeclaringType == typeof(string) &&
node.Arguments.Count == 1)
{
var c = node.Arguments[0] as ConstantExpression;
if (c != null)
{
string s = c.Value as string;
if (s != null)
{
s = s.Replace("'", "''");
node = Expression.Call(node.Object, node.Method, Expression.Constant(s));
}
}
}
return base.VisitMethodCall(node);
}
}
Here is an example.
public IQueryable<Product> GetProduct()
{
return new MySqlQueryableWrapper<Product>(this.ObjectContext.product);
}

Categories