We Keep Coding

Expression to get LINQ with Contains to EF for SQL IN() where on entities child's property equals value

I have a simple need to filter all parents out of the returned collection where there is no match on a field, that is called by name from a string, doesn't match a value presented. What I am after is if parent object has child object, and that child objects property "foo"(called by string) doesn't or does equal a value bar, the parent object is filtered from the collection appropriately.
Here is my linq ef call
var field = "bar";
var values = new List<string>{"foo","fuYu"};
var dataPage = _aim_context.ae_s_bld_c.AsNoTracking();
var result = dataPage.Where(x =>
DbHelper.byPropertyContains(x.udfs, field, values)
);
// NOTE `udfs` is a ONE-to-ONE with `ae_s_bld_c`
What I am looking to see is something like the SQL of
SELECT [m].[id],[m.udfs].[bar],
FROM [dbo].[ae_s_bld_c] AS [m]
INNER JOIN [dbo].[ae_s_bld_c_udf] AS [m.udfs]
ON ([m].[multitenant_id] = [m.udfs].[multitenant_id])
WHERE ([m].[multitenant_id] = 1.0)
AND ([m.udfs].[bar] IN ('foo','fuYu')) --< Goal line
The way I have approached this was to get an expression set up to take the List<string> and make the SQL. I have read near 50 articles and SO posts, but have not figured out exactly why I am not getting this just yet as everyone seems to have different ideas, and most are not in line with dotnet core 2.1+ it seems.
Here is what I am sitting at currently after many many iterations. NOTE: it is a little different from what I am after as I am giving my current trail.
My current context linq try
//...
dataPage = dataPage.Where(DbHelper.byPropertyContains<ae_s_bld_c>("udfs", field, values));
//...
I think it would be better if it was like the first example I put up, but that was what I have landed on since I have had a time lining it up with x=>x.udfs, both as x=> funName(x.udfs) and x=> x.udfs.funName()
My static method to build the expression
public static class DbHelper
{
public static Expression<Func<T, bool>> byPropertyContains<T>(string node, string field, List<string> value) {
//trying to take parent item and get it's property by string name because
// doing the function in linq like x=>x.udfs was not working right
// but that is the prefered I think
var property_parameter = Expression.Parameter(typeof(T), "x");
var property = Expression.PropertyOrField(property_parameter, node);
var selector_parameter = Expression.Parameter(property.Type, "y");
var selector = Expression.PropertyOrField(selector_parameter, field);
var methodInfo = typeof(List<string>).GetMethod("Contains", new Type[] {
typeof(string)
});
var list = Expression.Constant(value, typeof(List<string>));
var body = Expression.Call(methodInfo, list, selector);
return Expression.Lambda<Func<T, bool>>(body, selector_parameter);
}
}
Update
Per the request of #NetMage I have tried to work backwards with LINQpad. I think I am close but it is hard to tell with teh output. I am putting it up here for reference. To be clear, the property name of the child will be a string of the name. The best outcome is I could have a name like udfs.foo where I can test on any level if the values contain by string name, but really ok with it starting here,
var result = dataPage.Where(x =>
DbHelper.byPropertyContains(x.udfs, field, values)
);

Let start from here. You need an equivalent of something like this
var result = dataPage.Where(x => values.Contains(x.udfs.{field}));
where field is a string returning property dynamically specified by name.
In EF Core you don't even need to deal with building expresions by hand, because EF Core provides a special SQL translatable function for accessing simple properties by name called EF.Property.
With that method the solution is simple as that:
var result = dataPage
.Where(x => values.Contains(EF.Property<string>(x.udfs, field)));

Can't find property or field on a dynamic object

I using Dynamic Linq to do some database queries and it's working really well up to now. I can pass a string to a Select to select fields like so:
var output = myDatabaseTable.Select("Foo, Bar");
For example. The power obviously being when you pass a string variable rather than hardcoded strings. The problem I'm running into now is that the library using IEnumerable instead of IEnumerable<T> because, obviously, it can't know T until runtime. I'm using this to select data and eventually return it to a client and it works fine for spitting out raw data, but now I want to be able to do some more processing before returning the data and that requires getting the query to run on the database first. I can do something like this:
var materializedResults = output.Cast<dynamic>().ToList();
And that will make the query run. But the problem is, once I've done that, it seems I can't use dynamic linq anymore. For example, if I did something like this:
var foos = materializedResults.Select("Foo");
I now get a System.Linq.Dynamic.ParseException with the message No property of field 'Foo' exists in type 'Object' (Note: I can see in the debugger that the materializedResults does actually have all the expected properties).
So after casting to a List so I can potentially iterate through it and modify some of the values, I can no longer query it.
So my question is, how can I take a dynamic query (with select, group by, order by etc provided as strings), materialize the results and then actually process those result dynamically?
I thought maybe if I could cast to the actual type rather than dynamic it might work, so I tried this:
var d = output.Cast<dynamic>().ToList();
MethodInfo method = typeof(Enumerable).GetMethod("Cast", new[] {typeof(IEnumerable)});
method = method.MakeGenericMethod(d.First().GetType());
output = method.Invoke(d, new[] {d}) as IEnumerable;
Which is ugly and requires me to cast twice. The first time to dynamic so I can get the type from the first item then again to that type.

If you do YourStuff.Cast<dynamic>.ToList(), you will receive IEnumerable<object>, and there is no property Foo on the type object.
The question you might be asking, how can you get IList<TheActualType>?! You can do it this way:
// for IEnumerable
public static IList ToAnonymousList(this IEnumerable enumerable)
{
var enumerator = enumerable.GetEnumerator();
if (!enumerator.MoveNext())
throw new Exception("?? No elements??");
var value = enumerator.Current;
var returnList = (IList) typeof (List<>)
.MakeGenericType(value.GetType())
.GetConstructor(Type.EmptyTypes)
.Invoke(null);
returnList.Add(value);
while (enumerator.MoveNext())
returnList.Add(enumerator.Current);
return returnList;
}
// for IQueryable
public static IList ToAnonymousList(this IQueryable source)
{
if (source == null) throw new ArgumentNullException("source");
var returnList = (IList) typeof (List<>)
.MakeGenericType(source.ElementType)
.GetConstructor(Type.EmptyTypes)
.Invoke(null);
foreach (var elem in source)
returnList.Add(elem);
return returnList;
}
It's a simple extension method that can later be used, as such:
var test = (new[]
{
new
{
Property1 = "10",
Property2 = "10",
Property3 = 1
}
}
.Select("New(Property1, Property2)"))
.ToAnonymousList();

Your Cast is Defaulting to which will cause an error
(output as System.Collections.Generics.IEnumerable)
This Cast Specified the Correct Interface Try again
(output as System.Collections.IEnumerable).Cast<dynamic>().ToList()

How to swap the data source associated with a Linq query?

I have read in other answers how the paramters of a linq query can be changed at runtime. But is it possible to change the data source that the query browses through after the query has been created? Can perhaps the query be given an empty wrapper in which data sources can be plugged in and out, unbeknownst to the query?
For example, let's assume this situation:
// d1 is a dictionary
var keys = from entry in d1
where entry.Value < 5
select entry.Key;
Now, let's assume rather than modifying d1 I want myQuery to stay the same, except I'd like it to process an entirely new dictionary, d2.
The reason for this is that I'd like to decouple who provides the query from who provides the data source. I.e. think the dictionaries as metadata associated with services and the query as the mean for the consumers of services to discover which set of services match their criteria. I need to apply the same query to the metadata of each service.
I guess one parallel that I'm (maybe erroneously) making is with Regular Expression: one can compile a regular expression and then apply it to any string. I'd like to do the same with queries and dictionaries.

A LINQ query is really nothing other than a method call. As such, you can't "reassign" the object to which you've already called a method after it's been setup.
You could simulate this by creating a wrapper class that would allow you to change what actually gets enumerated, ie:
public class MutatingSource<T> : IEnumerable<T>
{
public MutatingSource(IEnumerable<T> originalSource)
{
this.Source = originalSource;
}
public IEnumerable<T> Source { get; set; }
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
public IEnumerator<T> GetEnumerator()
{
return Source.GetEnumerator();
}
}
This would allow you to create the query, then "change your mind" after the fact, ie:
var someStrings = new List<string> { "One", "Two", "Three" };
var source = new MutatingSource<string>(someStrings);
// Build the query
var query = source.Where(i => i.Length < 4);
source.Source = new[] {"Foo", "Bar", "Baz", "Uh oh"};
foreach(var item in query)
Console.WriteLine(item);
This will print Foo, Bar, and Baz (from the "changed" source items).
Edit in response to comments/edit:
I guess one parallel that I'm (maybe erroneously) making is with Regular Expression: one can compile a regular expression and then apply it to any string. I'd like to do the same with queries and dictionaries.
A query is not like a regular expression, in this case. The query is translated directly into a series of method calls, which will only work against the underlying source. As such, you can't change that source (there isn't a "query object", just the return value of a method call).
A better approach would be to move the query into a method, ie:
IEnumerable<TKey> QueryDictionary<TKey,TValue>(IDictionary<TKey,TValue> dictionary)
{
var keys = from entry in dictionary
where entry.Value < 5
select entry.Key;
return keys;
}
You could then use this as needed:
var query1 = QueryDictionary(d1);
var query2 = QueryDictionary(d2);

Linq-to-SQL: Combining (OR'ing) multiple "Contains" filters?

I'm having some trouble figuring out the best way to do this, and I would appreciate any help.
Basically, I'm setting up a filter that allows the user to look at a history of audit items associated with an arbitrary "filter" of usernames.
The datasource is a SQL Server data base, so I'm taking the IQueryable "source" (either a direct table reference from the db context object, or perhaps an IQueryable that's resulted from additional queries), applying the WHERE filter, and then returning the resultant IQueryable object....but I'm a little stumped as to how to perform OR using this approach.
I've considered going the route of Expressions because I know how to OR those, but I haven't been able to figure out quite how to do that with a "Contains" type evaluation, so I'm currently using a UNION, but I'm afraid this might have negative impact on performance, and I'm wondering if it may not give me exactly what I need if other filters (in addition to user name filtering shown here) are added in an arbirary order.
Here is my sample code:
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
// Take allowed values...
List<string> searchStrings = new List<string>();
// <SNIP> (This just populates my list of search strings)
IQueryable<X> oReturn = null;
// Step through each iteration, and perform a 'LIKE %value%' query
string[] searchArray = searchStrings.ToArray();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
if (i == 0)
// For first step, perform direct WHERE
oReturn = source.Where(x => x.Username.Contains(value));
else
// For additional steps, perform UNION on WHERE
oReturn = oReturn.Union(source.Where(x => x.Username.Contains(value)));
}
return oReturn ?? source;
}
This feels like the wrong way to do things, but it does seem to work, so my question is first, is there a better way to do this? Also, is there a way to do a 'Contains' or 'Like' with Expressions?
(Editted to correct my code: In rolling back to working state in order to post it, I apparently didn't roll back quite far enough :) )
=============================================
ETA: Per the solution given, here is my new code (in case anyone reading this is interested):
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
List<string> searchStrings = new List<string>(AllowedValues);
// <SNIP> build collection of search values
string[] searchArray = searchStrings.ToArray();
Expression<Func<X, bool>> expression = PredicateBuilder.False<X>();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
expression = expression.Or(x => x.Username.Contains(value));
}
return source.Where(expression);
}
(One caveat I noticed: Following the PredicateBuilder's example, an empty collection of search strings will return false (false || value1 || ... ), whereas in my original version, I was assuming an empty list should just coallesce to the unfiltered source. As I thought about it more, the new version seems to make more sense for my needs, so I adopted that)
=============================================

You can use the PredicateBuilder from the LINQkit to dynamically construct your query.

Is there any way to delay- execute a delegate against an IQueryable<T> after/during execution?

I expose an IQueryable method from my business layer for use in other layers. I would like to execute a function against each of the items in the enumeration, once the query has executed down-level.
It seems like there should be an event that is raised after the query executes, so that I can then operate on the results from this common layer.
Something like:
public IQueryable<User> Query()
{
return _Repository.Query<User>().ForEachDelayed(u=> AppendData(u));
}
I want the ForEachDelayed function to return the IQueryable without executing the query. The idea is, once the query is executed, the results are passed through this delegate.
Is there something like this out there? If not, is there an event like "IQueryable.OnExecute" that I can subscribe to?
Any help would be awesome -- thanks!
EDIT:
I thought I had the answer with this:
var users = from u in _Repository.Query<User>()
select AppendData(u);
return users;
But now, I get the following error:
Method 'AppendData(User)' has no supported translation to SQL.
I really need a delegate to run AFTER the query has executed.

The Easy Way
Another option that is much simpler would be to append what you need using an Iterator, see the example below. The downside of this approach is that everything would have been pulled into memory while the iteration is performed so that AppendData is only executed on the returned data. This is similar to the option by #Chris but you are not forced to create a new Func and Select query every time you want to use the method.
static void Main(string[] args)
{
var data = new List<int> { 1, 2, 3, 4, 5 }.AsQueryable().AppendData();
Console.WriteLine("Before execute");
Console.Write("{ ");
foreach (var x in data)
Console.Write("{0}, ", x);
Console.WriteLine( "}");
Console.WriteLine("After execute");
Console.Read();
}
// Append extra data. From this point on everything is in memory
// This should be generic, i.e. AppendData<T>, but I an easy way to "append"
static IEnumerable<int> AppendData(this IEnumerable<int> data)
{
Console.WriteLine("Adding");
foreach (var x in data)
yield return x * 2;
}
The Hard Way
I do not believe there are any built in events like this but you could probably add your own by implementing your own IQueryable wrapper.
Here is a series on writing a complete IQueryable provider which will also show you where you could fit your wrapper, most likely around that IQueryProvider.
Good luck and Hope this helps.

Have you had a look at ToExpandable()?
http://tomasp.net/blog/linq-expand.aspx
You could always call ToList() before executing your code - that will invoke the SQL, then you can run your code on the result set -
var users = from u in _Repository.Query<User>().ToList();
//this will invoke AppendData() for each u
var retVal = from u in users
select AppendData(u);
return retVal;

Right now you are using some ORM, correct? I'm pretty sure you can achieve this using the lambda expression form by explicitly using the IEnumerable version of select, instead of IQueryable.
var users = (_Repository.Query<User>() as IEnumerable<User>).Select(u => AppendDate(u));
return users;
When the query is actually executed it will fetch all of the data returned by Query() and then do the select in memory.

I haven't tried but I think you could use automapper to map to a different or similar object returning IQuerible and use "AfterMap" on the mapping to execute your code after the query is done.