Sorry if the title is misleading, wasn't sure how to describe this one.
My end goal is to have an extension method of IQueryable<T> and some form (see below for example) of expression that will allow me to have to return an IQueryable<EntityIndex<T>> (or similar) which contains the original T in the Entity field, and an array/ienumerable containing the elements as describe by the some form of expression.
I know that doesn't really make sense, hopefully it will after an example...
This is what I have so far:
class EntityIndex<T, TKey>
{
T Entity { get; set; }
// Doesn't have to be IEnumerable, whatever is easier
IEnuermable<TKey> Index { get; set; }
}
static class Elsewhere
{
[Extension()]
public IQueryable<EntityIndex<T, TKey>> IndexBy<T, TKey>(this IQueryable<T> source, Expression<Func<T, TKey[]>> indexSelector)
{
return source.Select(n => new EntityIndex<T, TKey> {
Entity = n,
Index = new T[] { n }.Select(indexSelector)
});
}
}
Note: The above does not compile, it's simply there to try and show what I'm trying to achieve.
I've used the standard selector, but sub-optimally, had to arbitrarily create an array of T on the assignment to the 'Index' property to be able to apply the selector. I'm hoping a better choice of parameter may resolve this, but possibly not. The main issue is this doesn't compile so if there is a minor tweak that will allow it to work that's fine by me, if you can understand my gibberish and understand what I'm trying to do, and happen to know a better way to go about it I'd be greatly appreciative.
Ideally, I need the solution to be understood by the L2S engine, which I'm not convinced the above is thanks to the introduction of the EntityIndex class, but I'm holding out hope that it'll treat it as an anonymous class.
EDIT:
Good point Damien, the bigger picture is probably much easier to describe...
I want an extension method that accepts an expression, the expression should describe which fields on the entity to index, which will be used after this particular expression to allow a criterion (where clause) to be applied to the selected fields.
Long story short, in a number of places in code we have a wildcard string search. If I have an EntityA with Property1, Property2, Property3, etc, it is not uncommon to see code such as:
Handwritten, please excuse minor typos
public string[] WildcardSearch(string prefixText, int count)
{
string searchTerm = prefixText.Replace(wildcard, string.Empty);
if (prefixText.StartsWith(wildcard) && prefixText.EndsWith(wildcard)) {
return entitySet.Where(n => n.Property1.Contains(searchTerm) || n.Property2.Contains(searchTerm)).Select(n => n.Property3).ToArray();
} else if (prefixText.StartsWith(wildcard)) {
return entitySet.Where(n => n.Property1.EndsWith(searchTerm) || n.Property2.EndsWith(searchTerm)).Select(n => n.Property3).ToArray();
// you get the picture, same with EndsWith, no wildcards defaults to contains...
}
}
EDIT:
Further clarification - using the above WildcardEarch as an example, what I was hoping for was to be able to have a selector as follows or similar:
Func<EntityA, IEnumerable<string>> indexSelector = n => new string[] {
n.Property1,
n.Property2
};
// Alternatively, a ParamArray of keySelector might work?
Func<EntityA, string>[] keySelectors = new Func<EntityA, string>[] {
n => n.Property1,
n => n.Property2
};
Given an adequate expression describing which fields on the entity to search, returning the IQueryable<EntitySearch<T>> as shown above, I hoped to be able to apply a single criterion, similar to:
Func<EntitySearch<T>, bool> criterion = n => false;
if (wildcardIsContains) {
criterion = n => n.Values.Any(x => x.Contains(searchTerm));
} else if (wildCardIsStartsWith) {
criterion = n => n.Values.Any(x => x.Contains(searchTerm));
//etc
}
Given the extension at the very top that I can't get to work, and this criterion logic, I should be able to take an IQueryable<T>, select some fields, and apply an appropriate wildcard search on the fields, finally returning IQueryable<T> again having added the filtering.
Thanks¬!
Please comment if you need more information/clarification...
EDIT:
Fair one #subkamren and thanks for the interest. Some non-generic examples may be of use. I'll draft something up and add them shortly. For the time being, some clarification based on your comment...
Given an IQueryable<Animal> I want an extension allowing me to select fields on Animal which I intend to search/index by. For example, Animal.Description, Animal.Species.Name etc. This extension should return something like an IIndexedQueryable<Animal>. That is the issue I'm trying to deal with in the question above. The wider picture mentioned, which I'd be exceptionally pleased if you're willing to help with, is as follows:
The IIndexedQueryable<T> interface in turn I would like an extension for which could take a string search term. The extension should resolve the wildcards within the search term, extend the original IQueryable with the necessary criterion to perform a search on the indexed fields, and return an IQueryable<T> again.
I appreciate this could be done in a single step, but I hoped to do it this way so that later on I can look into adding a third extension method applicable to IIndexedQueryable<T> allowing me to perform a freetext search with SQL Server... ^^ Make any sense?
That's the bigger picture at least, this question deals primarily with being able to specify the fields I aim to index in such a way I can use them thereafter as mentioned here.
So something like:
public static IEnumerable<EntityIndex<T, Y>> IndexBy<T, Y>(this IEnumerable<T> entities, Func<T, Y> indexSelector) {
return entities.Select(e => new EntityIndex<T, Y> { Entity = e, IndexValue = indexSelector(e) });
}
Noting that generically defining EntityIndex with the TIndexType (called Y here) is important because you don't know ahead of time what the index is. The use of a generic allows Y to be an enumeration, thus the following would work as an index selector:
// Assuming Animal has attributes "Kingdom", "Phylum", "Family", "Genus", "Species"
// this returns an enumeration of EntityIndex<Animal, String[]>
var animalsClassified = someAnimals.IndexBy(a => new String[] { a.Kingdom, a.Phylum, a.Family, a.Genus, a.Species });
EDIT (Adding further detail):
Using the above, you can group the results by unique index value:
var animalClassifications = animalsClassified
.SelectMany(ac => ac.IndexValue.Select(iv => new { IndexValue = iv, Entity = ac.Entity }))
.GroupBy(kvp => kvp.IndexValue)
What I've described here, by the way, is (a very simplified form of) the MapReduce algorithm as popularized by Google. A distributed form of the same is commonly used for keyword identification in text search, where you want to build an index of (search term)->(list of containing documents).
Related
I am trying to use a custom Function inside a Linq Select that is used with EF.
I want to project each item of tblMitarbeiter onto one tblMitarbeiterPersonalkostenstelleHistories that is valid ad the given date.
This should be done with an extension method so that I do not repeat myself ;)
I can only get it to work when used directly on the DbSet, but not inside a Select.
How can I teach EF to recognize my Method (3.) as if I would be writing it out (1.)?
void Main()
{
var date = DateTime.Now;
// 1. works, returns IEnumerable<tblMitarbeiterPersonalkostenstelleHistories>
tblMitarbeiters
.Select(m => m.tblMitarbeiterPersonalkostenstelleHistories.Where(p => p.ZuordnungGültigAb <= date).OrderByDescending(p => p.ZuordnungGültigAb).FirstOrDefault())
.Dump();
// 2. works, returns one tblMitarbeiterPersonalkostenstelleHistories
tblMitarbeiterPersonalkostenstelleHistories
.GetValidItemForDate(p => p.ZuordnungGültigAb, date)
.Dump();
// 3. throws NotSupportedException
tblMitarbeiters
.Select(m => m.tblMitarbeiterPersonalkostenstelleHistories.GetValidItemForDate(p => p.ZuordnungGültigAb, date))
.Dump();
// 4. throws NotSupportedException
tblMitarbeiters
.Select(m => m.tblMitarbeiterPersonalkostenstelleHistories.AsQueryable().GetValidItemForDate(p => p.ZuordnungGültigAb, date))
.Dump();
}
public static class QueryableExtensions
{
public static T GetValidItemForDate<T>(this IQueryable<T> source, Expression<Func<T, DateTime>> selector, DateTime date)
{
var dateAccessor = Expression.Lambda<Func<T, DateTime>>(Expression.Constant(date), selector.Parameters);
var lessThanOrEqual = Expression.LessThanOrEqual(selector.Body, dateAccessor.Body);
var lambda = Expression.Lambda<Func<T, bool>>(lessThanOrEqual, selector.Parameters);
return source.Where(lambda).OrderByDescending(selector).FirstOrDefault();
}
public static T GetValidItemForDate<T>(this IEnumerable<T> source, Func<T, DateTime> selector, DateTime date) =>
source.Where(i => selector(i) <= date).OrderByDescending(selector).FirstOrDefault();
}
You can, to some extent, split up complex LINQ expressions using LINQKit. If you'll excuse me, I'll use an example model that's less germanic:
public class Employee
{
public long Id { get; set; }
public virtual ICollection<EmployeeHistoryRecord> HistoryRecords { get; set; }
}
public class EmployeeHistoryRecord
{
public long Id { get; set; }
public DateTime ValidFrom { get; set; }
public long EmployeeId { get; set; }
public Employee Employee { get; set; }
}
If I understood your question correctly, it should be identical to yours where it matters.
When using LINQKit, and LINQ in general, you must understand that the only tool you have at your disposal when reusing query code, without using stored procedures, is breaking apart and stitching together expressions.
Your utility method would translate to something like this:
private static Expression<Func<IEnumerable<TItem>, TItem>> GetValidItemForDate<TItem>(
Expression<Func<TItem, DateTime>> dateSelector,
DateTime date)
{
return Linq.Expr((IEnumerable<TItem> items) =>
items.Where(it => dateSelector.Invoke(it) <= date)
.OrderByDescending(it => dateSelector.Invoke(it))
.FirstOrDefault())
.Expand();
}
What this method does is dynamically create an expression whose input is an IEnumerable<TItem> that returns a TITem. You can see it's pretty similar to the code you're extracting. A few things to note:
The source collection is not a parameter of the utility method, but of the expression returned.
You have to call the Invoke() extension method from LinqKit on any expressions you're "plugging into" this one.
You should call Expand() on the result if you used any Invoke()s inside it. This will make LINQKit replace the calls to Invoke() in the expression tree with the expression being invoked. (This isn't 100% necessary, but it makes it easier to fix errors when expansion fails for some reason. If you don't Expand() in every helper method, any error that happens during expansion will manifest in the method that does the expansion, and not in the method that actually contains the offending code.)
You then use this similarly, again using Invoke():
var db = new EmployeeHistoryContext();
var getValidItemForDate = GetValidItemForDate((EmployeeHistoryRecord cab) => cab.ValidFrom, DateTime.Now);
var historyRecords = db.Employees.AsExpandable().Select(emp => getValidItemForDate.Invoke(emp.HistoryRecords));
(I've only tested this code against an empty database, insofar that it doesn't make EntityFramework throw a NotSupportedException.)
Here, you should note:
The subexpression you're plugging into the one you're passing into Select() needs to be saved in a local variable, LINQKit doesn't support method calls during expansion.
You need to call AsExpandable() on the first IQueryable in the chain, so LINQKit gets to work its magic.
You're probably not going to be able to use extension method call syntax inside the expression like in your question.
All the subexpressions have to be determined before expansion occurs.
These limitations stem from the fact that what you're doing isn't really calling methods. You're building one ginormous expression from a bunch of smaller ones, but the resulting expression itself still has to be something that LINQ-to-Entities will understand. On the other hand, the input has to be something LINQKit will understand, and it only handles expressions of the form localVariable.Invoke(). Any dynamism has to be in the code outside this expression tree. Basically, it's doing the same as your solution 2, just using syntax more intuitive than building the expression tree programmatically.
Last, but not least: when doing this, do not go overboard. Complex EF queries are already really hard to debug when anything goes wrong, because you're not told where in your code the problem is. If the query was assembled dynamically from bits and pieces all over your codebase, debugging some errors (like the delightful "Unable to cast the type X to type Y") will easily become a nightmare.
(For future questions: I think it's usually a good idea when if you make a code sample from scratch, instead of using bits from your actual codebase. They might be overly domain-specific, and understanding the names might require some context you take for granted. Identifiers should ideally be simple English names everyone can understand. I can maybe speak enough German to interview for a job in it, but "Mitarbeiterpersonalkostenstellehistorie" is just hard to keep in my head and reason about when I haven't actually worked on your project long enough to be familiar with what it's supposed to mean.)
Working with EPiServer Find and trying to build a generic facet funcionality for it to simplify managing which facet should be enabled. I would like to construct two generic methods, one for adding active filters to perform the hits search and one to perform the available facet filters remaining.
The first method will perform the following (specific code for brand filter):
var brandFilter = client.BuildFilter<FashionProduct>();
foreach (var facet in SelectedGroup.Facets.Where(x => x.Selected))
{
brandFilter = brandFilter.Or(x => x.Brand.Match(facet.Key));
}
query = query.Filter(brandFilter);
I would like to be able to call it in a generic way so I could base the available facets on some simple list of strings or objects. Like this:
query = AddFilterFacet<FashionProduct>(query, "Brand", SelectedGroup.Facets)
So the method would take the type of object to filter on, the query to append the filters on, the name of the property to filter on and the list of values to add.
The second method is similar but relates more to perform the following:
facetQuery = facetQuery.TermsFacetFor(x => x.Brand)
...
var brandFacets = facetResult.TermsFacetFor(x => x.Brand).Terms;
Is it possible to build this kind of functionality? The biggest questionmark I have is how to translate the "Brand" input string to be the Brand Property in x => x.Brand
private void AddFilterFacet<T>(IClient client, ref ITypeSearch<T> query, string propertyName, List<FacetOption> facets)
{
var filter = client.BuildFilter<T>();
foreach (var facet in facets)
{
filter = filter.Or(x => x.????.Match(facet.Key));
}
query = query.Filter(filter);
}
The .Or method takes a
System.Linq.Expressions.Expression<Func<T, Find.Api.Querying.Filter>>
so perhaps something can be used to make a proper generic call to it
It's definitely possible to create generic lambda expressions, it's just not easy and requires a lot of reflection code.
I haven't done it in a while, but maybe if you look at the code i created for something similar a while ago (Generic lambda expressions) it'll help. I'm sure someone with fresher experience will help you out here soon enough.
Decimal precision attribute <-- take a look a this answer witch has code to genereate modelBuilder.Entity<CLASS>().Property(OBJECT=> OBJECT.PROPERTY).HasPrecision(12, 10) automatically from an attribute in a class
Currently coding in C#, I wonder if there is a way to factor the code as presented below
Entity1 = GetByName("EntityName1");
Entity2 = GetByName("EntityName2");
Entity3 = GetByName("EntityName3");
The idea would be to get a single call in the code, factoring the code by placing the entities and the strings in a container and iterating on this container to get a single "GetByName()" line. Is there a way to do this?
You can use LINQ:
var names=new[]{"EntityName1","EntityName2","EntityName3",.....};
var entities=names.Select(name=>GetByName(name)).ToArray();
Without ToArray, Select will return an IEnumerable that will be reevalueated each time you enumerate it - that is, GetByName will be called each time you enumerate the enumerable.
ToArray() or ToList will create an array (or list) you can use multiple times.
You can also call ToDictionary if you want to be able to access the entities by name:
var entities=names.ToDictionary(name=>name,name=>GetByName(name));
All this assumes that the entities don't already exist or that GetByName has to do some significant work to retrieve them. If the entities exist you can simply put them in a Dictionary<String,Entity>. If the entities have a Name property you can use ToDictionary to create the dictionary in one statement:
var entities=new []{entity1,entity2,entity3,...};
var entitiesDict=entities.ToDictionary(entity=>entity.Name,entity=>entity);
Do you mean something like the below (where entities is the collection of Entity1, Entity1 & Entity3):
var results = entities.Select(e => GetByName(e.Name));
It depends on what you're looking for. If you need to set the variables in a single line, that won't work. You could play with reflection if you're dealing with fields or properties, but honestly that seems messier than what you've got already.
If the data-structure doesn't matter, and you just need the data and are able to play with it as you see so fit, I'd probably enumerate it into a dictionary. Of course, that's pretty tightly coupled to what you've got now, which looks like it's a fake implementation anyway.
If you want to do that, it's pretty straight-forward. It's your choice how you create the IEnumerable<string> that's represented below as entityNames. You could use an array initializer as I do, you could use a List<string> that you build over time, you could even yield return it in its own method.
var entityNames = new[] { "EntityName1", "EntityName2", "EntityName3" };
var dict = entityNames.ToDictionary(c => c, c => GetByName(c));
Then it's just a matter of checking those.
var entity1 = dict["EntityName1"];
Or enumerating over the dictionary.
foreach(var kvp in dict)
{
Console.WriteLine("{0} - {1}", kvp.Key, kvp.Value);
}
But realistically, it's hard to know whether that's preferable to what you've already got.
Ok, here is an idea.
You can declare this function.
IReadOnlyDictionary<string, T> InstantiateByName<T>(
Func<string, T> instantiator
params string[] names)
{
return names.Distinct().ToDictionary(
name => name,
name => instantiator(name))
}
which you could call like this,
var entities = InstantiateByName(
GetByName,
"EntityName1",
"EntityName2",
"EntityName3");
To push the over-engineering to the next level,
you can install the Immutable Collections package,
PM> Install-Package Microsoft.Bcl.Immutable
and modify the function slightly,
using Microsoft.Immutable.Collections;
IReadOnlyDictionary<string, T> InstantiateByName<T>(
Func<string, T> instantiator
params string[] names,
IEqualityComparer<string> keyComparer = null,
IEqualityComparer<T> valueComparer = null)
{
if (keyComparer == null)
{
keyComparer = EqualityComparer<string>.Default;
}
if (valueComparer == null)
{
valueComparer = EqualityComparer<T>.Default;
}
return names.ToImmutableDictionary(
name => name,
name => instantiator(name),
keyComparer,
valueComparer);
}
The function would be used in the exactly the same way. However, the caller is responsible for passing unique keys to the function but, an alternative equality comparer can be passed.
I'm reconstructing the following statement.
IEnumerable<String>
input = ...,
filter = ...,
output = input.Where(filter.Contains(element));
For now, it works as supposed to but the words matched this way need to be exact. In the language of my customer there are a lot of conjugations and a requirement is posted to use joker characters ("dog" should match "dog", "doggy" and "dogmatic").
I've suggested the following change. Now sure, though, if it can be regarded as smooth for the eyes. Can someone suggest an improvement or is it as good as it gets?
IEnumerable<String>
input = ...,
filter = ...,
output = input.Where(word => filter.Any(head => word.StartsWith(head)))
I was considering IEqualityComparer implementation but that's only for objects of the same type, while my condition is on String contra IEnumerable.
Generally, what you already have as your LINQ statement is fine and I don't see a big issue with about it being "smooth on the eyes" (LINQ calls can often get even more out of hand than this).
If you want, you could move the filter.Any(head => word.Startswith(head)) into a separate Func<string, bool> delegate and pass that in:
Func<string, bool> myConstraint = word => filter.Any(head => word.StartsWith(head));
output = input.Where(myConstraint);
You can also move the constraint construction to a separate method which may open the door to some flexibility with your client if matching rules change or have to cover even more complicated cases:
private Func<string, bool> BuildConstraints()
{
filter = ...,
if (CheckEqualityOnly)
return word => filter.Contains(word);
else
return word => filter.Any(head => word.StartsWith(head));
}
output = input.Where(BuildConstraints());
I'm looking to make certain functions as generic as possible.
In my MVC applications I have to convert a number of complex IEnumerable objects to SelectLists for dropdown display etc.
At first I created a static List class with conversion methods for each complex object. This amounted to a lot of code. I next started using linq statements to do the conversion:
var list = (from o in SessionTypes select new SelectListItem { Value = o.ID.ToString(), Text = o.Title }).ToList();
but again, this still was a lot of code when taken over many such conversions.
I finally settled something similar to:
public IEnumerable<SelectListItem> ToSelectList<T>(IEnumerable<T> enumerable, Func<T, string> value, Func<T, string> text)
{
return enumerable.Select(f => new SelectListItem()
{
Value = value(f),
Text = text(f),
});
}
and to make it easier to use, I made it an extension method:
public static IEnumerable<SelectListItem> ToSelectList<T>(this IEnumerable<T> enumerable, Func<T, string> value, Func<T, string> text)
{
return enumerable.Select(f => new SelectListItem()
{
Value = value(f),
Text = text(f),
});
}
So now, all I have to do is:
var list = SessionTypes.ToSelectList(o => o.ID.ToString(), o => o.Title) as List<SelectListItem>;
I have similar methods such as .ToDictionary too.
Is this overusing Extension methods? I worry that I'm hiding too much code away in extension methods which might be distorting my models, would it be better to use linq conversions for transparency?
You can bind IDictionary<,> to the DropDownList DataSource directly, in WebForms you need to specify mappings like DataValueField="Key" and DataTextField="Value". Considering that why not just use LINQ ToDictionary() method?
dropDownList.DataValueField = "Key";
dropDownList.DataTextField = "Value";
dropDownList.DataSource =
SessionTypes.ToDictionary(k => k.ID.ToString(),
v => v.Title);
This guideline is from C# in Depth :
"An extension method is reasonably valid if it's applicable to all
instances of the extended type. If it's only appropriate in certain
situations, then make it clear that the method is not part of type by
leaving it as a "normal" static method".
As long as you are okey in exposing this extension method on all instances of type IEnumerable, then you are good to go.
On a lighter note, I would rather call the extension method as ToSelectListItem, rather than ToSelectList, to make it clear for the end user. Again this is my personal preference.
This is a perfectly acceptable usage of extension methods.
So long as they are descriptively named and do what they say you should not have problems.