I have read in other answers how the paramters of a linq query can be changed at runtime. But is it possible to change the data source that the query browses through after the query has been created? Can perhaps the query be given an empty wrapper in which data sources can be plugged in and out, unbeknownst to the query?
For example, let's assume this situation:
// d1 is a dictionary
var keys = from entry in d1
where entry.Value < 5
select entry.Key;
Now, let's assume rather than modifying d1 I want myQuery to stay the same, except I'd like it to process an entirely new dictionary, d2.
The reason for this is that I'd like to decouple who provides the query from who provides the data source. I.e. think the dictionaries as metadata associated with services and the query as the mean for the consumers of services to discover which set of services match their criteria. I need to apply the same query to the metadata of each service.
I guess one parallel that I'm (maybe erroneously) making is with Regular Expression: one can compile a regular expression and then apply it to any string. I'd like to do the same with queries and dictionaries.
A LINQ query is really nothing other than a method call. As such, you can't "reassign" the object to which you've already called a method after it's been setup.
You could simulate this by creating a wrapper class that would allow you to change what actually gets enumerated, ie:
public class MutatingSource<T> : IEnumerable<T>
{
public MutatingSource(IEnumerable<T> originalSource)
{
this.Source = originalSource;
}
public IEnumerable<T> Source { get; set; }
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
public IEnumerator<T> GetEnumerator()
{
return Source.GetEnumerator();
}
}
This would allow you to create the query, then "change your mind" after the fact, ie:
var someStrings = new List<string> { "One", "Two", "Three" };
var source = new MutatingSource<string>(someStrings);
// Build the query
var query = source.Where(i => i.Length < 4);
source.Source = new[] {"Foo", "Bar", "Baz", "Uh oh"};
foreach(var item in query)
Console.WriteLine(item);
This will print Foo, Bar, and Baz (from the "changed" source items).
Edit in response to comments/edit:
I guess one parallel that I'm (maybe erroneously) making is with Regular Expression: one can compile a regular expression and then apply it to any string. I'd like to do the same with queries and dictionaries.
A query is not like a regular expression, in this case. The query is translated directly into a series of method calls, which will only work against the underlying source. As such, you can't change that source (there isn't a "query object", just the return value of a method call).
A better approach would be to move the query into a method, ie:
IEnumerable<TKey> QueryDictionary<TKey,TValue>(IDictionary<TKey,TValue> dictionary)
{
var keys = from entry in dictionary
where entry.Value < 5
select entry.Key;
return keys;
}
You could then use this as needed:
var query1 = QueryDictionary(d1);
var query2 = QueryDictionary(d2);
Related
so basically, I have a project which uses EF Core. In order to shorten my lambdas when comparing if two objects (class Protocol) are equal, I've overridden my Equals method and overloaded the == and != operators. However, LINQ doesn't seem to care about it, and still uses reference for determining equality. Thanks
As I've said before, I've overridden the Equals method and overloaded the == and != operators. With no luck. I've also tried implementing the IEquatable interface. Also no luck.
I am using:
EF Core 2.2.4
//the protocol class
[Key]
public int ProtocolId {get;set;}
public string Repnr {get;set;}
public string Service {get;set;}
public override bool Equals(object obj)
{
if (obj is Protocol other)
{
return this.Equals(other);
}
return false;
}
public override int GetHashCode()
{
return $"{Repnr}-{Service}".GetHashCode();
}
public bool Equals(Protocol other)
{
return this?.Repnr == other?.Repnr && this?.Service == other?.Service;
}
public static bool operator ==(Protocol lhs, Protocol rhs)
{
return lhs.Equals(rhs);
}
public static bool operator !=(Protocol lhs, Protocol rhs)
{
return !lhs.Equals(rhs);
}
//the problem
using (var db = new DbContext())
{
var item1 = new Protocol() { Repnr = "1666", Service = "180" };
db.Protocols.Add(item1 );
db.SaveChanges();
var item2 = new Protocol() { Repnr = "1666", Service = "180" };
var result1 = db.Protocols.FirstOrDefault(a => a == item2);
var result2 = db.Protocols.FirstOrDefault(a => a.Equals(item2));
//both result1 and result2 are null
}
I would expect both result1 and result2 to be item1. However, they're both null. I know I could just do a.Repnr == b.Repnr && a.Service == b.Service, but that just isn't as clean. Thanks
To understand why the incorrect equality comparer is used, you have to be aware about the difference between IEnumerable<...> and IQueryable<...>.
IEnumerable
An object that implements IEnumerable<...>, is an object that represents a sequence of similar objects. It holds everything to fetch the first item of the sequence, and once you've got an item of the sequence you can get the next item, as long as there is a next item.
You start enumerating either explicitly by calling GetEnumerator() and repeatedly call MoveNext(). More common is to start enumerating implicitly by using foreach, or LINQ terminating statements like ToList(), ToDictionary(), FirstOrDefault(), Count() or Any(). This group of LINQ methods internally uses either foreach, or GetEnumerator() and MoveNext() / Current.
IQueryable
An object that implements IQueryable<...> also represents an enumerable sequence. The difference however, is that this sequence usually is not held by your process, but by a different process, like a database management system.
The IQueryable does not (necessarily) hold everything to enumerate. Instead it holds an Expression and a Provider. The Expression is a generic description about what must be queried. The Provider knows which process will execute the query (usually a database management system) and how to communicate with this process (usually something SQL-like).
An IQueryable<..> also implements IEnumerable<..>, so you can start enumerating the sequence as if it was a standard IEnumerable. Once you start enumerating an IQueryable<...> by calling (internally) GetEnumerator(), the Expression is sent to the Provider, who translates the Expression into SQL and executes the query. The result is presented as an enumerator, which can be enumerated using MoveNext() / Current.
This means, that if you want to enumerate an IQueryable<...>, the Expression must be translated into a language that the Provider supports. As the compiler does not really know who will execute the query, the compiler can't complain if your Expression holds methods or classes that your Provider doesn't know how to translate to SQL. In such cases you'll get a run-time error.
It is easy to see, that SQL does not know your own defined Equals method. In fact, there are even several standard LINQ functions that are not supported. See Supported and Unsupported LINQ Methods (LINQ to Entities).
So what should I do if I want to use an unsupported function?
One of the things that you could do is move the data to your local process, and then call the unsupported function.
This can be done using ToList, but if you will only use one or a few of the fetched items, this would be a waste of processing power.
One of the slower parts of a database query is the transport of the selected data to your local process. Hence it is wise to limit the data to the data that you actually plan to use.
A smarter solution would be to use AsEnumerable. This will fetch the selected data "per page". It will fetch the first page, and once you've enumerated through the fetched page (using MoveNext), it will fetch the next page.
So if you only use a few of the fetched items, you will have fetched some items that are not used, but at least you won't have fetched all of them.
Example
Suppose you have a local function that takes a Student as input and returns a Boolean
bool HasSpecialAbility(Student student);
Requirement: give me three Students that live in New York City that have the special Ability.
Alas, HasSpecialAbility is a local function, it can't be translated into Sql. You'll have to get the Students to your local process before calling it.
var result = dbContext.Students
// limit the transported data as much as you can:
.Where(student => student.CityCode == "NYC")
// transport to local process per page:
.AsEnumerable()
// now you can call HasSpecialAbility:
.Where(student => HasSpecialAbility(student))
.Take(3)
.ToList();
Ok, you might have fetched a page of 100 Students while you only needed 3, but at least you haven't fetched all 25000 students.
Currently coding in C#, I wonder if there is a way to factor the code as presented below
Entity1 = GetByName("EntityName1");
Entity2 = GetByName("EntityName2");
Entity3 = GetByName("EntityName3");
The idea would be to get a single call in the code, factoring the code by placing the entities and the strings in a container and iterating on this container to get a single "GetByName()" line. Is there a way to do this?
You can use LINQ:
var names=new[]{"EntityName1","EntityName2","EntityName3",.....};
var entities=names.Select(name=>GetByName(name)).ToArray();
Without ToArray, Select will return an IEnumerable that will be reevalueated each time you enumerate it - that is, GetByName will be called each time you enumerate the enumerable.
ToArray() or ToList will create an array (or list) you can use multiple times.
You can also call ToDictionary if you want to be able to access the entities by name:
var entities=names.ToDictionary(name=>name,name=>GetByName(name));
All this assumes that the entities don't already exist or that GetByName has to do some significant work to retrieve them. If the entities exist you can simply put them in a Dictionary<String,Entity>. If the entities have a Name property you can use ToDictionary to create the dictionary in one statement:
var entities=new []{entity1,entity2,entity3,...};
var entitiesDict=entities.ToDictionary(entity=>entity.Name,entity=>entity);
Do you mean something like the below (where entities is the collection of Entity1, Entity1 & Entity3):
var results = entities.Select(e => GetByName(e.Name));
It depends on what you're looking for. If you need to set the variables in a single line, that won't work. You could play with reflection if you're dealing with fields or properties, but honestly that seems messier than what you've got already.
If the data-structure doesn't matter, and you just need the data and are able to play with it as you see so fit, I'd probably enumerate it into a dictionary. Of course, that's pretty tightly coupled to what you've got now, which looks like it's a fake implementation anyway.
If you want to do that, it's pretty straight-forward. It's your choice how you create the IEnumerable<string> that's represented below as entityNames. You could use an array initializer as I do, you could use a List<string> that you build over time, you could even yield return it in its own method.
var entityNames = new[] { "EntityName1", "EntityName2", "EntityName3" };
var dict = entityNames.ToDictionary(c => c, c => GetByName(c));
Then it's just a matter of checking those.
var entity1 = dict["EntityName1"];
Or enumerating over the dictionary.
foreach(var kvp in dict)
{
Console.WriteLine("{0} - {1}", kvp.Key, kvp.Value);
}
But realistically, it's hard to know whether that's preferable to what you've already got.
Ok, here is an idea.
You can declare this function.
IReadOnlyDictionary<string, T> InstantiateByName<T>(
Func<string, T> instantiator
params string[] names)
{
return names.Distinct().ToDictionary(
name => name,
name => instantiator(name))
}
which you could call like this,
var entities = InstantiateByName(
GetByName,
"EntityName1",
"EntityName2",
"EntityName3");
To push the over-engineering to the next level,
you can install the Immutable Collections package,
PM> Install-Package Microsoft.Bcl.Immutable
and modify the function slightly,
using Microsoft.Immutable.Collections;
IReadOnlyDictionary<string, T> InstantiateByName<T>(
Func<string, T> instantiator
params string[] names,
IEqualityComparer<string> keyComparer = null,
IEqualityComparer<T> valueComparer = null)
{
if (keyComparer == null)
{
keyComparer = EqualityComparer<string>.Default;
}
if (valueComparer == null)
{
valueComparer = EqualityComparer<T>.Default;
}
return names.ToImmutableDictionary(
name => name,
name => instantiator(name),
keyComparer,
valueComparer);
}
The function would be used in the exactly the same way. However, the caller is responsible for passing unique keys to the function but, an alternative equality comparer can be passed.
I have to write a query in a web application using LINQ but I need to change that query into an array list. How can I change the query below to do this?
var resultsQuery =
from result in o["SearchResponse"]["Web"]["Results"].Children()
select new
{
Url = result.Value<string>("Url").ToString(),
Title = result.Value<string>("Title").ToString(),
Content = result.Value<string>("Description").ToString()
};
If you really need to create an ArrayList, you can write new ArrayList(resultsQuery.ToArray()).
However, you should use a List<T> instead, by writing resultsQuery.ToList().
Note that, in both cases, the list will contain objects of anonymous type.
There is a .ToArray() method that'll convert IEnumerable to an Array.
ArrayList doesn't have a constructor or Add(Range) method that takes an IEnumerable. So that leaves two choices:
Use an intermediate collection that does implement ICollection: as both Array and List<T> implement ICollection can be used via the ToArray() or ToList() extension methods from LINQ.
Create an instance of ArrayList and then add each element of the result:
var query = /* LINQ Expression */
var res = new ArrayList();
foreach (var item in query) {
res.Add(item);
}
The former method is simple to do but does mean creating the intermediate data structure (which of the two options has a higher overhead is an interesting question and partly depends on the query so there is no general answer). The latter is more code and does involve growing the ArrayList incrementally (so more memory for the GC, as would be the case for an intermediate Array or List<T>).
If you just need this in one place you can just do the code inline, if you need to do it in multiple places create your own extension method over IEnumerable<T>:
public static class MyExtensions {
public static ArrayList ToArrayList<T>(this IEnumerable<T> input) {
var col = input as ICollection;
if (col != null) {
return new ArrayList(col);
}
var res = new ArrayList();
foreach (var item in input) {
res.Add(item);
}
return res;
}
}
I'm having some trouble figuring out the best way to do this, and I would appreciate any help.
Basically, I'm setting up a filter that allows the user to look at a history of audit items associated with an arbitrary "filter" of usernames.
The datasource is a SQL Server data base, so I'm taking the IQueryable "source" (either a direct table reference from the db context object, or perhaps an IQueryable that's resulted from additional queries), applying the WHERE filter, and then returning the resultant IQueryable object....but I'm a little stumped as to how to perform OR using this approach.
I've considered going the route of Expressions because I know how to OR those, but I haven't been able to figure out quite how to do that with a "Contains" type evaluation, so I'm currently using a UNION, but I'm afraid this might have negative impact on performance, and I'm wondering if it may not give me exactly what I need if other filters (in addition to user name filtering shown here) are added in an arbirary order.
Here is my sample code:
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
// Take allowed values...
List<string> searchStrings = new List<string>();
// <SNIP> (This just populates my list of search strings)
IQueryable<X> oReturn = null;
// Step through each iteration, and perform a 'LIKE %value%' query
string[] searchArray = searchStrings.ToArray();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
if (i == 0)
// For first step, perform direct WHERE
oReturn = source.Where(x => x.Username.Contains(value));
else
// For additional steps, perform UNION on WHERE
oReturn = oReturn.Union(source.Where(x => x.Username.Contains(value)));
}
return oReturn ?? source;
}
This feels like the wrong way to do things, but it does seem to work, so my question is first, is there a better way to do this? Also, is there a way to do a 'Contains' or 'Like' with Expressions?
(Editted to correct my code: In rolling back to working state in order to post it, I apparently didn't roll back quite far enough :) )
=============================================
ETA: Per the solution given, here is my new code (in case anyone reading this is interested):
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
List<string> searchStrings = new List<string>(AllowedValues);
// <SNIP> build collection of search values
string[] searchArray = searchStrings.ToArray();
Expression<Func<X, bool>> expression = PredicateBuilder.False<X>();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
expression = expression.Or(x => x.Username.Contains(value));
}
return source.Where(expression);
}
(One caveat I noticed: Following the PredicateBuilder's example, an empty collection of search strings will return false (false || value1 || ... ), whereas in my original version, I was assuming an empty list should just coallesce to the unfiltered source. As I thought about it more, the new version seems to make more sense for my needs, so I adopted that)
=============================================
You can use the PredicateBuilder from the LINQkit to dynamically construct your query.
I have a method that gets rows from my database. It looks like this:
public static IEnumerable<Dictionary<string, object>> GetRowsIter()
{
_resultSet.ReadFirst();
do
{
var resultList = new Dictionary<string, object>();
for (int fieldIndex = 0; fieldIndex < _resultSet.FieldCount; fieldIndex++)
{
resultList.Add(_resultSet.GetName(fieldIndex),
_resultSet.GetValue(fieldIndex));
}
yield return resultList;
} while (_resultSet.ReadRelative(1));
yield break;
}
This is great when I want to return all the rows. But sometimes I want to return only some of the rows.
I am planning on writing my own method (GetRowsIterWhere(string column, object whereValue)), but I am wondering if I can use the linq where on my method.
I admit I don't know how it would work, becuase I am advancing the reader with a ReadRelative(1) to get to the next value. So even if it "thinks" it is skipping rows, it will not really skip them.
I am really concerned with performance here (I am currently refactoring from Linq-To-Datasets because it was way way way too slow.)
So, my question is, do I need to write my own Where type method or can I change the one above to work with the linq where method?
Yes you can use LINQ Where, but you'll need to build the predicate yourself. It isn't tricky. Something like (from memory; no compiler to hand):
var param = Expression.Parameter(typeof(T), "row");
var body = Expression.Equal(
Expression.PropertyOrField(param, column),
Expression.Constant(whereValue));
var lambda = Expression.Lambda<Func<T,bool>>(body, param);
then:
IQueryable<T> source = ...
var filtered = source.Where(lambda);
This will cause the Where to be executed at the server (for example, in TSQL), removing most of the network IO (asusming a sensible filter).