What is the exact use of AsEnumerable? Will it change non-enumerable collection to enumerable
collection?.Please give me a simple example.
From the "Remarks" section of the MSDN documentation:
The AsEnumerable<TSource> method has no effect
other than to change the compile-time
type of source from a type that
implements IEnumerable<T> to
IEnumerable<T> itself.
AsEnumerable<TSource> can be used to choose
between query implementations when a
sequence implements IEnumerable<T> but also has a different set
of public query methods available. For
example, given a generic class Table
that implements IEnumerable<T> and has its own methods such
as Where, Select, and SelectMany, a
call to Where would invoke the public
Where method of Table. A Table type
that represents a database table could
have a Where method that takes the
predicate argument as an expression
tree and converts the tree to SQL for
remote execution. If remote execution
is not desired, for example because
the predicate invokes a local method,
the AsEnumerable<TSource>
method can be used to hide the custom
methods and instead make the standard
query operators available.
If you take a look in reflector:
public static IEnumerable<TSource> AsEnumerable<TSource>(this IEnumerable<TSource> source)
{
return source;
}
It basically does nothing more than down casting something that implements IEnumerable.
Nobody has mentioned this for some reason, but observe that something.AsEnumerable() is equivalent to (IEnumerable<TSomething>) something. The difference is that the cast requires the type of the elements to be specified explicitly, which is, of course, inconvenient. For me, that's the main reason to use AsEnumerable() instead of the cast.
AsEnumerable() converts an array (or list, or collection) into an IEnumerable<T> of the collection.
See http://msdn.microsoft.com/en-us/library/bb335435.aspx for more information.
From the above article:
The AsEnumerable<TSource>(IEnumerable<TSource>) method has no
effect other than to change the compile-time type of source from a type
that implements IEnumerable<T> to IEnumerable<T> itself.
After reading the answers, i guess you are still missing a practical example.
I use this to enable me to use linq on a datatable
var mySelect = from table in myDataSet.Tables[0].AsEnumerable()
where table["myColumn"].ToString() == "Some text"
select table;
AsEnumerable can only be used on enumerable collections. It just changes the type of the collection to IEnumerable<T> to access more easily the IEnumerable extensions.
No it doesn't change a non-enumerable collection to an enumerable one. What is does it return the collection back to you as an IEnumerable so that you can use it as an enumerable. That way you can use the object in conjunction with IEnumerable extensions and be treated as such.
Here's example code which may illustrate LukeH's correct explanation.
IEnumerable<Order> orderQuery = dataContext.Orders
.Where(o => o.Customer.Name == "Bob")
.AsEnumerable()
.Where(o => MyFancyFilterMethod(o, MyFancyObject));
The first Where is Queryable.Where, which is translated into sql and run in the database (o.Customer is not loaded into memory).
The second Where is Enumerable.Where, which calls an in-memory method with an instance of something I don't want to send into the database.
Without the AsEnumerable method, I'd have to write it like this:
IEnumerable<Order> orderQuery =
((IEnumerable<Order>)
(dataContext.Orders.Where(o => o.Customer.Name == "Bob")))
.Where(o => MyFancyFilterMethod(o, MyFancyObject));
Or
IEnumerable<Order> orderQuery =
Enumerable.Where(
dataContext.Orders.Where(o => o.Customer.Name == "Bob"),
(o => MyFancyFilterMethod(o, MyFancyObject));
Neither of which flow well at all.
static void Main()
{
/*
"AsEnumerable" purpose is to cast an IQueryable<T> sequence to IEnumerable<T>,
forcing the remainder of the query to execute locally instead of on database as below example so it can hurt performance. (bind Enumerable operators instead of Queryable).
In below example we have cars table in SQL Server and are going to filter red cars and filter equipment with some regex:
*/
Regex wordCounter = new Regex(#"\w");
var query = dataContext.Cars.Where(car=> article.Color == "red" && wordCounter.Matches(car.Equipment).Count < 10);
/*
SQL Server doesn’t support regular expressions therefore the LINQ-to-db providers will throw an exception: query cannot be translated to SQL.
TO solve this firstly we can get all cars with red color using a LINQ to SQL query,
and secondly filtering locally for Equipment of less than 10 words:
*/
Regex wordCounter = new Regex(#"\w");
IEnumerable<Car> sqlQuery = dataContext.Cars
.Where(car => car.Color == "red");
IEnumerable<Car> localQuery = sqlQuery
.Where(car => wordCounter.Matches(car.Equipment).Count < 10);
/*
Because sqlQuery is of type IEnumerable<Car>, the second query binds to the local query operators,
therefore that part of the filtering is run on the client.
With AsEnumerable, we can do the same in a single query:
*/
Regex wordCounter = new Regex(#"\w");
var query = dataContext.Cars
.Where(car => car.Color == "red")
.AsEnumerable()
.Where(car => wordCounter.Matches(car.Equipment).Count < 10);
/*
An alternative to calling AsEnumerable is ToArray or ToList.
*/
}
The Enumerable.AsEnumerable method can be used to hide a type's custom implementation of a standard query operator
Consider the following example. we have a custom List called MyList
public class MyList<T> : List<T>
{
public string Where()
{
return $"This is the first element {this[0]}";
}
}
MyList has a method called Where which is Enumerable.Where() exact same name. when I use it, actually I am calling my version of Where, not Enumerable's version
MyList<int> list = new MyList<int>();
list.Add(4);
list.Add(2);
list.Add(7);
string result = list.Where();
// the result is "This is the first element 4"
Now how can I find the elements which are less than 5 with the Enumerable's version of Where?
The answer is: Use AsEnumerable() method and then call Where
IEnumerable<int> result = list.AsEnumerable().Where(e => e < 5);
This time the result contains the list of elements that are less than 5
Related
I am getting the error about LINQ to Entities does not recognize the method 'System.String Format but in the past I was able to do this when I have included .AsEnumerable() is there something different I need to do because of the GroupBy section?
select new PresentationLayer.Models.PanelMeeting
{
GroupId = pg.GroupId,
MeetingId = pmd.MeetingId,
GuidelineName = pmv.GuidelineName,
PanelDisclosuresAttendanceURL = string.Format("{0}?MeetingId={1}&GroupId=0",PanelDisclosureLink, pmd.MeetingId),
}).GroupBy(g => new
{
g.MeetingId,
g.GroupId
})
.AsEnumerable()
.SelectMany(grp => grp.AsEnumerable()).ToList(),
You have to be aware of the difference between an IEnumerable<...> and an IQueryable<...>.
IEnumerable
An object that implements IEnumerable<...> represents a sequence of similar items. You can ask for the first element of the sequence, and as long as you've got elements you can ask for the next element. IEnumerable objects are supposed to be executed within your own process. IEnumerable objects hold everything to enumerate the sequence.
At its lowest level, this is done using GetEnumerator() / MoveNext() / Current:
IEnumerable<Customer> customers = ...
IEnumerator<Customer> enumerator = customers.GetEnumerator();
while (enumerator.MoveNext())
{
// There is a next Customer
Customer customer = enumerator.Current;
ProcessCustomer(customer);
}
If you use foreach, then internally GetEnumerator / MoveNext / Current are called.
If you look closely to LINQ, you will see that there are two groups of LINQ methods. Those that return IEnumerable<TResult> and those that dont't return IEnumerable<...>
LINQ functions from the first group won't enumerate the query. They use deferred execution, or lazy execution. In the comments section of every LINQ method, you'll find this description.
The LINQ functions of the other group will execute the query. If you look at the reference source of extension class Enumerable, you'll see that they internally use foreach, or at lower level use GetEnumerator / MoveNext / Current
IQueryable
An object that implements IQueryable<...> seems like an IEnumerable. However, it represents the potential to fetch data for an Enumerable sequence. The data is usually provided by a different process.
For this, the IQueryable holds an Expression and a Provider. The Expression represents what must be fetched in some generic format. The Provider knows who will provide the data (usually a database management system) and how to communicate with this DBMS (usually SQL).
When you start enumerating the sequence, deep inside using GetEnumerator, the Expression is sent to the Provider, who will try to translate it into SQL. The data is fetched from the DBMS, and returned as an Enumerable object. The fetched data is accessed by repeatedly calling MoveNext / Current.
Because the database is not contacted until you start enumerating, you'll have to keep the connection to the database open until you've finished enumerating. You've probably made the following mistake once:
IQueryable<Customer> customers;
using (var dbContext = new OrderDbContext(...))
{
customers = dbContext.Customers.Where(customer => customer...);
}
var fetchedCustomers = customers.ToList();
Back to your question
In your query, you use string.Format(...). Your Provider doesn't know how to translate this method into SQL. Your Provider also doesn't know any of your local methods. In fact, there are even several standard LINQ methods that are not supported by LINQ to entities. See Supported and Unsupported LINQ methods.
How to solve the problem?
If you need to call unsupported methods, you can use AsEnumerable to fetch the data. All LINQ methods after AsEnumerable are executed by your own process. Hence you can call any of your own functions.
Database Management systems are extremely optimized in table handling. One of the slower parts of a database query is the transport of the selected data to your local process. Hence, let the DBMS do all selecting, try to transport as little data as possible.
So let your DBMS do your Where / (Group-)Join / Sum / FirstOrDefault / Any etc. String formatting can be done best by you.
In your String.Format you use PanelDisclosureLink and pmd.MeetingId. It will probably be faster if your process does the formatting. Alas you forgot to give us the beginning or your query.
I'm not sure where your PanelDisclosureLink comes from. Is it a local variable? If that is the case, then PanelDisclosuresAttendanceURL will be the same string for every item in your group. Is this intended?
var panelDisclosureLine = ...;
var result = dbContext... // probably some joining with Pgs, Pmds and Pmvs,
.Select(... => new
{
GroupId = pg.GroupId,
MeetingId = pmd.MeetingId,
GuidelineName = pmv.GuidelineName,
})
// make groups with same combinations of [MeetingId, GroupId]
.GroupBy(joinResult => new
{
MeetingId = joinResult.MeetingId,
GroupId = joinResult.GroupId,
},
// parameter resultSelector: use the Key, and all JoinResult items that have this key
// to make one new:
(key, joinResultItemsWithThisKey) => new
{
MeetingId = key.MeetingId,
GroupId = key.GroupId,
GuideLineNames = joinResultsItemsWithThisKey
.Select(joinResultItem => joinResultItem.GuideLineName)
.ToList(),
})
So by now the DBMS has transformed your join result into objects with
[MeetingId, GroupId] combinations and a list of all GuideLineNames that have belong to
this [MeetingId, GroupId] combination.
Now you can move it to your local process and use String.Format.
.AsEnumerable()
.SelectMany (fetchedItem => fetchedItem.GuideLineNames,
(fetchedItem, guideLineName) => PresentationLayer.Models.PanelMeeting
{
GroupId = fetchedItem.GroupId,
MeetingId = fetchedItem.MeetingId,
GuidelineName = guidelineName,
PanelDisclosuresAttendanceURL = string.Format("...",
PanelDisclosureLink,
fetchedItem.MeetingId);
Note: in my parameter choice plurals are collections; singulars are elements of these collections.
PanelDisclosuresAttendanceURL = string.Format("{0}?MeetingId={1}&GroupId=0",PanelDisclosureLink, pmd.MeetingId),
}).
.GroupBy
If you want to use string.Format you first have to get the data from the server.
You can just move the .GroupBy( ... ) and then the .AsEnumerable() call to the top, before select new PresentationLayer.Models.PanelMeeting { ... }. If you are not selecting too much data that way...
I'm using System.Linq.Dynamic to allow me to dynamically select a list of fields from a query like this:
finalQuery = query.Select(string.Format("new({0})", string.Join(",", selectors)));
Where selectors is just a List<string> with all the fields I want. This works great, but this version of the extension method Select returns an IQueryable. Not this is not IQueryable<T>. If I have an IQueryable<T> I can simply do a .ToList() to convert it into a list and force the query to actually run on the database, but with the non-generic IQueryable, that method doesn't exists.
This is because ToList is inherited from IEnumerable<T> which IQueryable<T> inherits and IQueryable, obviously, doesn't.
So what's the most efficient way to get an IQueryable to execute the query and give me back a list? I can do this:
List<object> rtn = new List<object>();
foreach (var o in finalQuery)
{
rtn.Add(o);
}
But it seems like their ought to be an easier way.
Edit: In response to suggestions, I tried both:
finalQuery.Cast<object>().ToList();
and:
finalQuery.Cast<dynamic>().ToList();
Which both give NotSupportedExceptions with the message:
Unable to cast the type 'DynamicClass1' to type 'System.Object'. LINQ to Entities
only supports casting EDM primitive or enumeration types.
This appears to be a limitation in the way LINQ to Entities translates IQueryable.Cast with anonymous types. You can work around this by using it as an IEnumerable (your working example does this). This causes the code to do the cast in the .NET runtime after it's retrieved from the DB, instead of trying to handle it in the DB engine. E.g.
IEnumerable finalQuery = query.Select(string.Format("new({0})",
string.Join(",", selectors)));
var result = finalQuery.Cast<dynamic>().ToList();
Or
public static IList<T> CastToList<T>(this IEnumerable source)
{
return new List<T>(source.Cast<T>());
}
var finalQuery = query.Select(string.Format("new({0})",
string.Join(",", selectors)));
var result = finalQuery.CastToList<dynamic>();
I am using EF 4 with repository pattern which has the generic query method shown below:
public IEnumerable<T> Query(Expression<Func<T, bool>> filter)
{
return objectSet.Where(filter);
}
I know I can select a complete object like this:
context.PeriodRepository.Query(a => a.EntityId == selectedEntityId);
But I want to pass a Linq query that returns it as type instead of IEnumerable<type> using a LINQ expression without changing the method. Please advise me how to do that.
Use the First() or FirstOrDefault() methods and pass in a predicate to find the element you want if it is not the first.
A Query describes the operations performed on the list; you must perform the query to get the result(s).
Using .ToList()/.ToArray() will return all items that match the query. Use .First() to get the first item that matches or .FirstOrDefault() to get the first item or the default value for no matches.
The default value for a class is null.
I think your code should be like this:
var myMatch = context.PeriodRepository
.Query(a => a.EntityId == selectedEntityId)
.First();
Also note that, First() will throw an exception when there is no match.
I have a class Literal and a Tag is inheriting from it.
I would like to do the following but I am getting
Unable to cast object of type 'WhereListIterator`1[Core.Literal]' to type 'System.Collections.Generic.List`1[Core.Tag]'.
private List<Literal> literals;
public List<Tag> Tags
{
get { return (List<Tag>)literals.Where(x => x is Tag); }
}
thanks
You would be better off doing:
literals.OfType<Tag>().ToList();
This gives you a List<Tag>.
You can also do:
var asList = new List<Tag>(literals.OfType<Tag>());
Casting simply does not work because LINQ works in terms of either IEnumerable<T> or IQueryable<T> which neither use List as a backing implementation for the results. The second method I posted uses a constructor overload of List<T> the takes in an IEnumerable<T> as its initial collection of objects. Also in this scenario the OfType<T> method from LINQ is a much cleaner, shorter form of essentially filtering a list with Where(x -> x is T).
Also, OfType<T> in this scenario is a much better idea, because the result is an IEnumerable<T> of your target type. Where(x => x is T) will return an IEnumerable<T> of the original source's type. So that's why (List<Tag>)literals.Where(x => x is Tag).ToList() emit an error for invalid casts.
More information on ToList
More information on OfType
literals.Select(x => x as Tag).Where(x => x != null).ToList()
Note that this will return new list. You won't be able to do where and modify original list by this. Also this can be done like this: literals.OfType<Tag>().ToList() and it will return IList<Tag>
Update: modified type
List<Tag>)literals.Where(x => x is Tag).ToList();
or even better :
literals.OfType<Tag>();
then you can create a new list from it:
new List<Tag>(literals.OfType<Tag>());
Depending on the effect you want you could do this:
public List<Tag> Tags
{
get { return literals.Where(x => x is Tag).ToList(); }
}
Do realize that this is not a cast but the creation of a list!
How about adding ToList()?
Your getter becomes: literals.Where(x => x is Tag).ToList();
try the following:
literals.Where(x => x is Tag).Cast<Tag>().ToList();
it works if Tag is derived from Literal
The Where method returns the same type as the source, so you would have to cast each item, then use the ToList method to create a list from the result:
return literals.Where(x => x is Tag).Select(x => (Tag)x).ToList();
You can also us the OfType method:
return literals.OfType<Tag>().ToList();
I really like PredicateBuilder. It allows me to build all sorts of queries very dynamically. The predicate variable can be passed around to different objects and they can add onto it with values they know about, etc. Except when I am needing to use a .Contains on a hashed collection. Bzzt! Crash and burn.
For instance (example/pseudo code, this may or may not compile/run):
protected Expression<Func<MyClass, bool>> GetWherePredicate()
{
string[] selectedValues = Request.Form.GetValues("checkbox1") ?? new string[0];
HashSet<int> selectedIDs = new HashSet<int>(selectedValues.Cast<int>());
Expression<Func<MyClass, bool>> predicate = PredicateBuilder.True<MyClass>();
predicate = predicate.And(s => selectedIDs.Contains(s.ID));
return predicate;
}
protected void Retrieve()
{
Expression<Func<MyClass, bool>> predicate = GetWherePredicate();
IEnumerable<MyClass> retrievedValues = MyDataContext.GetTable<MyClass>.Where(predicate);
}
When I try to do that, I get a NotSupportedException: Method 'Boolean Contains(Int32)' has no supported translation to SQL due to the selectedIDs HashSet not being in scope. If I do this all in the same method, then it works fine.
I need to know the right way to get my predicate there to resolve or compile or whatever so that it can be used in a different scope from where the HashSet is declared. Any help?
UPDATE: I had this pretty wrong. The code below works fine, so there is no scope conflict. Thanks Jay.
string[] selectedValues = Request.Form.GetValues("checkbox1") ?? new string[0];
Expression<Func<MyClass, bool>> predicate = PredicateBuilder.True<MyClass>();
predicate = predicate.And(s => selectedValues.Contains(s.ID.ToString()));
From the exception you cite, it seems unlikely that scope is a factor here.
My answer to this is that you need to declare selectedIDs as IEnumerable<int> instead of HashSet<int> (or just cast it before calling Contains(), but that doesn't account for it working when all in the same method, so I'm unsure.
Without seeing any actual code that is exhibiting this behaviour, it will be difficult to troubleshoot any further.
Do not get razzle-dazzled by the name Contains... there are many methods that are named that, and not many have supported translations into SQL.
Enumerable.Contains<T> and List<T>.Contains are methods with supported translations.
HashSet<T>.Contains is a method that has no supported translation.
There are many resolutions, but I recommend:
IEnumerable<string> selectedValueQuery =
Request.Form.GetValues("checkbox1") ?? Enumerable.Empty<string>();
List<string> selectedIds = selectedValueQuery
.Cast<int>()
.Distinct()
.ToList();
Although a simpler solution might be:
IEnumerable<int> selectedIDs = new HashSet<int>(selectedValues.Cast<int>());
Edit: It's not about concrete implementation of Contains at all. It's about whether the linqtosql query provider can identify the method and translate it into an IN (list) sql expression. The recognition code looks at the type of the parameter used in the expression. The recognition code does not use polymorphism/implementation nor does it walk the inheritence tree looking for other possibilities.
List<int> myList = new List<int>(){1, 2, 3};
IList<int> myIList = myList;
IEnumerable<int> myIEnumerable = myList;
//works by List<T>.Contains()
db.Customers.Where(c => myList.Contains(c.CustomerID));
//doesn't work, no translation for IList<T>.Contains
db.Customers.Where(c => myIList.Contains(c.CustomerID));
//works by Enumerable.Contains<T>()
db.Customers.Where(c => myIEnumerable.Contains(c.CustomerID));
//works by Enumerable.Contains<T>()
db.Customers.Where(c => Enumerable.Contains(myIEnumerable, c.CustomerID));
Even though these parameters reference the same instance, different translation behaviors occur because the type of the parameter is different.
.Contains() is not called as it is translated, therefore its implementation is irrelevant. It could throw NotImplementedException or return true;