Yesterday I was working on a code refactor and came across an exception that I really couldn't find much information on. Here is the situation.
We have an a pair of EF entities that have a many to many relationship through a relation table. The objects in question look like this, leaving out the unnecessary bits.
public partial class MasterCode
{
public int MasterCodeId { get; set; }
...
public virtual ICollection<MasterCodeToSubCode> MasterCodeToSubCodes { get; set; }
}
public partial class MasterCodeToSubCodes
{
public int MasterCodeToSubCodeId { get; set; }
public int MasterCodeId { get; set; }
public int SubCodeId { get; set; }
...
}
Now, I attempted to run a LINQ query against these entities. We use a lot of LINQ projections into DTOs. The DTO and the query follow. masterCodeId is a parameter passed in.
public class MasterCodeDto
{
public int MasterCodeId { get; set; }
...
public ICollection<int> SubCodeIds { get; set; }
}
(from m in MasterCodes
where m.MasterCodeId == masterCodeId
select new MasterCodeDto
{
...
SubCodeIds = (from s in m.MasterCodeToSubCodes
select s.SubCodeId).ToList(),
...
}).SingleOrDefaultAsync();
The internal query throws the following exception
Expression of type 'System.Data.Entity.Infrastructure.ObjectReferenceEqualityComparer' cannot be used for constructor parameter of type 'System.Collections.Generic.IEqualityComparer`1[System.Int32]'
We have done inner queries like this before in other places in our code and not had any issues. The difference in this one is that we aren't new-ing up an object and projecting into it but rather returning a group of ints that we want to put in a list.
I have found a workaround by changing the ICollection on MasterCodeDto to IEnumerable and dropping the ToList() but I was never able to find out why I couldn't just select the ids and return them as a list.
Does anyone have any insight into this issue? Normally returning just an id field and calling ToList() works fine when it is not part of an inner query. Am I missing a restriction on inner queries that prevents an operation like this from happening?
Thanks.
Edit: To give an example of where this pattern is working I'll show you an example of a query that does work.
(from p in Persons
where p.PersonId == personId
select new PersonDto
{
...
ContactInformation = (from pc in p.PersonContacts
select new ContactInformationDto
{
ContactInformationId = pc.PatientContactId,
...
}).ToList(),
...
}).SingleOrDefaultAsync();
In this example, we are selecting into a new Dto rather than just selecting a single value. It works fine. The issues seems to stem from just selecting a single value.
Edit 2: In another fun twist, if instead of selecting into a MasterCodeDto I select into an anonymous type the exception is also not thrown with ToList() in place.
I think you stumbled upon a bug in Entity Framework. EF has some logic for picking an appropriate concrete type to materialize collections. HashSet<T> is one of its favorites. Apparently (I can't fully follow EF's source code here) it picks HashSet for ICollections and List for IEnumerable.
It looks like EF tries to create a HashSet by using the constructor that accepts an IEqualityComparer<T>. (This happens in EF'sDelegateFactory class, method GetNewExpressionForCollectionType.) The error is that it uses its own ObjectReferenceEqualityComparer for this. But that's an IEqualityComparer<object>, which can not be converted to an IEqualityComparer<int>.
In general I think it is best practice not to use ToList in LINQ queries and to use IEnumerable in collections in DTO types. Thus, EF will have total freedom to pick an appropriate concrete type.
Related
I've written the below function to query the SQLite DB in my Xamarin forms app. But, since I have to call .ToList() twice, I'm not very confident about it. Is this bad code? Any feedback will be highly appreciated.
public static List<string> GetAllLocationIds()
{
try
{
lock (CollisionLock)
{
//TableQuery<TResult> First .ToList() result
//List<string> Second .ToList() result
return Database.Table<Location>().ToList().Select(loc=>loc.LocationId).ToList();
}
}
catch (Exception ex)
{
Insights.Report(ex);
return null;
}
}
Performing .Select directly on Database.Table<Location>() results in the following exception.`
System.MissingMethodException: Default constructor not found for type
System.String at System.RuntimeType.CreateInstanceMono`
Yes it is.
On
Database.Table<Location>().ToList()
You are materializing all of Table Location. Then you are only selecting the LocationId in Memory.
Instead use:
Database.Table<Location>().Select(loc=>loc.LocationId).ToList();
Which is working directly on IQueryable<Location> and only materializes the LocationId. Assuming Table<Location> is IQueryable<Location>.
You just can not do Linq projections to string types like that with sqlite-net(-pcl) as it needs a default parameterless constructor.
What follows is the "best way" simulate a "Linq projection" that I have found when Mobile memory and performance is considered.
Use a custom Class with only the columns that need projected
Use a SQL query with only the columns needed to map to that custom class (where filter in the select statement if needed)
Convert to custom type
Actual Table Class:
class Location
{
[PrimaryKey]
public int Column1 { get; set; }
public int Column2 { get; set; }
~~~
public string LocationId { get; set; }
}
Now make a new class that describes your "projection" needs, in this case I only want the LocationId column.
Projection Class
class SimpleList
{
public string LocationId { get; set; }
}
SQL Select (selecting only the columns that map to the projection class)
SQLiteConnection.Query<SimpleList>("select LocationId from [Location]")
Now you have a List<SimpleList>, you can convert it to a List<string> if you really need to:
SQLiteConnection.Query<SimpleList>("select LocationId from [Location]").ConvertAll(x => x.LocationId);
Is it worth it? If you have a large number of rows and/or columns in your table and cannot use a deferred query and/or avoid a Linq projection... IMHO yes... Use the profiler to confirm ;-)
If you have a couple dozen rows? Maybe not, but even then the number of temp. objects that get instanced is reduced, and for me that is a win on mobile.
I am using System.Linq.Dynamic, version on github's repository.
I am NOT interested in NON System.Linq.Dynamic solution.
I am trying to perform select on nested collection's property. Let us imagine we have following situation:
public class Region
{
public int Id { get; set; }
public List<Town> Towns { get; set; }
}
public class Town
{
public int Id { get; set; }
public string Name { get; set; }
}
Would it be possible to 'Select' region's id and it's town's names?
Something of a kind:
someListofRegions.Select("new(Id, Towns.Name)")
where "new(Id, Towns.Name)" is the dynamic Linq expression.
of course example above fails.
You can't perform a select on a nested collection's property as this would require flattening of the resulting collection.
Usually you would use SelectMany() to do this flattening, but since you want to use System.Linq.Dynamic and I don't believe this library has a dynamic SelectMany() this probably isn't possible. You could write your own SelectMany() though using expression trees which shouldn't be too difficult.
Alternatively you may find GroupBy is more suited to your needs here anyway, I can't personally see the benefit in wanting a collection of Region IDs and Town Names - there would be loads of duplicate region IDs.
How to I check if a nested model object, has any items.
Ie. if I have an object/viewmodel:
public class CarViewModel
{
public string Type { get; set; }
public long ID { get; set; }
public virtual IQueryable<Feature> Features { get; set; }
}
public class Feature
{
public string Offer { get; set; }
public decimal Rate { get; set; }
public virtual CarViewModel CarViewModel { get; set; }
}
...and it is populated as follows - so that 1 car object has 2 additional features, and the other car object, has no additional features:
[
{"Type":"SoftTop","ID":1,
"Features":
[{"Offer":"Alloys","Rate":"500"},{"Offer":"Standard","Rate":"100"}]},
{"Type":"Estate","ID":2,
"Features":[]}
]
So in my code, I had "Cars" populated with the data above:
foreach (var car in Cars)
{
if (!car.Features.Any())
{
car.Type = "Remove";
}
}
However, I get the message: This method is not supported against a materialized query result. at the if (!car.Features.Any()) line.
I got the same error when trying if (car.Features.Count()==0)
Is there a way of checking if the number of Features is 0?
Or is there a linq way of removing any items from the object, where the number of features is 0?
Thank you,
Mark
UPDATE
I changed the viewModel to use IEnumerable and then the following:
cars=cars.Where(x => x.Feature.Count()>0).ToList();
That seems to work - although I'm not 100% sure. If anyone can say whether this is a "bad" fix or not, I'd appreciate it.
Thanks, Mark
Try fetching the results first then checking the count
car.Features.ToList().Count
I don't think there any anything wrong with the fix - when you're using IQueryable<T> that came from a Linq to DB (L2S, Entity Framework, etc) you pretty much have to materialise it before you can use things like Any() or Count() when you ask for these things inside foreach.
As to why this is - I actually am not 100% certain and I believe that the error is a bit misleading in this respect, but I think that what it's complaining about is that neither Cars not car.Features() has actually been fully evaluated and run yet (i.e you are only starting to hit the database at the point when you go foreach ... in your code because it's IQueryable<T>).
However on a broader note I'd recommend you not use IQueryable<T> in your Viewmodels, much safer to use IEnumerable<T> - no chance of accidentally setting off a database access when rendering your view, for example.
And also when you are returning data from your DataLayer or wherever, a good rule of thumb is to materialise it as quickly as possible so as to be able to move on with an actual list of actual things as opposed to a "promise to go and look" for certain things in the database at some unspecificed point in the future :) So your DataLayers should only ever return IEnumerable<T>'s
You can always cast an IEnumerable to IQueryable if for some reason you need to...
I am trying to replace a nasty LINQ 2 SQL hit with some dapper queries to improve performanace. In doing so I have to weave a bunch of different objects together in order to create the big object required to hold all the information I need for ASN information.
The current problem I am having is with an abstract class Orders, this class is implemented by two seperate classes AutionOrder and MerchantOrder using a discriminator property.
Since I cannot use dapper to create a object that is an abstract class I am instead using one of the public classes. however when it goes to build the object it is failing inside of GetSettableProps it is finding the proper DeclaringType but the GetProperty method is returning null when it is looking for an property that is internal or is an EntitySet. I've tried to hack around it using t.BaseType.GetProperty as well as p.GetAccessors().First().GetBaseDefinition().DeclaringType.GetProperty(p.Name).GetSetMethod(true) with no success.
dummy objects:
Order
OrderID, Name, Address, RowVersion(internal), Shipments(EntitySet),OrderDetails(EntitySet), Customer(EntityRef)
Shipment
ShipmentID, OrderID, TrackingNumber
OrderDetails
OrderDetailID, OrderID, Product, QTY, Price
Customer
CustomerID, Name,
For this particular SQL hit I am trying to grab some of the 1 to 1 relationship mappings I need.
SELECT o.* from Orders as o left join Customers as c on o.CustomerID = c.CustomerID where o.OrderID in (1,2,3);
This is what I am using to utilize dapper and let it do it's magic:
using (var connection = new SqlConnection(_ConnectionString))
{
connection.Open();
results = connection.Query<MerchantOrder, MerchantCustomer, MerchantOrder>(sql.ToString(),
(o, c) => { o.Customer = c; return o; },
splitOn: "CustomerID");
}
If I change Order to be a public class this problem goes away though, but this is not a desired side-effect. It is failing when trying to set the propInfo for RowVersion - switching this to public instead of internal solved this problem - although not desired. But then it fails when it is trying to create the Shipments objects for Order. Again none of this is an issue when Order is a public class.
Also I am doing separate queries to pull in Many to one relationships such as Shipments to Orders and OrderDetails to Orders and normalizing the results into a proper Order Object.
MerchantOrder is pretty much an empty class with no real special logic. The discriminating different here is just how we end up finding the CustomerID which is abstracted away prior to the actual SQL hit anyway.
Also I am using the latest version of dapper as of 12/20/2011.
I really like dapper, but this problem is making my head asplode - so thanks for the help!
This was a bug, that is now fixed in trunk:
public class AbstractInheritance
{
public abstract class Order
{
internal int Internal { get; set; }
protected int Protected { get; set; }
public int Public { get; set; }
public int ProtectedVal { get { return Protected; } }
}
public class ConcreteOrder : Order
{
public int Concrete { get; set; }
}
}
// http://stackoverflow.com/q/8593871
public void TestAbstractInheritance()
{
var order = connection.Query<AbstractInheritance.ConcreteOrder>("select 1 Internal,2 Protected,3 [Public],4 Concrete").First();
order.Internal.IsEqualTo(1);
order.ProtectedVal.IsEqualTo(2);
order.Public.IsEqualTo(3);
order.Concrete.IsEqualTo(4);
}
One side note is that, by design, we do not set private fields or properties in the base classes. The behaviour can be magical and not consistent.
Eg:
class A { private int a {get; set;} }
class B : A { private int a {get; set;} }
class C: B {}
// What should "select 1 a" do? Set it on A? Set it on B? Set it on Both? Set it on neither?
We went with "set it on neither"
I think is not possible (because of the abstract class) without modifying your code.
I had a similar problem and ended up creating a new object private to the assembly where I have my repositories that derived from the abstract base class.
This class is not abstract and only visible to the repository class that stores the data, this class had all the required methods for the actual table.
Let's say I need to display a list of customers, but only want to display the Name and somehow associate the key to the name within a list control.
It would probably be costly to retrieve the entire list of customers and all it's properties. In this scenario, would it be better to create another class with the properties that are required (in this case Id and Name)?
A basic implementation could look like this:
public class Customer {
public int Id { get; set; }
public string Firstname { get; set; }
public string Lastname { get; set; }
public int Age { get; set; }
.....
}
public class CustomerListView {
public int Id { get; set; }
public string Name { get; set; }
}
public interface IRepository<T> {
public T Find(int id);
public IEnumerable<T> FindAll();
....
}
public class Repository<T>: IRepository<T> {
....
}
public class CustomerRepository: Repository<Customer> {
public IEnumerable<CustomerListView> FindAllListView();
}
Would this approach be appropriate? What other options would there be?
In order to achieve such goals, I create a simple 'View' class, for example CustomerView, which just contains the properties that are needed to display an overview.
My Repository then has a method which returns a collection of these CustomerView objects.
I mostly use NHibernate in my projects. Nhibernate allows you to use 'projections'.
So, what I do in my repository is this:
(note that the code below is just some pseudo-code; it won't compile).
public IList<CustomerView> GetAllCustomers()
{
ICriteria crit = _session.CreateCriteria (typeof(Customer));
crit.AddProjection ( ... );
crit.SetResultTransformer (new EntityToBeanTransformer(typeof(CustomerView));
return crit.ToList();
}
In fact, it comes down to this: I tell my O/R mapper that it should query Customers, but that it should return entities of type 'CustomerView'.
In the defintion of the projection, I also define which properties of the Customer class map to which properties of the CustomerView class.
Then, the O/R mapper is smart enough to generate a very simple query, which only retrieves those fields that are required to populate the CustomerView class.
For instance, the query that is executed can be as simple as:
SELECT customerid, customername FROM tblCustomer
If you use IQueryable as your return instead of IEnumerable than there is no cost of doing:
CustomerRepository().GetAll().Find(1) because AsQueryable doesn't actually execute until you request data. That means LINQ can optimize it out to a:
SELECT .... FROM .... WHERE ID = 1 instead of
GET EVERYTHING. FIND WHERE THE ID = 1
See this post for an explanation:
Why use AsQueryable() instead of List()?
Using this approach you could create an anonymous class and futher narrow down the data going over the wire to just what you want. That way the query generated by LINQ is optimized to the fullest.
If you have to retrieve the list form a Database then your proposal makes some sense but I would look into a Linq and anonymous type solution.
If the list of Customers already exists in memory then there there are no savings.
You could combine the techniques used by Nissan and Frederik (anonymous types and NHibernate) by using Linq-to-NHibernate.
Item #31 in Bill Wagner's More Effective C# says "limit type scope by using anonymous types", and I agree. BTW, I recommend the whole book.