Can you return IQueryable which is composed of two or more different subclasses ? Here's my attempt to show what I mean. It throws the error:
System.NotSupportedException: Types in
Union or Concat have members assigned
in different order..
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new OrderItemA {
// etc
} as OrderItem;
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new OrderItemB {
//etc
} as OrderItem;
return a.Concat<OrderItem>(b);
Try doing the concat on IEnumerable instead of IQueryable:
return a.AsEnumerable().Concat(b.AsEnumerable());
If you need an IQueryable result you could do this:
return a.AsEnumerable().Concat(b.AsEnumerable()).AsQueryable();
Doing this will force the concat to happen in-memory instead of in SQL, and any additional operations will also happen in-memory (LINQ To Objects).
However, unlike the .ToList() example, the execution should still be deferred (making your data lazy loaded).
My guess is that this is because you are using LINQ in an LINQ-to-SQL context.
So using Concat means that LINQ2SQL will need to join both query into a SQL UNION query which might be where the System.NotSupportedException originated from.
Can you try this:
return a.ToList().Concat<OrderItem>(b.ToList());
And see if it make any difference?
What the above does is that it executes the query twice and then concatenate them in-memory instead of hot-off-SQL as to avoid the query translation problem.
It might not be the ideal solution, but if this work, my assumption is probably correct, that it's a query translation problem:
More information about Union and Concat translation to SQL:
http://blog.benhall.me.uk/2007/08/linq-to-sql-difference-between-concat.html
http://msdn.microsoft.com/en-us/library/bb399342.aspx
http://msdn.microsoft.com/en-us/library/bb386979.aspx
Hope this helps.
Interestingly after reading your post and a bit of testing, I realized that what your actually doing does seem to work just fine for me given that the projection part you show as ellipsis in both of your queries match. You see, LINQ to SQL appears to construct the underlying projection for the SQL select command based off of the property assignment statements as opposed to the actual type being materialized so as long as both sides have the same number, type, and order (not sure about this) of member assignments the UNION query should be valid.
My solution that I've been working with is to create a property on my DataContext class which acts much like a SQL View in that it allows me to write a query (in my case a Union between two different tables) and then use that query as if it is itself like a table when composing read-only select statements.
public partial class MyDataContext
{
public IQueryable<MyView> MyView
{
get
{
var query1 =
from a in TableA
let default_ColumnFromB = (string)null
select new MyView()
{
ColumnFromA = a.ColumnFromA,
ColumnFromB = default_ColumnFromB,
ColumnSharedByAAndB = a.ColumnSharedByAAndB,
};
var query2 =
from a in TableB
let default_ColumnFromA = (decimal?)null
select new MyView()
{
ColumnFromA = default_ColumnFromA,
ColumnFromB = b.ColumnFromB,
ColumnSharedByAAndB = b.ColumnSharedByAAndB,
};
return query1.Union(query2);
}
}
}
public class MyView
{
public decimal? ColumnFromA { get; set; }
public string ColumnFromB { get; set; }
public int ColumnSharedByAAndB { get; set; }
}
Notice two key things:
First of all the projection formed by the queries which make up both halves of the Union have the same number, type, and order of columns. Now LINQ may require the order to be the same (not sure about this) but it is definitely true that SQL does for a UNION and we can be sure that LINQ will require at least the same type and number of columns and these "columns" are known by the member assignments and not from the properties of the type you are instantiating in your projection.
Secondly LINQ currently doesn't allow for multiple constants to be used within a projections for queries which formulate a Concat or Union and from my understanding this is mainly because these two separate queries are separately optimized before the Union operation is processed. Normally LINQ to SQL is smart enough to realize that if you have a constant value which is only being used in the projection, then why send it to SQL just to have it come right back the way it was instead of tacking it on as a post process after the raw data comes back from SQL Server. Unfortunately the problem here is that this is a case of LINQ to SQL being to smart for it's own good, as it optimizes each individual query too early in the process. The way I've found to work around this is to use the let keyword to form a range variable for each value in the projection which will be materialized by getting it's value from a constant. Somehow this tricks LINQ to SQL into carrying these constants through to the actual SQL command which keeps all expected columns in the resulting UNION. More on this technique can be found here.
Using this techinque I at least have something reusable so that no matter how complex or ugly the actual Union can get, especially with the range variables, that in your end queries you can write queries to these pseudo views such as MyView and deal with the complexity underneath.
Can you do the projection after the concat?
// construct the query
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new {
type = "A"
item = oi
}
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new {
type = "B"
item = oi
}
var temp = a.Concat<OrderItem>(b);
// create concrete types after concatenation
// to avoid inheritance issue
var result = from oi in temp
select (oi.type == "A"
? (new OrderItemA {
// OrderItemA projection
} as OrderItem)
: (new OrderItemB {
// OrderItemB projection
} as OrderItem)
);
return result
Not sure if the ternary operator works in LINQ2SQL in the above scenario but that might help avoid the inheritance issue.
Related
Imagine the following classes:
class Person
{
public string Name { get; set; }
public int Age { get; set; }
}
class Underage
{
public int Age { get; set; }
}
And I do something like this:
var underAge = db.Underage.Select(u => u.Age) .ToList()/AsEnumerable()/AsQueryable()
var result = db.Persons.Where(p => underAge.Contains(p.Age)).ToList();
What is the best option? If I call ToList(), the items will be retrieved once, but If I choose AsEnumerable or AsQueryable will they be executed everytime a person gets selected from database in the Where() (if that's how it works, I don't know much about what a database does in the background)?
Also is there a bigger difference when Underage would contain thousands of records vs a small amount?
None.
You definitely don't want ToList() here, as that would load all of the matching values into memory, and then use it to write a query for result that had a massive IN (…) clause in it. That's a waste when what you really want is to just get the matching values out of the database in the first place, with a query that is something like
SELECT *
FROM Persons
WHERE EXISTS(
SELECT NULL FROM
FROM Underage
WHERE Underage.age = Persons.age
)
AsEnumerable() often prevents things being done on the database, though providers will likely examine the source and undo the effects of that AsEnumerable(). AsQueryable() is fine, but doesn't actually do anything in this case.
You don't need any of them:
var underAge = db.Underage.Select(u => u.Age);
var result = db.Persons.Where(p => underAge.Contains(p.Age)).ToList();
(For that matter, check you really do need that ToList() in the last line; if you're going to do further queries on it, it'll likely hurt them, if you're just going to enumerate through the results it's a waste of time and memory, if you're going to store them or do lots of in-memory operations that can't be done in Linq it'll be for the better).
If you just leave your first query as
var underage = db.Underage.Select(u => u.Age);
It will be of type IQueryable and it will not ping your database ( acutaly called deferred execution). Once you actually want to execute a call to the database you can use a greedy operator such as .ToList(), .ToArray(), .ToDictionary(). This will give your result variable an IEnumerable collection.
See linq deferred execution
Your should use join to fetch the data.
var query = (from p in db.Persons
join u in db.Underage
on u.Age equals p.Age
select p).ToList();
If you not use .ToList() in above query it return IEnumerable of type Person and actual data will be fetch when you use the object.
All are equally good ToList(), AsEnurmeable and Querable based on the scenario you want to use. For your case ToList looks good for me as you just want to fetch list of person have underage
I have a large select expression to reuse in several classes. For the DRY principle I have chosen to create a property that returns the Expression to the caller code
protected virtual Expression<Func<SezioneJoin, QueryRow>> Select
{
get
{
return sj => new QueryRow
{
A01 = sj.A.A01,
A01a = sj.A.A01a,
A01b = sj.A.A01b,
A02 = sj.A.A02,
A03 = sj.A.A03,
A11 = sj.A.A11,
A12 = sj.A.A12,
A12a = sj.A.A12a,
A12b = sj.A.A12b,
A12c = sj.A.A12c,
A21 = sj.A.A21,
A22 = sj.A.A22,
..............
Lots of assignements
};
}
}
Now I can successfully use the property if I do
var query = dataContext.entity.Join(...).Where(x => ...).Select(Select);
But the following will not compile:
from SezioneJoin sj in (
from A a in ...
join D d in ... on new { ... } equals new { ... }
where
d.D13 == "086" &&
!String.IsNullOrEmpty(a.A32) && a.A32 != "086"
orderby a.A21
orderby a.prog
select new SezioneJoin{...})
select Select
Error is
Unable to cast 'System.Linq.IQueryable<System.Linq.Expressions.Expression<System.Func<DiagnosticoSite.Data.Query.SezioneJoin,DiagnosticoSite.Data.Query.QueryRow>>>' into 'System.Linq.IQueryable<DiagnosticoSite.Data.Query.QueryRow>'
I can understand that the LINQ syntax requires the body of the select statement to be the inner type of the IQueryable that it returns, so the compiler is fooled into returning a list of expressions. With the Lambda syntax, the expression is a parameter that is either compiled in-line or returned by some other method (even dynamically!).
I would like to ask if there is any way to circumvent this and avoid defining large select expressions inline
protected virtual Expression> Select
I'd avoid using the names of any of the Linq-mapped methods (Select, Where, GroupBy, OrderBy, OrderByDescending) as member names. It works in this case, but when it causes problems by matching the definitions for those it can be confusing if you aren't in the habit of just not using those names unless you deliberately want to override Linq.
On a related note. Consider that:
from var item in source select item.Something
is equivalent to:
source.Select(item => item.Something);
Therefore:
from SezioneJoin sj in (/*…*/) select Select;
is equivalent to:
(/*…*/).Select(sj => Select);
That is, you arent' creating a query that executes the expression in Select, but one that returns the expression itself.
You should either just use the form .Select(Select) or use select sj => (Select)(sj) but that second one will (if I even have the parentheses correct to stop it clashing with Queryable.Select, I haven't tested that) call the Select property every time so is at best wasteful and at worse not going to be something a query provider can make use of, so it will fail with most linq-providers. In all, use the .Select(Select) form (and change the name).
(On a separate note, if you're going to buffer an expression, actually buffer it; create a private Expression<Func<SezioneJoin, QueryRow>> once and return it in the property's getter, rather than creating it every time).
Simply use extension method in place of last LINQ select statement:
var query = from SezioneJoin sj ... select new SezioneJoin{...});
var projection = query.Select(Select);
My problem is that I cannot get Where statement working correctly with Entity Framework.
I have a class that has int and long properties that define one row in the database. Let's call those properties Id & Parent
public class Update
{
public long Id { get; set; }
public int Parent { get; set; }
}
Then I have list of those Updates and I want that each database row should match one those updates (both properties) in the list.
What I have tried is that I convert those updates to an anonymous type and try to find only those rows that have a perfect match: both int & long are same.
// anon type
var mapping = updates.Select(o => new { id = o.Id, parent = o.Parent}).ToList();
var results = from fa in Uvw_MyTable
// PROBLEMATIC WHERE
where mapping.Contains(new { id = fa.Id, parent = fa.Parent })
select new { fa };
But unfortunately I always get an error: Unable to create a constant value of type 'Anonymous type'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
I have also tried to use Any like this:
where mapping.Any(a => fa.Id == a.id && fa.Parent == a.parent)
But I get the same error. The same thing happens also when I use the List of Update objects. Unable to create a constant value of type 'xxx.Update'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
Any thoughts how to solve this issue?
Update
This exactly same query works very well in the LINQPad 4
Update 2
Is the problem related to this question?
Update 3
As Maarten pointed out this problem needs solution that uses different approach to re-produce the same results.
You are actually executing two queries in your code.
The first query is:
var mapping = updates.Select(o => new { id = o.Id, parent = o.Parent}).ToList();
The second query is (ok, the query is not iterated over, so technically the query is not yet executed here - nitpicking):
var results = from fa in Uvw_MyTable
where mapping.Contains(new { id = fa.Id, parent = fa.Parent })
select new { fa };
The first query is defined and executed in your single statement since you are using the ToList() method. Since this is now a list of objects, instead of the IQueryable, EF cannot translate this into SQL. You have to leave it as an IQueryable. If you change it to the following, it should work.
var mapping = updates
.Select(o => new { id = o.Id, parent = o.Parent}); // NOT ToList() !!
var results = from fa in Uvw_MyTable
where mapping.Contains(new { id = fa.Id, parent = fa.Parent })
select new { fa };
To explain the error that EF gives - EF can only translate either IQueryable's into SQL, or primitive types (ints, strings, etc) into parameters. EF cannot translate objects into SQL, since it has no way of transporting the value of objects to the database in a meaningful way.
UPDATE:
The first query (I see now) is not a query at all, it's source is a variable called updates. If this is a list of objects in-memory, than this query cannot be translated as such, and you have to redesign you query (for example, loading Uvw_MyTable is memory). If 'updates' is an IQueryable (from the same context), than my answer given above should work.
I'm not sure I've understood your code/question correctly, but it would seem to me that something like this should work:
var results = from fa in Uvw_MyTable
where updates.Any(u => u.Id fa.Id && u.Parent = fa.Parent)
select fa;
I'm not clear on what your desired behaviour with the OtherTable is, but the general form of the linq expression above is a lot more efficient than the one you're using.
The key point is that you don't need to create anonymous types in order to compare multiple fields; therefore the first select (into mapping) is redundant.
In entity framework or Linq to sql, linq queries will ultimately be translated to sql queries. your linq query cant be translated to sql queries as you are using types or classes which is not convertible to sql .
EDIT: forgot to say I'm using Fluent NHibernate, even though the tag could hint about it anyway.
I have these entity classes:
class OuterLevel
{
ICollection<MidLevel> mid_items;
... other properties
}
class MidLevel
{
OuterLevel parent;
Inner1 inner1;
Inner2 inner2;
... other properties
}
class Inner1
{
int id;
string description;
}
class Inner2
{
int id;
string description;
}
I need to build a Linq query that returns a list of OuterLevel objects with all children populated properly.
Supposing all mappings are correct and working, the hard part I'm finding here is that the resulting query should be something like
SELECT * FROM OuterLevelTable OLT INNER JOIN MidLevelTable MLT ON (MLT.parentID = OLT.ID) INNER JOIN
Inner1Table ON (MLT.Inner1ID = Inner1Table.ID) INNER JOIN
Inner2Table ON (MLT.Inner2ID = Inner2Table.ID)
WHERE (Inner1Table.someproperty1 = somevalue1) AND (Inner2Table.someproperty2 = somevalue2)
The main problem is that two joins start from MidLevel object downward the hierarchy, so I cannot figure out which Fetch and FetchMany combination can be used without having the resulting query join two times the MidLevelTable, such as the following does:
return All().FetchMany(x => x.mid_items).ThenFetch(x => x.inner1).FetchMany(x => x.mid_items).ThenFetch(x => x.inner2);
I would like to return a IQueryable that can be further filtered, so I would prefer avoiding Query and QueryOver.
Thanks in advance,
Mario
what you want is not possible because when you filter on the joined tables the resulting records are not enough to populate the collections anyway. You better construct the query in one place to further tune it or set the collection batch size to get down SELECT N+1.
I have a method query like this:
public IList<BusinessObject> GetBusinessObject(Guid? filterId)
{
using (var db = new L2SDataContext())
{
var result = from bo in db.BusinessObjects
where (filterId.HasValue)
? bo.Filter == filterId.value
: true
orderby bo.Name
select SqlModelConverters.ConvertBusinessObject(bo);
return result.ToList();
}
}
At runtime, this throws a System.InvalidOperationException: Nullable object must have a value.
Looking at the Debugger, the problem is my Where Clause: Linq To SQL tries to convert the entire thing to SQL, so even if filterId is NULL, it will still try to access filterId.value.
I thought/hoped the C# compiler/CLR would evaluate that where clause as a code block and only send one of the two branches to Linq To SQL, but that's not how it works.
My refactored version works, but is not very elegant:
public IList<BusinessObject> GetBusinessObject(Guid? filterId)
{
using (var db = new L2SDataContext())
{
var temp = from bo in db.BusinessObjects select bo;
if(filterId.HasValue) temp = temp.Where(t => t.Filter == filterId.Value);
var result = from t in temp
orderby t.Name
select SqlModelConverters.ConvertBusinessObject(bo);
return result.ToList();
}
}
I know that Lazy-Evaluation will make sure that only one query is really sent, but having that temp object in there isn't that great really.
Did you try:
where filterId == null || t.Filter == filterId
Your fix is exactly correct. You are effectively trying to build up a query dynamically, based on your function input. It's a good idea to omit the where clause instead of supplying WHERE TRUE anyway. If I were writing this query, I would go with your fixed version myself.
It's not as pretty as using the language keywords, but it's still the right way to approach the query in my opinion.