This is my code:
var query = context.SomeEntities.Select(e => new SomeDto
{
Name = e.Title,
})
The variable query is IQueryable.
var list = new List<SomeDto>
{
new SomeDto
{
Name = "SomeName1"
},
new SomeDto
{
Name = "SomeName2"
}
};
Now I need union these collections,I write
query = query.Union(list)
But in this place I get exception
Unable to create a constant value of type 'SomeDto'. Only primitive types or enumeration types are supported in this context
I know that I can do ToList() for first collection,but I need union collections without call to database.How can I do this?
As user3825493 pointed, there are answers regarding similar issues. The first link pretty much explains the whole thing.
Your problem is that due to the fact that Union is mapped to SQL UNION, arguments of the operation are to be converted to data types acceptable by Entity Framework. However, the only types acceptable by it for items inside IEnumerable input parameters are, as specified in the error you get, primitive types and enum's.
You should either:
var unionOfQueryWithList = query.AsEnumerable().Union(list)
or, if you like to use the SQL facilities, write a stored procedure which performs the union as part of its logic and find the way to pass your data into SQL, such as this.
The former loads all the results of the query into a memory before performing the union. It is convenient for the cases when you have all the pre-filtered data returned from EF and only want to use the output for later processing in your code.
The later takes you out of the EF zone and required for the cases when you have operations which will benefit from running on SQL engine, such as filtering or joining DB data based on a complex data you have access to in your code.
either you convert the query to IEnumerable or you can try this :
list.Union(query); // use Union() of list not query :)
it will work and you still don't need to get data from DB you can reverse the order if you want :)
Related
It is just an example of my situation
var query =
Employee.Join(department,
emp => emp.depId,
dep=> dep.Id,
(emp, dep) =>
new EmployeeModel{ Name = emp.Name, Total = GetTotal(emp)});
public string GetTotal(emp)
{
//dynamically decide which column to pass to the stored procedure, that is
//why I am passing the whole object
total = sp(emp.column1, emp.column2);//here I pass the parameters to SP
return total;
}
And I get an exception here, I do not know how to solve it.
Any help would be greatly appreciated.
Thanks
Of course this is going to happen. Linq to entities doesn't know what GetTotal() is, and given how it is going to call a stored procedure for each row returned I suggest you move the join itself to a stored procedure and forget LINQ for a while here.
You have to understand the difference between IEnumerable and IQueryable.
IEnumerable
An object that implements IEnumerable, represents and enumerable sequence. It holds everything to ask for the first element of the sequence, and as long as you've got an element, you can ask for the next one.
At its lowest level this is done by calling GetEnumerator and repeatedly calling MoveNext / Current, until there are no more elements.
Higher level methods as foreach and IEnumerable LINQ methods that don't return IEnumerable<TResult>, like ToList / ToDictionary / Count / FirstOrDefault / Any / etc. all deep inside use GetEnumerator / MoveNext / Current.
IQueryable
An object that implements IQueryable, doesn't represent the enumerable sequence itself, it represents the potential to get an Enumerable sequence.
For this, an IQueryable has an Expression and a Provider.
The Expression represents what data must be fetched in some generic format. The Provider knows who will provide the data (quite often a database management system) and what language is used to communicate with this DBMS (usually SQL).
As long as you concatenate LINQ methods that return IQueryable<...>, you are only changing the Expression. The DBMS is not contacted.
When you start enumerating the sequence, either directly by calling GetEnumerator / MoveNext, or indirectly by using foreach, or calling a LINQ method that returns anything but IQueryable<...>, then the Expression is sent to the Provider who will translate the Expression into SQL and fetch the data. The fetched data is represented as an enumerable sequence, which your code can access using MoveNext / Current repeatedly.
The problem is, that your provider doesn't know how to translate the call to GetTotal(...) into SQL. In fact, although the guys who wrote Entity Framework did a marvellous job, there are several LINQ methods that your Provider can't handle. See Supported and Unsupported LINQ Methods (LINQ to Entities).
So you can't call your own methods when using IQueryable. Luckily there are several solutions for this.
AsEnumerable
Your DBMS is extremely optimized to select the data in the query. One of the slower parts of a database query is the transfer from the selected data from the DBMS to your local process. Hence it is wise to limit the amount of data being transferred.
If your DBMS doesn't need the output of your method, consider using AsEnumerable(). This will transfer your selected data to your local process in a smart way.
Depending on the Provider, it can fetch the data "per page", instead of fetching all data, so if you end your LINQ with Take(3).ToList(), it will not have fetched all 1000 Customers from your database.
The result of the fetched data is represented as an IEnumerable<...> to you. The LINQ statements after that are executed on your local machine, so you can call your local methods.
Alas, as you use GetTotal in your Join, you can't use this method. If you would use AsEnumerable before the Join, you would transfer all Employees and all Departments to your local process, who had to do the join.
Convert GetEmp
You use GetEmp in parameter resultSelector of your Join. This parameter has the following generic format
System.Linq.Expressions.Expression<Func<TOuter,TInner,TResult>> resultSelector
Of in your case:
Expression<Employee, Department, EmployeeModel>
So you'll have to change your GetEmp, such that it creates this expression.
Expression<Employee, Department, EmployeeModel> EmployeeModelExpression =
(employee, department) => new EmployeeModel
{
Name = employee.Name,
Total = ???
};
Alas you forgot to tell us how you calculate the Total in SP.
You should only use fairly simple calculations, like:
Total = employee.Column1 + employee.Column2,
Total = (employee.Column1 < 50) ? employee.Column1 : employee.Column2,
Total = employee.Column1 ?? employee.Column2
I have a local List with entities, some hundreds, and I have a SQL Server table where I store the ID of the successful processed entities, some millions. I would like to know, which entities form my local set are not yet processed i.e. are not in the SQL Table.
The first approach is to iterate through the local list with the following Linq statement:
Entity entity = db.Entities.FirstOrDefault(m => m.ID == ID);
if (entity == null) { NewList.Add(ID) }
the NewList would then contain all the new entities. However this is very slow.
In LINQ, how would you send the entire local list to the SQL Server with one call and then return the ones not in the SQL table?
Do you really have to create a temporary table with my local list, then left-join on the already processed table and return the ones with a null?
Use .Contains method to retrieve already processed ids
and Except to create list of not yet processed ids.
var localList = new List<int> { 1, 2, 3 };
var processed = db.Entities
.Where(entity => localList.Contains(entity.Id))
.Select(entity => entity.Id)
.ToList();
var notProcessed = localList.Except(processed).ToList();
It will depend on provider, but .Contains should generate sql like:
SELECT Id FROM Entity WHERE Id IN (1, 2, 3)
suggestion:
create a temp table and insert your IDs
select your result on the SQL side
EDIT:
"Can you do that in LINQ?"
TL;DR:
yes* but that's an ugly piece of work, write the SQL yourself
*)depends on what you mean with "in" LINQ, because that is not in the scope of LINQ. In other words: a LINQ expression is one layer too abstract, but if you happen to have an LINQ accessible implementation for this, you can use this in your LINQ statements
on the LINQ expression side you have something like:
List<int> lst = new List<int>() { 1,2,3 };
List<int> result = someQueryable.Where(x=>lst.Contains(x.ID)).Select(x=>x.ID).ToList();
the question now is: what happens on the SQL side (assuming the queryable leads us to a SQL database)?
the queryable provider (e.g. Entity Framework) somehow has to translate that into SQL, execute it and come back with the result
here would be the place to modify the translation...
for example examine the expression tree with regard to the object that is the target for the Contains(...) call and if it is more than just a few elements, go for the temp table approach...
the very same LINQ expression can be translated into different SQL commands. The provider decides how the translation has to be done.
if your provider lacks support for large Contains(...) cases, you will probably experience poor performance... good thing is usually nobody forces you to use it this way ... you can skip linq for performance optimized queries, or you could write a provider extension yourself but then you are not on the "doing something with LINQ"-side but extending the functionality of your LINQ provider
if you are not developing a large scalable product that will be deployed to work with different DB-Backends, it is usually not worth the effort... the easier way to go is to write the sql yourself and just use the raw sql option of your db connection
I am using Telerik Open/Data Access ORM against an ORACLE.
Why do these two statements result in different SQL commands?
Statement #1
IQueryable<WITransmits> query = from wiTransmits in uow.DbContext.StatusMessages
select wiTransmits;
query = query.Where(e=>e.MessageID == id);
Results in the following SQL
SELECT
a."MESSAGE_ID" COL1,
-- additional fields
FROM "XFE_REP"."WI_TRANSMITS" a
WHERE
a."MESSAGE_ID" = :p0
Statement #2
IQueryable<WITransmits> query = from wiTransmits in uow.DbContext.StatusMessages
select new WITransmits
{
MessageID = wiTranmits.MessageID,
Name = wiTransmits.Name
};
query = query.Where(e=>e.MessageID == id);
Results in the following SQL
SELECT
a."MESSAGE_ID" COL1,
-- additional fields
FROM "XFE_REP"."WI_TRANSMITS" a
The query generated with the second statement #2 returns, obviously EVERY record in the table when I only want the one. Millions of records make this prohibitive.
Telerik Data Access will try to split each query into database-side and client-side (or in-memory LINQ if you prefer it).
Having projection with select new is sure trigger that will make everything in your LINQ expression tree after the projection to go to the client side.
Meaning in your second case you have inefficient LINQ query as any filtering is applied in-memory and you have already transported a lot of unnecessary data.
If you want compose LINQ expressions in the way done in case 2, you can append the Select clause last or explicitly convert the result to IEnumerable<T> to make it obvious that any further processing will be done in-memory.
The first query returns the full object defined, so any additional limitations (like Where) can be appended to it before it is actually being run. Therefore the query can be combined as you showed.
The second one returns a new object, which can be whatever type and contain whatever information. Therefore the query is sent to the database as "return everything" and after the objects have been created all but the ones that match the Where clause are discarded.
Even though the type were the same in both of them, think of this situation:
var query = from wiTransmits in uow.DbContext.StatusMessages
select new WITransmits
{
MessageID = wiTranmits.MessageID * 4 - 2,
Name = wiTransmits.Name
};
How would you combine the Where query now? Sure, you could go through the code inside the new object creation and try to move it outside, but since there can be anything it is not feasible. What if the checkup is some lookup function? What if it's not deterministic?
Therefore if you create new objects based on the database objects there will be a border where the objects will be retrieved and then further queries will be done in memory.
I am writing a reporting system which is flexible. As a part of it I create SQl statements by concatenating like
sql=" select * from a_v where ename=1 "
I want to know How I can use
db.Database.SqlQuery(sql)
to return collection of anonymous records so it is truly flexible.
I can't find a way to do it as it seems to be strongly typed.
Can it be made to return anonymous type records . An example would be great
I may have interpreted your goal incorrectly, but it sounds to me that you want to use projections. You can return lists of anonymous types from Linq. An example would be:
var anonymousListOfBoxes =
from b in CustomerBoxes
select new {
b.Customer,
b.BoxID,
b.Barcode
};
now, anonymousListOfBoxes will be a System.Linq.IQueryable<(anonymous)>. Each item will have 3 properties - Customer, BoxID and Barcode. The list will not be a collection of CustomerBoxes as it would be if you did not use a projection with "select".
I have a linq to sql query where I have to perform union two set of records.
And I do not have equal number of fields, so added the null
eg my psuedo code is
var Values=( from c in containers
some joins
select new PValues
{
regionid=r.regionid,
roomid=r.roomid,
floorid=r.floorid,
maxarea=r1.maxarea,
minarea=r1.minarea,
avgarea=r1.avgarea,
maxheight=r1.maxheight,
minheight=r1.minheight
})
.union
( from c in containers
some joins
select new PValues
{ regionid=j.regionid,
roomid=j.roomid,
floorid=j.floorid,
maxarea=null,
minarea=null,
avgarea=null,
maxheight=j1.maxheight,
minheight=j1.minheight
})
after googling some hours I came to understand that it is bug in 3.5 framework.
Now I want to retrieve the result.
How do I do that
I tried framing into two seperate iqueryable
var a= first query
var b =second query
ilist result =a.union b
This too results in the same error.
How should I form it
Thanks in advance
Regards
Hema
This is most likely the result of a known issue with LINQ to SQL. When a column value is referenced twice, it gets optimized out of the result set (even though you may need it to make unions line up).
The most general-purpose work-around is to use let statements:
var Values=(
from c in containers
some joins
//You'll need one let statement for each COLUMN, even if they share a value.
let maxAreaValue = null
let minAreaValue = null
let avgAreaValue = null
select new PValues
{
regionid=j.regionid,
roomid=j.roomid,
floorid=j.floorid,
maxarea=maxAreaValue,
minarea=minAreaValue,
avgarea=avgAreaValue,
maxheight=j1.maxheight,
minheight=j1.minheight
});
More information:
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=355734
http://slodge.blogspot.com/2009/01/linq-to-sql-and-union-all-queries.html
SqlException about UNION, INTERSECT and EXCEPT
In SQL, this typically means that you need to cast those nulls as the appropriate data type of the first part of the union, so make them the default values for the types you need, like "" or String.Empty for string values, or 0 for ints, etc...
Cast the nulls as mentioned by #GalacticJello, but also cast the first query's parameters that will have nulls in the other query to a nullable version of the type. For instance, if minarea is decimal, cast it to a decimal?; you can do this via: new Nullable(i.minarea) to ensure it infers the correct type.
It's probably inferring a different anonymous signature for both queries. Also, to mitigate the problems, make a class with these properties and change the query from:
select new { .. }
to
select new MyObj { .. }
And that will resolve it too.