I have a local collection of record Id's (integers).
I need to retrieve records that have every one of their child records' ids in that local collection.
Here is my query:
public List<int> OwnerIds { get; private set; }
...
filteredPatches = from p in filteredPatches
where OwnerIds.All(o => p.PatchesOwners.Select(x => x.OwnerId).Contains(o))
select p;
I am getting this error:
Local sequence cannot be used in Linq to SQL implementation of query operators except the Contains() operator.
I understand that .All() isn't supported by Linq to SQL, but is there a way to do what I am trying to do?
Customers where
OrderIds in the child collection are a subset of the IDs in the in-memory collection.
from c in myDC.Customer
where c.Orders.All(o => myList.Contains(o.ID))
select c;
Customers where
OrderIds in the in-memory collection are a subset of the IDs in the child collection.
from c in myDC.Customers
where (from o in c.Orders
where myList.Contains(o.ID)
group o.ID by o.ID).Distinct().Count() == myList.Count()
select c;
Customers where
OrderIds in the in-memory collection are set-equal to the IDs in the child collection.
from c in myDC.Customers
let Ids = c.Orders.Select(o => o.ID).Distinct()
where Ids.Count() == myList.Count()
&& Ids.All(id => myList.Contains(id))
select c;
All of these generated sql for me.
PS - these presume the IDs are already distinct in myList. If they aren't yet, use:
myList = myList.Distinct().ToList();
PSS - good for lists up to ~2000 items. Higher than that will get translated to sql, and then sql server will barf at the number of parameters.
I don't know of a way to do it with Linq to SQL. The problem is that you need to get your list over to the server so that it can query against it. (your list is in memory on your machine, SQL Server needs to do the filtering on the server)
With straight SQL, you could use a regular SELECT statement with the "in()" operator to do that. (don't go over 1,000 items in the "in")
You could insert all of the ID's into a temp table in SQL, and then join to the table (you could use LINQ with this solution, but it requires 2 steps - the insert (assuming you have a "sets" table), and then the joined query (and then a cleanup query to remove your set).
You could LINQ query without the filter condition and then filter on your in-memory set (not recommended if the unfiltered result set could be large).
what the compiler says is...
OwnerIds.Contains(someVariable)
is supported and it will be translated as:
WHERE someVariable IN (OwnerId1, OwnerId2, OwnerIdN)
now, we don't have all the informations of you query but if you can reformulate what you're trying to do to use Contains, you'll be ok.
Could you do a join OwnerIds?
Error said "Local sequence (means OwnerIds) cannot be used in Linq to SQL implementation of query operators except the Contains() operator."
So you can do:
1) load ALL filteredPatches rows from SQL
var loadedData = filteredPatches.Select(i => i).ToList();
2) filter data as simple local sequence
var result = loadedData.Where(i => i.PatchesOwners.All(o => OwnerIds.Contains(o.ID)));
Related
I have a local List with entities, some hundreds, and I have a SQL Server table where I store the ID of the successful processed entities, some millions. I would like to know, which entities form my local set are not yet processed i.e. are not in the SQL Table.
The first approach is to iterate through the local list with the following Linq statement:
Entity entity = db.Entities.FirstOrDefault(m => m.ID == ID);
if (entity == null) { NewList.Add(ID) }
the NewList would then contain all the new entities. However this is very slow.
In LINQ, how would you send the entire local list to the SQL Server with one call and then return the ones not in the SQL table?
Do you really have to create a temporary table with my local list, then left-join on the already processed table and return the ones with a null?
Use .Contains method to retrieve already processed ids
and Except to create list of not yet processed ids.
var localList = new List<int> { 1, 2, 3 };
var processed = db.Entities
.Where(entity => localList.Contains(entity.Id))
.Select(entity => entity.Id)
.ToList();
var notProcessed = localList.Except(processed).ToList();
It will depend on provider, but .Contains should generate sql like:
SELECT Id FROM Entity WHERE Id IN (1, 2, 3)
suggestion:
create a temp table and insert your IDs
select your result on the SQL side
EDIT:
"Can you do that in LINQ?"
TL;DR:
yes* but that's an ugly piece of work, write the SQL yourself
*)depends on what you mean with "in" LINQ, because that is not in the scope of LINQ. In other words: a LINQ expression is one layer too abstract, but if you happen to have an LINQ accessible implementation for this, you can use this in your LINQ statements
on the LINQ expression side you have something like:
List<int> lst = new List<int>() { 1,2,3 };
List<int> result = someQueryable.Where(x=>lst.Contains(x.ID)).Select(x=>x.ID).ToList();
the question now is: what happens on the SQL side (assuming the queryable leads us to a SQL database)?
the queryable provider (e.g. Entity Framework) somehow has to translate that into SQL, execute it and come back with the result
here would be the place to modify the translation...
for example examine the expression tree with regard to the object that is the target for the Contains(...) call and if it is more than just a few elements, go for the temp table approach...
the very same LINQ expression can be translated into different SQL commands. The provider decides how the translation has to be done.
if your provider lacks support for large Contains(...) cases, you will probably experience poor performance... good thing is usually nobody forces you to use it this way ... you can skip linq for performance optimized queries, or you could write a provider extension yourself but then you are not on the "doing something with LINQ"-side but extending the functionality of your LINQ provider
if you are not developing a large scalable product that will be deployed to work with different DB-Backends, it is usually not worth the effort... the easier way to go is to write the sql yourself and just use the raw sql option of your db connection
I went through the many questions that were asked regarding this and tried to find solution but no luck. So here is my situation:
private IQueryable<tblB> MT;
var IDs = (from z in db1.tblA
where z.TA == User.Identity.Name
select z).ToArray();
MT = from s in db2.tblB
join a in IDs on s.BP equals a.BP
select new tblB() { LastName = s.LastName});
return View(MT.ToPagedList(pageNumber, pageSize));
I'm getting exception at the return statement - $exception {"Unable to create a constant value of type 'tblA'. Only primitive types or enumeration types are supported in this context."} System.NotSupportedException
When I debug IDs array, I see it has data from tblA but 2nd query with join doesn't seem to work. What mistake am I making. Help!!!
You need to use Contains in order to generate and IN sql clause:
First, change the first query to return the primitive data you need:
var IDs = (from z in db1.tblA
where z.TA == User.Identity.Name
select z.BP).ToArray();
Then use that in-memory list in the second query:
MT = from s in db2.tblB
where IDs.Contains(s.BP)
select new tblB() { LastName = s.LastName});
By the way, this is not a 2 contexts operations. You're loading data from the first context into memory (notice the .ToArray()) and then using these in-memory data to query the second context.
So you want as a result all elements of tblB that have a property BP equal to at least one of the BP properties of the elements in IDs.
The problem is that after ToList() IDs is in local memory, while your db1.tblB is on the database. You have to bring both sequences to the same platform, either both to your database, or both to local memory. Whatever is more efficient depends on the actual sizes of the sequences and of the results
Use Contains if you want to perform the query on database side. The complete list of IDs will be transferred to the database, where the query will be executed and the results will be transferred to local memory.
Use this method if IDs is fairly short and your result is not almost the complete tblA
var result = db2.tblB
.Where(elementOfTableB => IDs.Contains(elementOftableB);
No need to create a new tblB object, apparently you want the complete tblB object.
Use AsEnumerable if you expect that there are a lot of IDs in comparison to the number of element in tblB. Transferring the IDs to the database would take considerably more time than transferring the complete tblB to local memory.
db2.TblB // take all tblBelements
.AsEnumerable() // bring to local memory
.join(IDs, // join with list IDs
tblBElement => tblBElement.BP // take from each tblBElement the BP
idElement => idElement.BP // take from each ID the BP
(tblBElement, idElement) => // when they match take the two records
// and get what you want.
tblBElement); // in this case the complete tblBElement
I'm trying to join a row of parent data to a related collection which has been squeezed into a single piece of data.
I have an IQueryable of Orders:
IQueryable<Order> orderList = context.Set<Order>
.Where("OrderDate >= #0", startDate)
.Where("OrderDate <= #0", endDate)
and a related IList<KeyValuePair<int, string>> where each KVP contains the OrderID and a concatenated string of the product names from each Order Line. I want to join the Value from the correct KeyValuePair to the Order info based on the OrderID Key.
To illustrate, the desired output would look something like:
OrderNum OrderDate Customer State OrderTotal Products_Ordered
12345 12/12/2012 J.Bloggs WA $25.50 Bolts, Hammer, Suregrip Clamp
I am trying a linq join that looks like this:
IQueryable result = from o in orders
join line in orderLines on o.OrderID equals line.Key
select new
{
o.OrderNum
o.OrderDate,
o.Customer.CustomerFullName,
o.DeliverAddress.State,
o.TotalPrice,
line.Value
}
The method performing the join seems to work, but when I access the returned IQueryable, I get a NotSupportedException: Unable to create a constant value of type 'System.Collections.Generic.KeyValuePair`2'. Only primitive types or enumeration types are supported in this context.
What am I doing wrong?
Your IQueryable is actually a specialized entity framework implementation that, when iterated, attempts to construct an SQL query by examining an expression tree, execute this query, and return an enumerable over the results. This is fragile, and your projections and queries can't be arbitrarily complex, or the expression -> SQL converter has no idea what to do with it.
Fixing this by materializing your IQueryable first is fine, but you don't even really need to do that. Why have a list of what is essentially tuples, when what you want is a dictionary that maps the order ID to a bunch of data?
IDictionary<int, string> orderLines = new Dictionary<int, string>();
// Add a dummy item
orderLines[1234] = "Hello, this, is, a, test";
// Get your combined view
// Assuming you have an order with 1234 as the ID, this should work
var result = from o in orders
select new
{
o.OrderNum
o.OrderDate,
o.Customer.CustomerFullName,
o.DeliverAddress.State,
o.TotalPrice,
Products = orderLines[o.OrderID]
}
Entity Framework has issues with some types, because it tries to push them to the backing database. The quick and easy solution is to pull the data down before performing the join.
var orderList = context.Set<Order>
.Where("OrderDate >= #0", startDate)
.Where("OrderDate <= #0", endDate)
.ToList(); // ------> Relevant line <------
var result = from o in orderList
join line in orderLines on o.OrderID equals line.Key
select new
{
o.OrderNum
o.OrderDate,
o.Customer.CustomerFullName,
o.DeliverAddress.State,
o.TotalPrice,
line.Value
};
It's just a change in where the joining happens, RDBMS- or client-side.
Performance won't be an issue, assuming you don't have a ton of data and all rows will be matched. Of course, you will have to pull down data that doesn't get joined, so if only a few of orderList will be in result, that might be worth a second thought, design-wise. The only alternative would be in pushing the KeyValuePair items to the server first, which probably isn't what you're looking for.
Joining a query result (IQueryable) with a normal enumerable stored in memory (IEnumerable<...>) doesn't make much sense. Think about it, the query doesn't get processed until it's materialized, injecting your own data in queries is done in the query text itself -- that's not what you really want, now is it?
I think what you expect from this is best achieved by first materializing the IQueryable into an IEnumerable<>, then doing a plain LINQ join on two IEnumerable<>s, which is trivial.
It's not like you'd be using the IEnumerable<> to filter the result set on the server side, you're not really losing any performance here.
Edit: Note that if you are using the IEnumerable<> to filter the results on the server side, you can do that! EF (I assume that's what you're using) has very strong special cases for things like IQueriable<>.Any<>() with an IEnumerable<>.Contains<>() inside it -- it inserts the literal values in the query text. It's just the actual join that doesn't make much sense in this context.
I have a requirement to create a linq query that selects data from a database table where a certain functional result is true.
I am using linq-to-sql and the function works well and I can write a basic query to get the data.
My issue is that I have an in-memory list of parameters and I essentially need to run the linq query multiple times (once for every parameter list item) and aggregate the results.
I have tried using the .Any() as a join but linq doesn't like joining non-database result sets with database result sets.
Some Sample Code:
Parameter list: // lets call it "l"
{
One,
Two,
Three
}
Query
From w in words where funcWord(l.item) == true select w;
So I would require a query that can run the above query once for every item in l and aggregate the results.
Any help is appreciated.
Thanks.
Multiple from statements will go through every combination, sort of like nested foreach statements:
from item in list
from w in words
where funcWord(item, w.name) == true
select w;
Interestingly, this gets translated to a SelectMany statement that is similar to Arithmomaniac's answer.
Try SelectMany, which aggregates the result of a one-to-many function, applied to each member.
In this case, the members are the elements of list, and the one-to-many function is your above, l-dependent query (though I rewrote it as a lambda function.)
list.SelectMany(l => words.Where(w => funcWord(l.item, w.name)));
Using the Entity Framework, when one executes a query on lets say 2000 records requiring a groupby and some other calculations, does the query get executed on the server and only the results sent over to the client or is it all sent over to the client and then executed?
This using SQL Server.
I'm looking into this, as I'm going to be starting a project where there will be loads of queries required on a huge database and want to know if this will produce a significant load on the network, if using the Entity Framework.
I would think all database querying is done on the server side (where the database is!) and the results are passed over. However, in Linq you have what's known as Delayed Execution (lazily loaded) so your information isn't actually retrieved until you try to access it e.g. calling ToList() or accessing a property (related table).
You have the option to use the LoadWith to do eager loading if you require it.
So in terms of performance if you only really want to make 1 trip to the Database for your query (which has related tables) I would advise using the LoadWith options. However, it does really depend on the particular situation.
It's always executed on SQL Server. This also means sometimes you have to change this:
from q in ctx.Bar
where q.Id == new Guid(someString)
select q
to
Guid g = new Guid(someString);
from q in ctx.Bar
where q.Id == g
select q
This is because the constructor call cannot be translated to SQL.
Sql's groupby and linq's groupby return differently shaped results.
Sql's groupby returns keys and aggregates (no group members)
Linq's groupby returns keys and group members.
If you use those group members, they must be (re-)fetched by the grouping key. This can result in +1 database roundtrip per group.
well, i had the same question some time ago.
basically: your linq-statement is converted to a sql-statement. however: some groups will get translated, others not - depending on how you write your statement.
so yes - both is possible
example:
var a = (from entity in myTable where entity.Property == 1 select entity).ToList();
versus
var a = (from entity in myTable.ToList() where entity.Property == 1 select entity).ToList();