We Keep Coding

Linq slow while SQL fast

When I'm trying the next linq query, it is slow (1.5 s):
var rslt = (from t in context.Set<SUB_Transactions>()
where
t.UpdateDate > query.LastUpdate &&
t.TransactionID > query.Index
select new
{
TransactionID = t.TransactionID
}).OrderBy(t => t.TransactionID).Take(query.Amount).ToList();
When converted to SQL, this query is super fast (40 ms):
SELECT TOP (300)
[Project1].[TransactionID] AS [TransactionID]
FROM ( SELECT
[Extent1].[TransactionID] AS [TransactionID]
FROM [dbo].[SUB_Transactions] AS [Extent1]
WHERE ([Extent1].[UpdateDate] > #p__linq__0) AND ([Extent1].[TransactionID] > #p__linq__1)
) AS [Project1]
ORDER BY [Project1].[TransactionID] ASC
What is going on here?
Removing the Take in the first query gives a fast result as well (given the fast that there are no new transactions)
There is a composite index on TransactionID and UpdateDate.

As far as your example goes, this could make your query faster but without some sample data it's not possible to test from our end:
var rslt = (from t in context.Set<SUB_Transactions>()
where t.TransactionID > query.Index // invert order of filter
&& t.UpdateDate > query.LastUpdate
order by t.TransactionID // you can orderby here
select t.TransactionID) // remove anonymous object
.Take(query.Amount)
.AsNoTracking() // you won't be changing IDs so no need to track them
.ToList();
You might also gain some performance if you used the DbSet<SUB_Transactions> instead of calling Set<SUB_Transactions>() which needs to locate the DbSet in the DbContext

Convert SQL to EF Linq

I have the following query:
SELECT COUNT(1)
FROM Warehouse.WorkItems wi
WHERE wi.TaskId = (SELECT TaskId
FROM Warehouse.WorkItems
WHERE WorkItemId = #WorkItemId)
AND wi.IsComplete = 0;
And since we are using EF, I'd like to be able to use the Linq functionality to generate this query. (I know that I can give it a string query like this, but I would like to use EF+Linq to generate the query for me, for refactoring reasons.)
I really don't need to know the results of the query. I just need to know if there are any results. (The use of an Any() would be perfect, but I can't get the write code for it.)
So... Basically, how do I write that SQL query as a LINQ query?
Edit: Table Structure
WorkItemId - int - Primary Key
TaskId - int - Foreign Key on Warehouse.Tasks
IsComplete - bool
JobId - int
UserName - string
ReportName - string
ReportCriteria - string
ReportId - int - Foreign Key on Warehouse.Reports
CreatedTime - DateTime

The direct translation could be something like this
var result = db.WorkItems.Any(wi =>
!wi.IsComplete && wi.TaskId == db.WorkItems
.Where(x => x.WorkItemId == workItemId)
.Select(x => x.TaskId)
.FirstOrDefault()));
Taking into account the fact that SQL =(subquery), IN (subquery) and EXISTS(subquery) in nowadays modern databases are handled identically, you can try this instead
var result = db.WorkItems.Any(wi =>
!wi.IsComplete && db.WorkItems.Any(x => x.WorkItemId == workItemId
&& x.TaskId == wi.TaskId));

Turns out that I just needed to approach the problem from a different angle.
I came up with about three solutions with varying Linq syntaxes:
Full method chain:
var q1 = Warehouse.WorkItems
.Where(workItem => workItem.TaskId == (from wis in Warehouse.WorkItems
where wis.WorkItemId == workItemId
select wis.TaskId).First())
.Any(workItem => !workItem.IsComplete);
Mixed query + method chain:
var q2 = Warehouse.WorkItems
.Where(workItem => workItem.TaskId == Warehouse.WorkItems
.Where(wis => wis.WorkItemId == workItemId)
.Select(wis => wis.TaskId)
.First())
.Any(workItem => !workItem.IsComplete);
Full query:
var q3 = (from wi in Warehouse.WorkItems
where wi.TaskId == (from swi in Warehouse.WorkItems
where swi.WorkItemId == workItemId
select swi.TaskId).First()
where !wi.IsComplete
select 1).Any();
The only problems with this is that it comes up with some really jacked up SQL:
SELECT
(CASE
WHEN EXISTS(
SELECT NULL AS [EMPTY]
FROM [Warehouse].[WorkItems] AS [t0]
WHERE (NOT ([t0].[IsComplete] = 1)) AND ([t0].[TaskId] = ((
SELECT TOP (1) [t1].[TaskId]
FROM [Warehouse].[WorkItems] AS [t1]
WHERE [t1].[WorkItemId] = #p0
)))
) THEN 1
ELSE 0
END) AS [value]

You can use the Any() function like so:
var result = Warehouse.WorkItems.Any(x => x.WorkItemId != null);
In short, you pass in your condition, which in this case is checking whether or not any of the items in your collection have an ID
The variable result will tell you whether or not all items in your collection have ID's.
Here's a helpful webpage to help you get started with LINQ: http://www.dotnetperls.com/linq

Subquery in the original SQL was a useless one, thus not a good sample for Any() usage. It is simply:
SELECT COUNT(*)
FROM Warehouse.WorkItems wi
WHERE WorkItemId = #WorkItemId
AND wi.IsComplete = 0;
It looks like, since the result would be 0 or 1 only, guessing the purpose and based on seeking how to write Any(), it may be written as:
SELECT CASE WHEN EXISTS ( SELECT *
FROM Warehouse.WorkItems wi
WHERE WorkItemId = #WorkItemId AND
wi.IsComplete = 0 ) THEN 1
ELSE 0
END;
Then it makes sense to use Any():
bool exists = db.WorkItems.Any( wi => wi.WorkItemId == workItemId & !wi.IsComplete );
EDIT: I misread the original query in a hurry, sorry. Here is an update on the Linq usage:
bool exists = db.WorkItems.Any( wi =>
db.WorkItems
.SingleOrDefault(wi.WorkItemId == workItemId).TaskId == wi.TaskId
&& !wi.IsComplete );
If the count was needed as in the original SQL:
var count = db.WorkItems.Count( wi =>
db.WorkItems
.SingleOrDefault(wi.WorkItemId == workItemId).TaskId == wi.TaskId
&& !wi.IsComplete );
Sorry again for the confusion.

Linq-to-SQL: arithmetic operation on consecutive elements

For example, I have a table:
Date |Value
----------|-----
2015/10/01|5
2015/09/01|8
2015/08/01|10
Is there any way using Linq-to-SQL to get a new sequence which will be an arithmetic operation between consecutive elements in the previously ordered set (for example, i.Value - (i-1).Value)? It must be executed on SQL Server 2008 side, not application side.
For example dataContext.GetTable<X>().OrderByDescending(d => d.Date).Something(.......).ToArray(); should return 3, 2.
Is it possible?

You can try this:
var q = (
from i in Items
orderby i.ItemDate descending
let prev = Items.Where(x => x.ItemDate < i.ItemDate).FirstOrDefault()
select new { Value = i.ItemValue - (prev == null ? 0 : prev.ItemValue) }
).ToArray();
EDIT:
If you slightly modify the above linq query to:
var q = (from i in Items
orderby i.ItemDate descending
let prev = Items.Where(x => x.ItemDate < i.ItemDate).FirstOrDefault()
select new { Value = (int?)i.ItemValue - prev.ItemValue }
).ToArray();
then you get the following TSQL query sent to the database:
SELECT ([t0].[ItemValue]) - ((SELECT [t2].[ItemValue]
FROM (SELECT TOP (1) [t1].[ItemValue]
FROM [Items] AS [t1]
WHERE [t1].[ItemDate] < [t0].[ItemDate]) AS [t2]
)) AS [Value]
FROM [Items] AS [t0]
ORDER BY [t0].[ItemDate] DESC
My guess now is if you place an index on ItemDate field this shouldn't perform too bad.

I wouldn't let SQL do this, it would create an inefficient SQL query (I think).
I could create a stored procedure, but if the amount of data is not too big I can also use Linq to objects:
List<x> items=dataContext.GetTable<X>().OrderByDescending(d => d.Date).ToList();//Bring data to memory
var res = items.Skip(1).Zip(items, (cur, prev) => cur.Value - prev.Value);
At the end, I might use a foreach for readability

Optimize LINQ query

I have a report that shows orders made to a determined merchant, and it was working fine until I needed to add a filter for payment status.
This is how I build the query, filter by filter:
var queryOrder = context.Orders.Select(m=>m);
if (viewModel.InitialDate.HasValue)
queryOrder = queryOrder.Where(m => m.CreatedDate.Date >= viewModel.InitialDate.Value);
(...) /* continues building the query, filter by filter */
if (viewModel.SelectedPaymentStatus != null)
queryOrder = queryOrder.Where(m => viewModel.SelectedPaymentStatus.Contains(m.Payments.Select(p => p.PaymentStatusId).Single().ToString()));
queryOrder = queryOrder.Where(m => m.MerchantId == merchantId);
When I run queryOrder, even if it's only a queryOrder.Count(), it takes over 1 minute to execute. Using SQL Server's profiling tool, I extracted the generated query as this:
SELECT [t0].[Id], [t0].[CustomerId], [t0].[MerchantId], [t0].[OrderNumber], [t0].[Amount], [t0].[SoftDescriptor], [t0].[ShippingMethod], [t0].[ShippingPrice], [t0].[IpAddress], [t0].[SellerComment], [t0].[CreatedDate]
FROM [dbo].[Order] AS [t0]
WHERE ([t0].[MerchantId] = #p0)
AND ((CONVERT(NVarChar,(
SELECT [t1].[PaymentStatusId]
FROM [dbo].[Payment] AS [t1]
WHERE [t1].[OrderId] = [t0].[Id]
))) IN (#p1, #p2, #p3, #p4, #p5, #p6, #p7, #p8))
the #p0 parameter is a Guid for merchantId, and the #p1 thru #p8 are numeral strings "1" thru "8", representing the paymentStatusId's.
If I skip the line:
if (viewModel.SelectedPaymentStatus != null)
queryOrder = queryOrder.Where(m => viewModel.SelectedPaymentStatus.Contains(m.Payments.Select(p => p.PaymentStatusId).Single().ToString()));
The query runs in under 1 second. But when I use it, the performance hits the floor. Any tips on how to solve this?

All your queries are deferred and this is both the good/bad part of linq. Try to split the queries and use some in-memory results.
Try removing the first query (doesn't make much sense really, you're returning the same collection) and amend the second query to be like this and see if it makes any difference.
var clause = context.Orders.Payments.Select(p => p.PaymentStatusId).Single().ToString();
if (viewModel.SelectedPaymentStatus != null)
var queryOrder = context.Orders.queryOrder.Where(m => viewModel.SelectedPaymentStatus.Contains(clause));

I've annotated the problem snippet:
if (viewModel.SelectedPaymentStatus != null) {
// Give me only orders from queryOrder where...
queryOrder = queryOrder.Where(
// ...my viewModel's SelectedPaymentStatus collection...
m => viewModel.SelectedPaymentStatus.Contains(
// ...contains the order's payment's PaymentStatusId...
m.Payments.Select(p => p.PaymentStatusId).Single()
// ... represented as a string?!
.ToString()
// Why are database IDs strings?
)
);
}
viewModel.SelectedPaymentStatus appears to be a collection of strings; therefore, you're asking the database to convert PaymentStatusId to an nvarchar and do string comparisons with the elements of SelectedPaymentStatus. Yuck.
Since viewModel.SelectedPaymentStatus is small, it might be better to create a temporary List<int> and use that in your query:
if (viewModel.SelectedPaymentStatus != null) {
// Let's do the conversion once, in C#
List<int> statusIds = viewModel.SelectedPaymentStatus.Select( i => Convert.ToInt32(i) ).ToList();
// Now select the matching orders
queryOrder = queryOrder.Where(
m => statusIds.Contains(
m.Payments.Select(p => p.PaymentStatusId).Single())
)
);
}

LINQ Count multiple values from a collection

In SQL what I'm trying to accomplish is
SELECT
SUM(CASE WHEN Kendo=1 THEN 1 ELSE 0 END) as KendoCount,
SUM(CASE WHEN Icenium=1 THEN 1 ELSE 0 END) as IceniumCount
FROM
Contacts
I'd like to do this in a C# program using LINQ.
Contacts is a List where Contact has many Booleans such as Kendo and Icenium and I need to know how many are true for each of the Booleans.

At least with LINQ to SQL, the downside of the count functions is that it requires separate SQL requests for each .count method. I suspect Jessie is trying to run a single scan over the table rather than multiple scans for each predicate. Depending on the logic and number of columns you are creating, this may not perform as well. Closer to the original request, try using sum with a ternary if clause as such (from Northwind):
from e in Employees
group e by "" into g
select new {
isUs = g.Sum (x => x.Country == "USA" ? 1 : 0),
NotUs = g.Sum (x => x.Country != "USA" ? 0 : 1)
}
LINQ to SQL generates the following (YMMV with other ORM's):
SELECT SUM(
(CASE
WHEN [t1].[Country] = #p1 THEN #p2
ELSE #p3
END)) AS [isUs], SUM(
(CASE
WHEN [t1].[Country] <> #p4 THEN #p5
ELSE #p6
END)) AS [NotUs]
FROM (
SELECT #p0 AS [value], [t0].[Country]
FROM [Employees] AS [t0]
) AS [t1]
GROUP BY [t1].[value]

var KendoCount = db.Contacts.Where(x => x.Kendo).Count();
var IceniumCount = db.Contacts.Where(x => x.Icenium).Count();

I would do this as two separate queries:
int kendoCount = db.Contacts.Count(c => c.Kendo);
int iceniumCount = db.Contacts.Count(c => c.Icenium);
Given that these queries will automatically translate into optimized SQL, this will likely be similar in speed or even potentially faster than any query option, and is far simpler to understand.
Note that, if this is for Entity Framework, you'll need to write this as:
int kendoCount = db.Contacts.Where(c => c.Kendo).Count();
int iceniumCount = db.Contacts.Where(c => c.Icenium).Count();

var result = Contacts
.GroupBy(c => new
{
ID = "",
})
.Select(c => new
{
KendoCount = c.Sum(k => k.Kendo ? 1 : 0),
IceniumCount = c.Sum(k => k.Icenium ? 1: 0),
})
.ToArray()