We Keep Coding

LINQ - Group By with Having?

I have a db table which is having data like below.
Name Tool Security QUANTITY PRICE
ABC ML XXX 100 50
ABC DB XXX -50 50
XYZ CS YYY 30 30
My requirement is to group the name and security and pick only that record which is having both negative and positive quantity. In T-SQL this is my query which is perfectly fine. Need similar in LINQ. For example in above it will give both rows for ABC & XXX.
select t1.* from MyTable as t1
inner join
(
select Name,Security from MyTable
group by Name, Security
HAVING min(Quantity)<0 and max(Quantity)>0
) as t2 on t1.Name=t2.Name and t1.Security =t2.Security
This is my inner query but it's not working.
var Positions = from r in lstpositions
group r by new { r.Name, r.Security} into grp
where grp.Min(x => x.Quantity<0) && grp.Max(x => x.Quantity >0)
select grp;
Any thoughts on this ?

The reason your query does not work is because you taking the min of the result of the comparison.
However I think you want any() not min and max
where grp.Any(x => x.Quantity<0) && grp.Any(x => x.Quantity >0)
This will check for any value below 0 and any value above 0. It will short circuit so it does not have traverse the entire list which should make it faster.

Or this
where grp.Min(x => x.Quantity) < 0 && grp.Max(x => x.Quantity) > 0

Speed up the linq group by statement

I have a table like this
UserID Year EffectiveDate Type SpecialExpiryDate
1 2015 7/1/2014 A
1 2016 7/1/2015 B 10/1/2015
there is no ExpriyDate in the table because it is only valid for one year, so the expiry date can be calculated from the effective date by adding a year.
The result I want to get is like this (the current year's effective date and the next year's expiry date)
UserID EffectiveDate ExpiryDate
1 7/1/2014 7/1/2016
And If the user's type is B, then there will be a special expiry date, so for this person, the result will be
UserID EffectiveDate ExpiryDate
1 7/1/2014 10/1/2015
Here is the code I wrote
var result = db.Table1
.Where(x => x.Year>= 2015 && (x.Type == "A" || x.Type == "B"))
.GroupBy(y => y.UserID)
.OrderByDescending(x => x.FirstOrDefault().Year)
.Select(t => new
{
ID = t.Key,
Type = t.FirstOrDefault().Type,
EffectiveDate = t.FirstOrDefault().EffectiveDate,
ExpiryDate = t.FirstOrDefault().SpecialExpiryDate != null ? t.FirstOrDefault().SpecialExpiryDate : (t.Count() >= 2 ? NextExpiryDate : CurrentExpiryDate)
}
);
The code can get the result I need, but the problem is that in the result set there are about 10000 records which took about 5 to 6 seconds. The project is for a web search API, so I want to speed it up, is there a better way to do the query?
Edit
Sorry I made a mistake, in the select clause it should be
EffectiveDate = t.LastOrDefault().EffectiveDate
but in the Linq of C#, it didn't support this LastOrDefault function transfered to sql, and it cause the new problem, what is the easiest way to get the second item of the group?

You could generate the calculated data on the fly, using a View in your database.
Something like this (pseudocode):
Create View vwUsers AS
Select
UserID,
Year,
EffectiveDate,
EffectiveData + 1 as ExpiryDate, // <--
Type,
SpecialExpiryDate
From
tblUsers
And just connect your LINQ query to that.

Try this:
var result =
db
.Table1
.Where(x => x.Year>= 2015 && (x.Type == "A" || x.Type == "B"))
.GroupBy(y => y.UserID)
.SelectMany(y => y.Take(1), (y, z) => new
{
ID = y.Key,
z.Type,
z.EffectiveDate,
ExpiryDate = z.SpecialExpiryDate != null
? z.SpecialExpiryDate
: (t.Count() >= 2 ? NextExpiryDate : CurrentExpiryDate),
z.Year,
})
.OrderByDescending(x => x.Year);
The .SelectMany(y => y.Take(1) effectively does the .FirstOrDefault() part of your code. By doing this once rather than for many properties you may improve the speed immensely.
In a test I performed using a similarly structured query I got these sub-queries being run when using your approach:
SELECT t0.increment_id
FROM sales_flat_order AS t0
GROUP BY t0.increment_id
SELECT t0.hidden_tax_amount
FROM sales_flat_order AS t0
WHERE ((t0.increment_id IS NULL AND #n0 IS NULL) OR (t0.increment_id = #n0))
LIMIT 0, 1
-- n0 = [100000001]
SELECT t0.customer_email
FROM sales_flat_order AS t0
WHERE ((t0.increment_id IS NULL AND #n0 IS NULL) OR (t0.increment_id = #n0))
LIMIT 0, 1
-- n0 = [100000001]
SELECT t0.hidden_tax_amount
FROM sales_flat_order AS t0
WHERE ((t0.increment_id IS NULL AND #n0 IS NULL) OR (t0.increment_id = #n0))
LIMIT 0, 1
-- n0 = [100000002]
SELECT t0.customer_email
FROM sales_flat_order AS t0
WHERE ((t0.increment_id IS NULL AND #n0 IS NULL) OR (t0.increment_id = #n0))
LIMIT 0, 1
-- n0 = [100000002]
(This continued on for two sub-queries per record number.)
If I ran my approach I got this single query:
SELECT t0.increment_id, t1.hidden_tax_amount, t1.customer_email
FROM (
SELECT t2.increment_id
FROM sales_flat_order AS t2
GROUP BY t2.increment_id
) AS t0
CROSS APPLY (
SELECT t3.customer_email, t3.hidden_tax_amount
FROM sales_flat_order AS t3
WHERE ((t3.increment_id IS NULL AND t0.increment_id IS NULL) OR (t3.increment_id = t0.increment_id))
LIMIT 0, 1
) AS t1
My approach should be much faster.

LINQ Count multiple values from a collection

In SQL what I'm trying to accomplish is
SELECT
SUM(CASE WHEN Kendo=1 THEN 1 ELSE 0 END) as KendoCount,
SUM(CASE WHEN Icenium=1 THEN 1 ELSE 0 END) as IceniumCount
FROM
Contacts
I'd like to do this in a C# program using LINQ.
Contacts is a List where Contact has many Booleans such as Kendo and Icenium and I need to know how many are true for each of the Booleans.

At least with LINQ to SQL, the downside of the count functions is that it requires separate SQL requests for each .count method. I suspect Jessie is trying to run a single scan over the table rather than multiple scans for each predicate. Depending on the logic and number of columns you are creating, this may not perform as well. Closer to the original request, try using sum with a ternary if clause as such (from Northwind):
from e in Employees
group e by "" into g
select new {
isUs = g.Sum (x => x.Country == "USA" ? 1 : 0),
NotUs = g.Sum (x => x.Country != "USA" ? 0 : 1)
}
LINQ to SQL generates the following (YMMV with other ORM's):
SELECT SUM(
(CASE
WHEN [t1].[Country] = #p1 THEN #p2
ELSE #p3
END)) AS [isUs], SUM(
(CASE
WHEN [t1].[Country] <> #p4 THEN #p5
ELSE #p6
END)) AS [NotUs]
FROM (
SELECT #p0 AS [value], [t0].[Country]
FROM [Employees] AS [t0]
) AS [t1]
GROUP BY [t1].[value]

var KendoCount = db.Contacts.Where(x => x.Kendo).Count();
var IceniumCount = db.Contacts.Where(x => x.Icenium).Count();

I would do this as two separate queries:
int kendoCount = db.Contacts.Count(c => c.Kendo);
int iceniumCount = db.Contacts.Count(c => c.Icenium);
Given that these queries will automatically translate into optimized SQL, this will likely be similar in speed or even potentially faster than any query option, and is far simpler to understand.
Note that, if this is for Entity Framework, you'll need to write this as:
int kendoCount = db.Contacts.Where(c => c.Kendo).Count();
int iceniumCount = db.Contacts.Where(c => c.Icenium).Count();

var result = Contacts
.GroupBy(c => new
{
ID = "",
})
.Select(c => new
{
KendoCount = c.Sum(k => k.Kendo ? 1 : 0),
IceniumCount = c.Sum(k => k.Icenium ? 1: 0),
})
.ToArray()

Obtain the missing number [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Find the smallest unused number in SQL Server
I have this table in Sql server
ID | LetterID | LetterName
ID => int and identity
LetterID => int and unique and NotNull
LetterName => string
LetterID fill from my C# application and user set number for it.for example 1,2,3,4,5,...,100,.. (increment an unit for each row) and now my LetterID is 100 but Sometimes user delete one row from table for example delete row where LetterID is 50, now for insert new row (in application) I suggested to him LetterID chose 50, How can I get the missing numbers from table?

var output = Enumerable.Range(1, list.Max(item => item.LetterID))
.Except(list.Select(item => item.LetterID))

get the blank/missing row
SELECT TOP 1 x.LetterID +1
FROM tablename x
WHERE NOT EXISTS(SELECT * FROM tablename xx WHERE xx.LetterID = x.LetterID + 1)
ORDER BY x.LetterID

select t1.ID, t1.LetterID-1
from yourtable t1
left join yourtable t2
on t1.LetterID = t2.LetterID+1
and t1.ID = t2.ID
where t2.ID is null
and t1.LetterID>(select MIN(LetterID) from yourtable where ID = t1.ID)

;with cte as
(
select max(LetterID) as id from your_table
union all
select id-1 from cte where id>1
)
select cte.id as LetterID
from cte
left join your_table yt on cte.id=yt.LetterID
where yt.id is null
order by cte.id

How should I 'rotate' or 'flatten' this research data? PIVOT, self join or something else?

I'm having a really hard time finding any examples that are close to what I'm doing, or I'm simply not understanding the examples I'm finding.
I have a research database that contains a persons responses to multiple questions at different points in time. 'Admin#' below represents which "administration" of the test the data represents. Or you can think of it as containing which 'time' the test was given, for example, time1, time2, time3
RespondentID# Admin# Question1 Question2 Question3 Question4 Question5
1 1 A B C D E
1 2 E D C B A
1 3 Q W E R T
2 1 Z X C V B
2 2 P O I U Y
2 3 Y H N U J
What I need to do now is arrange this data so that each set of responses for a particular respondent is in the same row. So we'd take the 5 question fields and turn them into 15 question fields,
RespondentID# Admin1Question1 Admin1Question2 Admin1Question3 Admin1Question4 Admin1Question5 Admin2Question1 Admin2Question2 Admin2Question3 Admin2Question4 Admin2Question5 Admin3Question1 Admin3Question2 Admin3Question3 Admin3Question4 Admin3Question5
As you can see, every field that begins with Admin1 would correspond to the row in the example above that has a Admin# value of 1.
Please forgive me if I'm not explaining this properly.
To further complicate matters, the maximum number of "administrations" or "times" can increase in the future. Currently it is 3, but it is possible that the same test could be administered 4, 5 or more times in the future. Whatever solution used for this problem can be static and then updated by hand to account for additional "times" in the future, but it would be awesome if the solution dynamically accounted for an unspecified number of "times".
This data is stored in a MS SQL 2005 database, so tsql is obviously an option, but if a better solution exists in C# or LINQ (the overall project is an asp.net app), I'm open to that as well. Whatever you think works best! :)
Thanks so much for reading my question!

The basic approach I'm following is to manually pivot using code like:
select RespondentID,
min(case when Admin=1 then Question1 else null end) Admin1_Question1,
min(case when Admin=2 then Question1 else null end) Admin2_Question1,
min(case when Admin=3 then Question1 else null end) Admin3_Question1
from tests
group by RespondentID
So, using a dynamic t-sql statement we build and execute the query for this as follows:
declare #select varchar(max)
select #select = coalesce(#select+',','')+
'min(case when Admin='+a+' then '+q+' else null end) as [Admin'+a+'_'+q+']'
from (select distinct cast(Adminas varchar(10)) a from tests) p1
cross join (
select 'Question1' q union
select 'Question2' union
select 'Question3' union
select 'Question4' union
select 'Question5'
) p2
order by a, q
declare #sql varchar(max)
set #sql = 'select RespondentID, '+#select+' from tests group by RespondentID'
execute(#sql)
It's not the most dynamic t-sql solution available, but it should work!

What I'm thinking is to take each record of 5 questions, and normalize Respondent, Admin and Question into a list of individual answers, which you can then group by Respondent.
var myResultsList = GetResultsFromDatabase();
var normalizedResults = myResultsList
.SelectMany(r=>new[]{
new{Respondent = r.RespondentId, Admin = r.AdminId, Question = 1, Answer= r.Question1},
new{Respondent = r.RespondentId, Admin = r.AdminId, Question = 2, Answer = r.Question2},
new{Respondent = r.RespondentId, Admin = r.AdminId, Question = 3, Answer = r.Question3},
new{Respondent = r.RespondentId, Admin = r.AdminId, Question = 4, Answer = r.Question4},
new{Respondent = r.RespondentId, Admin = r.AdminId, Question = 5, Answer = r.Question5},
};
//finding a single answer, by respondent, admin and question:
normalizedList.FirstOrDefault(x=>x.Respondent == 1 && x.Admin == 2 && x.Question == 1);
Now you have a list of an anonymous type with Respondent, Admin, Question and Answer fields. You can now group these elements by Respondent, and produce a Lookup (basically a Dictionary of Lists) keyed by Respondent ID:
var groupedResults = normalizedResults.GroupBy(r=>r.RespondentID);
//Get all records for Respondent # 1, ordered by Admin and Question:
var oneRespondentsResults = normalizedResults[1].OrderBy(x=>x.Admin).ThenBy(x=>x.Question);
If you really wanted to get fancy, you could set them up in a nested Dictionary structure and refer to the answers by a unique combination of key fields:
var nestedDictionary = normalizedResults
.ToDictionary(x=>x.Respondent,
x=>nestedDictionary.Where(x2=>x2.Respondent == x.Respondent)
.ToDictionary(x2=>x2.Admin,
x2=>nestedDictionary.Where(x3=>x3.Respondent == x2.Respondent && x3.Admin == x2.Admin)
.ToDictionary(x3=>x3.Question, x3=>x3.Answer)));
//All that mess makes getting to a single value pretty easy:
var answer = nestedDictionary[1][2][1]; //Respondent 1, Admin 2, Question 1
If these results need to be used outside the function that creates them, set up a struct or simple class to take the place of the anonymous type (you can still use the inferred array initializer), or use the nested Dictionary (which will be keyed or valued in the primitive types contained in the anonymous type).

After reading #KeithS's answer, I thought of the following approach using PIVOT and UNPIVOT:
Use UNPIVOT to normalize the original data to RepsondentID, FullQuestionID, Answer:
select RespondentID,
[FullQuestionID] = 'Admin'+cast(admin as varchar)+'_'+question,
Answer
from (
select RespondentID, Admin, Question1, Question2, Question3, Question4, Question5
from tests
) t UNPIVOT (
answer for question in (Question1, Question2, Question3, Question4, Question5)
) up
Then use PIVOT to un-normalize the data into your desired RespondentID list:
;with data as (
--unpivot code
)
select RespondentID, [Admin1_Question1], [Admin2_Question1], [Admin3_Question1]
from data
PIVOT (min(Answer) for FullQuestionID in
([Admin1_Question1], [Admin2_Question1], [Admin3_Question1])
) p
Then finally, you can use dynamic t-sql to build the list of all Admin/Question combinations. With everything together it looks like the following:
declare #list varchar(max)
select #list = coalesce(#list+',','')+'[Admin'+a+'_'+q+']'
from (select distinct cast(admin as varchar) a from tests) p1
cross join (
select 'Question1' q union
select 'Question2' union
select 'Question3' union
select 'Question4' union
select 'Question5'
) p2
order by a, q
declare #sql varchar(max)
set #sql =
';with data as (
select RespondentID, [FullQuestionID]=''Admin''+cast(Admin as varchar)+''_''+question, Answer
from (
select respondentID, Admin, Question1, Question2, Question3, Question4, Question5
from tests
) p
UNPIVOT
(answer for question in
(Question1, Question2, Question3, Question4, Question5)
) as unPvt
)
select respondentID, '+#list+'
from data d
PIVOT (min(answer) for FullQuestionID in
('+#list+')
) p'
exec(#sql)