Optimize a large in clause from list of integer - c#

I would like to join on a large list of integers in SQL Server instead of a big IN clause.
My query :
SELECT
mmr.idContact,
mmR.idQuestion as IdQuestion,
MIN(mmR.idResponse) AS IdResponse
FROM MatchResponses mmR
--JOIN Contact c on c.idContact = mmR.idContact //to show the linked ids
JOIN Contact c on c.idSpecific in (1,2,3...10000)
WHERE myId= 300
GROUP By mmR.idContact, mmr.idQuestion
order by idContact, idQuestion
The IN clause is way too long, I can join mmR and COntact with an idContact.
The query takes 44s I would like to make it shorter using a JOIN
How can I declare the integers table "on the go" ?

My idea would be to handle the integer table using a temporary table in SQL. If you know the lower and upper limit of the integer table, it is easy to generate a temp_table in SQL and use it as a sub query with "In" Clause. It will not affect much on query performance.
Better to handle these from the DB rather than using an intermediate code to such as C# unless it is the requirement.
If you can attach sample schema and data, I can provide the code for you.

Thanks for all, I reseigned myself to simply not use the join or anything, no filter. The filtering wasn't more efficient, I after used LinQ (I only had 200 rows)
For the people looking for a solution :
I could have inserted all of the id in a temp table be careful you can't insert more than 1000 rows so use this trick:
DECLARE #tempTable TABLE (id INT)
INSERT INTO #EMPLOYEEDETAILS(id)
SELECT * FROM (VALUES
(1),(2),
.....
(10000)
) A(Col1)
And then the JOIN :
SELECT
mmr.idContact,
mmR.idQuestion as IdQuestion,
MIN(mmR.idResponse) AS IdResponse
FROM MatchResponses mmR
JOIN Contact c on c.idContact = mmR.idContact
JOIN #tempTable con on con.id = c.idSpecific
WHERE myId= 300
GROUP By mmR.idContact, mmr.idQuestion
order by idContact, idQuestion

Related

SQL: How to check if all IDs in a list are in a table

first of all I am sorry if this question is too obvious, since I am quite new in SQL.
So, I have a list of IDs (variable, depending how many products the user chooses). And I want to check if all of them are in a table. If one of them is not, the result of the query should be null. If all of them are there, the result should be all the rows where those IDs are.
How can I do this?
Best regards,
Flavio
Do a LEFT JOIN from the list to the table on the ID field. You'll get a null if there is no record
You can even put a WHERE clause like 'WHERE List.ID IS NULL' to only see those that aren't in the table
Edit: Original Poster did not say they were using C# when I wrote this answer
UNTESTED:
Not sure if this is the most efficient but it seems like it should work.
1st it generates a count of items in the table for your list. Next it cross joins the 1 result record to a query containing the entire list ensuring the count matches the count in your provided list and limiting the results to your list.
SELECT *
FROM Table
CROSS JOIN (
SELECT count(*) cnt
FROM table
WHERE ID in (yourlist)) b
WHERE b.cnt = yourCount
and ID IN (YourList)
Running two in statements seems like it would be terribly slow overall; but my first step when writing SQL is usually to get something that works and then seek to improve performance if needed.
Get the list of Ids into a table, (you can pass them as a table variable parameter to a Stored proc), then in the stored proc, write
assuming the list of ids from C# is in table variable #idList
Select * from myTable
Where id in (Select id from #idList)
and not exists
(Select * from #idList
where id Not in
(Select id from myTable))

Efficient way to count related entities of a many to many relation in EF

I would like to know how to efficiently count (SQL server side) the amount of distinct count of results for a specific range of a related entity that has a many to many relationship.
This is the current situation in entity Framework:
Table1 1<------->∞ Table2
Table2 ∞<------->∞ Table4
Table 2 and Table 4 have a many to many relationship and are linked with Table3 in SQL.
What I want is the distinct count of table4 results related to a specific range of Table1.
In LinQ to SQL the query is this:
(from dc in Table1
join vc in Table2 on dc.Table1Id equals vc.Table2Id
join vcac in Table3 on vc.Table2Id equals vcac.Table3Id
join ac in Table4 on vcac.Table3Id equals ac.Table4Id
where dc.Table1Id > 200000
group ac by ac.Table4Id into nieuw
select new { acid= nieuw.Key}).Count()
This lets SQL server return the count directly.
Because the extra table for the many to many relation ( Table3) is gone, I have had problems converting this query to L2E in query syntax. ( since I cannot join table 4 with table 2 with an inner join).
I have this in chained syntax, however, is this efficient ( does this fetch the whole list, or does it let SQLserver do the count, as I'm not sure this is an efficient way to select, Table 2 contains about 30.000 entries, I don't want it to fetch this result just to count it):
context.Table4.Where(a => a.Table2.Any(v => v.Table1Id > 200000)).Select(a => aTable4Id).Distinct().Count();
How would I go converting the Linq2SQL query into L2E in the query syntax ? Or is the chained syntax fine in this situation ?
The .Select() method uses deferred execution, meaning it won't actually run the query until you need the results. At that point in the chain it still exists only as a query definition. Then you modify with .Distinct() before getting .Count() - which does query the database using a SQL GROUP BY statement. So you should be good.

Execute SELECT for all returned rows from another SELECT within the same query

With this query:
SELECT id FROM org.employees WHERE {some_condition}
For every row from the above query, I need to call:
SELECT * FROM org.work_schedule(#employeeId, #fromDate, #toDate)
where org.work_schedule is table-valued function that process all of the employee's available work schedules and constraints and return two DATETIME (start, end) columns representing the availabilities of the given employee for the provided date range.
I am thinking using a cursor on the first query and feed a temporary table that would be returned. Is this the only solution?
The project is in C# and I could also accomplish this in C# directly, but I suspect it would be more optimal to do this entirely in SQL (SQL Server 2008).
This seems localized, and I would generalize the question with :
How can I execute a query (SELECT) for every row returned by another query (SELECT) and return the entire results in one call (dynamically do SELECT UNION SELECT UNION ...)?
Thanks
You should use OUTER APPLY or CROSS APPLY instead of a cursor:
SELECT *
FROM ( SELECT id
FROM org.employees
WHERE {some_condition}) A
OUTER APPLY org.work_schedule(A.id, #fromDate, #toDate) B

sp_executesql runs in milliseconds in SSMS but takes 3 seconds from ado.net [duplicate]

This question already has an answer here:
Stored Proc slower from application than Management Studio
(1 answer)
Closed 9 years ago.
This is my dynamic query used on search form which runs in milliseconds in SSMS roughly between 300 to 400 ms:
exec sp_executesql N'set arithabort off;
set transaction isolation level read uncommitted;
With cte as
(Select ROW_NUMBER() OVER
(Order By Case When d.OldInstrumentID IS NULL
THEN d.LastStatusChangedDateTime Else d.RecordingDateTime End
desc) peta_rn,
d.DocumentID
From Documents d
Inner Join Users u on d.UserID = u.UserID
Inner Join IGroupes ig on ig.IGroupID = d.IGroupID
Inner Join ITypes it on it.ITypeID = d.ITypeID
Where 1=1
And (CreatedByAccountID = #0 Or DocumentStatusID = #1 Or DocumentStatusID = #2 )
And (d.JurisdictionID = #3 Or DocumentStatusID = #4 Or DocumentStatusID = #5)
AND ( d.DocumentStatusID = 9 )
)
Select d.DocumentID, d.IsReEfiled, d.IGroupID, d.ITypeID, d.RecordingDateTime,
d.CreatedByAccountID, d.JurisdictionID,
Case When d.OldInstrumentID IS NULL THEN d.LastStatusChangedDateTime
Else d.RecordingDateTime End as LastStatusChangedDateTime,
dbo.FnCanChangeDocumentStatus(d.DocumentStatusID,d.DocumentID) as CanChangeStatus,
d.IDate, d.InstrumentID, d.DocumentStatusID,ig.Abbreviation as IGroupAbbreviation,
u.Username, j.JDAbbreviation, inf.DocumentName,
it.Abbreviation as ITypeAbbreviation, d.DocumentDate,
ds.Abbreviation as DocumentStatusAbbreviation,
Upper(dbo.GetFlatDocumentName(d.DocumentID)) as FlatDocumentName
From Documents d
Left Join IGroupes ig On d.IGroupID = ig.IGroupID
Left Join ITypes it On d.ITypeID = it.ITypeID
Left Join Users u On u.UserID = d.UserID
Left Join DocumentStatuses ds On d.DocumentStatusID = ds.DocumentStatusID
Left Join InstrumentFiles inf On d.DocumentID = inf.DocumentID
Left Join Jurisdictions j on j.JurisdictionID = d.JurisdictionID
Inner Join cte on cte.DocumentID = d.DocumentID
Where 1=1
And peta_rn>=#6 AND peta_rn<=#7
Order by peta_rn',
N'#0 int,#1 int,#2 int,#3 int,#4 int,#5 int,#6 bigint,#7 bigint',
#0=44,#1=5,#2=9,#3=1,#4=5,#5=9,#6=94200,#7=94250
This sql is formed in C# code and the where clauses are added dynamically based on the value the user has searched in search form. It takes roughly 3 seconds to move from one page to 2nd. I already have necessary indexes on most of the columns where I search.
Any idea why would my Ado.Net code be slow?
Update: Not sure if execution plans would help but here they are:
It is possible that SQL server has created inappropriate query plan for ADO.NET connections. We have seen similar issues with ADO, usual solution is to clear any query plans and run slow query again - this may create better plan.
To clear query plans most general solution is to update statistics for involved tables. Like next for you:
update statistics documents with fullscan
Do same for other tables involved and then run your slow query from ADO.NET (do not run SSMS before).
Note that such timing inconsistencies may hint of bad query or database design - at least for us that is usually so :)
If you run a query repeatedly in SSMS, the database may re-use a previously created execution plan, and the required data may already be cached in memory.
There are a couple of things I notice in your query:
the CTE joins Users, IGroupes and ITypes, but the joined records are not used in the SELECT
the CTE performs an ORDER BY on a calculated expression (notice the 85% cost in (unindexed) Sort)
probably replacing the CASE expression with a computed persisted column which can be indexed speeds up execution.
note that the ORDER BY is executed on data resulting from joining 4 tables
the WHERE condition of the CTE states AND d.DocumentStatusID = 9, but AND's other DocumentStatusIDs
paging is performed on the result of 8 JOINed tables.
most likely creating an intermediate CTE which filters the first CTE based on peta_rn improves performance
.net by default uses UTF strings, which equates to NVARCHAR as opposed to VARCHAR.
When you are doing a WHERE ID = #foo in dot net, you are likely to be implicitly doing
WHERE CONVERT(ID, NVARCHAR) = #foo
The result is that this where clause can't be indexed, and must be table scanned. The solution is to actually pass each parameter into the SqlCommand as a DbParameter with the DbType set to VARCHAR (in the case of string).
A similar situation could of course occur with Int types if the .net parameter is "wider" than the SQL column equivalent.
PS The easiest way to "prove" this issue is to run your query in SSMS with the following above
DECLARE #p0 INT = 123
DECLARE #p1 NVARCHAR = "foobar" //etc etc
and compare with
DECLARE #p0 INT = 123
DECLARE #p1 VARCHAR = "foobar" //etc etc

Fetch n bind data from StoredProcedure with Join, LINQ then Split. using IMultipleResults,Linq2SQL

I have an application in which I am using a stored procedure and LINQ.
My procedure looks like this:
myProc
select col1, Col2, Col3 from Tab1 inner join Tab2 on col1=ColA join tab3 on Col1=ColD
Select cola, Colb, Colc from Taba inner join Tabb on cola=ColX join tabc on Cola=ColY
Select colP, ColQ, ColR from TabP inner join TabQ on colP=ColW join tabR on ColP=ColZ
I am executing this stored procedure in LINQ.
When I execute it I am getting the results in IMultipleResults.
Below is my code in LINQ:
[Function(Name = "dbo.MyProc")]
[ResultType(typeof(TabA))]
[ResultType(typeof(TabB))] .....
public IMultipleResults GetMultipleResults([Parameter(DbType = "VarChar(50)")] string i_Cola)
{
IExecuteResult result = this.ExecuteMethodCall(this, ((MethodInfo)(MethodInfo.GetCurrentMethod())), i_Cola);
return (IMultipleResults)result.ReturnValue;
}
When I execute it as follows:
MyContext mCtx = new MyContext()
var allResult = mCtx.GetMultipleResults(txtName.Text.Trim());
IEnumerable<Taba> TabaRes = allResult.GetResult<Taba>();
IEnumerable<TabB> TabbRes = allResult.GetResult<Tabb>();
I am getting the column values of tables but I want the Inner Joined columns also.
I've referred to many blogs and forums, such as...
Ben Hall's Blog
Guy Burstein's Blog
Microsoft's Blog
... to try to find a solution to my problem, but couldn't find any.
well better to use views in respect to SP(If u are not able to write a linq to sql query),if u are using Linq to Sql its worthLess to use Sp becoze it decrease the performance.and one more thing if u are using Linq to sql there is no Need to use any type of join if ur data base is completely normalized.
If still ur prob is not solved just show me the table structure and what the out u need i will write a query for u...
I am not sure I understand your need, but can you change the proc to return * or all the columns in all the tables you need?
If the stored procedure isn't returning the data required then the only option for LINQ would be for it to go query the database again itself.
This would mostly negate the point of having a stored procedure and be slower than just adding the required parameters to your stored procedure.
Maybe this would be a stupid question, but in your store procedure:
select col1, Col2, Col3 from Tab1
inner join Tab2 on col1=ColA join
tab3 on Col1=ColD
col1,col2,col3 are all from tab1, but you never did a select from the other tables that you are trying to do inner join. Did you tried
select col1,col2,col3,colA,colB...
-or-
select * from...

Categories