I am running SQL Server and I have a stored procedure. I want do a select statement with a WHERE IN clause. I don't know how long the list will be so right now I have tried something as follows
SELECT * FROM table1 WHERE id IN (#idList)
in this solution #idList is a varChar(max). but this doesn't work. I heard about passing in table values, but I am confused about how to do that. Any help would be great
I would suggest using a function to split the incoming list (use the link that Martin put in his comment).
Store the results of the split function in a temporary table or table variable and join it in your query instead of the WHERE clause
select * into #ids from dbo.Split(',', #idList)
select t.*
from table1 t
join #ids i
on t.id = i.s
The most efficient way would be to pass in a table valued parameter (if you're on SQL Server 2008), or an XML parameter (if you're on SQL Server 2005/2000). If your list is small (and you're on SQL Server 2005/2000), passing in your list as a comma (or otherwise) delimited list and using a split function to divide the values out into rows in a temporary table is also an option.
Whichever option you use, you would then join this table (either the table parameter, the table resulting from the XML select, or the temporary table created by the values from the split) to your main query.
Here is a table valued function that takes a nvarchar and returns a table to join on:
Create function [ReturnValues]
(
#Values nvarchar(4000)
)
Returns #ValueTable table(Value nvarchar(2000))
As
Begin
Declare #Start int
Declare #End int
Set #Start = 1
Set #End = 1
While #Start <= len(#Values)
Begin
Set #End = charindex(',', #Values, #Start)
If #End = 0
Set #End = len(#Values) + 1
Insert into #ValueTable
Select rtrim(ltrim(substring(#Values, #Start, #End - #Start)))
Set #Start = #End + 1
End
Return
End
GO
Binding an #idList parameter as you suggested is not possible with SQL.
The best would be bulk inserting the ids into a separated table and than query that table by using an subselect, or joining the IDs.
e.g.
INSERT INTO idTable (id, context) values (#idValue, 1);
INSERT INTO idTable (id, context) values (#idValue, 1);
INSERT INTO idTable (id, context) values (#idValue, 1); // as often as you like
SELECT * FROM table1, idTable WHERE table1.id == idTable.id and idTable.context = 1
The context must be a unique value that identifies the Id Range. That is important for running the stored proc parallel. Without the context information, running the stored procecure in parallel would mix the values from different selections.
If the number of parameters are reasonably small (< 100) you can use several parameters
SELECT * FROM table1 WHERE IN id IN (#id1, #id2, #id3)
If it is longer, look for a split function.
Related
I am facing a peculiar issue with loading a list of tables from a specific database (well rather a group of databases) while attached to the master database. Currently my query loads all of the databases on the server, then loops through those databases sending information back to the client via RAISERROR. As this loop is executing I need a nested loop to load all of the tables for the current database for later transmission as a SELECT once the query has completed. The issue I'm running into is that this will be executed as a single query inside of C# code. Ideally I would like to load everything in SQL and return it to the client for processing. For example:
WHILE (#dbLoop < #dbCount) BEGIN
-- Do cool things and send details back to client.
SET #dbName = (SELECT _name FROM dbTemp WHERE _id = #dbLoop);
-- USE [#dbName]
-- Get a count of the tables from info schema on the newly specified database.
WHILE (#tableLoop < #tableCount) BEGIN
-- USE [#dbName]
-- Do super cool things and load tables from info schema.
SET #tableLoop += 1;
END
SET #dbLoop += 1;
END
-- Return the list of tables from all databases to the client for use with SQLDataAdapter.
SELECT * FROM tableTemp;
This topic is pretty straight forward; I just need a way to access tables in a specified database (preferably by name) without having to change the connection on the SqlConnection object, and without having to have a loop inside of my C# code to process the same query on each database on the C# side. It would be more efficient to load everything in SQL and send it back to the application. Any help that can be provided on this would be great!
Thanks,
Jamie
All the tables are in the meta data you can just do a query against that and join to your list of schemas you want to look at.
SELECT tab.name
FROM sys.tables AS tab
JOIN sys.schemas AS sch on tab.schema_id = sch.schema_id
JOIN dbTemp temp on sch.name = temp.[_name]
This returns a list of the table to return back as a result set.
The statement USE [#dbName] takes effect AFTER it is run (usually via the GO statement.
USE [#dbName]
GO
The above 2 lines would make you start using the new Database. You cannot use this in the middle of your SQL or SP.
One other option which you can use is to use the dot notation, i.e., dbname..tablename syntax to query your tables.
double dot notation post
Okay, after spending all day working on this, I have finally come up with a solution. I load all the databases into a table variable, then I begin looping through those databases and send back their details to the client. After the database details themselves have been sent to the client via RAISERROR I then utilize sp_executesql to execute a new sub-query with the current database specified to get the list of tables for processing at the end of the primary. The example below demonstrates the basic structure of this process for others experiencing this issue in the future.
Thank you all once again for your help!
-Jamie
DECLARE #LoopCounter INT = 1, #DatabaseCount INT = 0;
DECLARE #SQL NVARCHAR(MAX), #dbName NVARCHAR(MAX);
DECLARE #Databases TABLE ( _id INT, _name NVARCHAR(MAX) );
DECLARE #Tables TABLE ( _name NVARCHAR(MAX), _type NVARCHAR(15) );
INSERT INTO #Databases
SELECT ROW_NUMBER() OVER(ORDER BY name) AS id, name
FROM sys.databases
WHERE name NOT IN ( 'master', 'tempdb', 'msdb', 'model' );
SET #DatabaseCount = (SELECT COUNT(*) FROM #Databases);
WHILE (#LoopCounter <= #DatabaseCount) BEGIN
SET #dbName = (SELECT _name FROM #Databases WHERE _id = #LoopCounter);
SET #SQL NVARCHAR(MAX) = 'SELECT TABLE_NAME, TABLE_TYPE
FROM [' + #dbName + '].INFORMATION_SCHEMA.TABLES';
INSERT INTO #Tables EXEC sp_executesql #SQL;
SET #LoopCounter += 1;
END
Currently, I have a stored procedure that returns all the columns (Select *). I am using this stored procedure to retrieve data from multiple tables, each table with different number of columns and obviously different column names, so Select * is convenient. However, now I need to retrieve only specific columns from each table so I want to pass the parameters something like this:
SELECT #param1, #param2, #param3, etc.
FROM #tableName
WHERE columnName = #tableId
The problem here is the number of parameters to be passed for the columns isn't set, as the table can have any number of columns. Is there a way to use some kind of loop or dynamic assignment so that I can pass any number of parameters as column names?
I know that I can filter out only the columns I want to use, and just leave out the rest, but that doesn't work in my case. I need the stored procedure to NOT return some specific columns with sensitive data.
I am using SQL Server 2008, ASP.NET MVC 4, and C# in my application.
If you are able to modify your stored procedure, you can easily put the required columns definitions as a parameter and use an auto-created temporary table:
CREATE PROCEDURE sp_GetDiffDataExample
#columnsStatement NVARCHAR(MAX) -- required columns statement (e.g. "field1, field2")
AS
BEGIN
DECLARE #query NVARCHAR(MAX)
SET #query = N'SELECT ' + #columnsStatement + N' INTO ##TempTable FROM dbo.TestTable'
EXEC sp_executeSql #query
SELECT * FROM ##TempTable
DROP TABLE ##TempTable
END
In this case you don't need to create a temp table manually - it is created automatically.
Hope this helps.
You can just pass one parameter that is a comma-delimited string of the columns you want to select and build a dynamic sql string.
#sql = 'SELECT ' + #param + ' FROM MyTable...';
EXECUTE (#sql);
If you use a dynamic sql solution, you should take care to guard against sql injection attacks.
You might also consider continuing to get all columns from the stored procedure, and showing only the columns the user wants in the front end.
Although I really do not like the stored procedure approach for this problem (I agree with #Gusman that building the query in C# is a better approach) you can make a stored procedure work without opening yourself to SQL Injection attacks. The following example is one simple way to do this:
Let's say that the table in question has the columns named COL1, COL2, and COL3. The stored procedure would accept a varchar(max) parameter named #IncludeCols and have code like:
SELECT CASE WHEN #IncludeCols LIKE '%#COL1#%' THEN COL1 ELSE '' END AS COL1,
CASE WHEN #IncludeCols LIKE '%#COL2#%' THEN COL2 ELSE '' END AS COL2,
CASE WHEN #IncludeCols LIKE '%#COL3#%' THEN COL3 ELSE '' END AS COL3
FROM <Table name>
WHERE <Where clause>
Yes, a column of every name will be returned, but data will only come from columns whose names are in the parameter. For example, if you wanted COL1 and COL3, the parameter value would be #COL1#COL3#. The # is important and must be on each side of every column name or any LIKE clause could get a false positive match.
If this is to export data to Excel then the best approach is likely to have a stored procedure that returns all of your columns minus the ones that should never be exported (passwords, other protected data, timestamps perhaps) and then filter out any additional unwanted columns in your front end.
You should never be using SELECT * for several reasons. Define what your application needs and program to that.
Currently development team is done their application, and as a tester needs to insert 1000000 records into the 20 tables, for performance testing.
I gone through the tables and there is relationship between all the tables actually.
To insert that much dummy data into the tables, I need to understand the application completely in very short span so that I don't have the dummy data also by this time.
In SQL server is there any way to insert this much data insertion possibility.
please share the approaches.
Currently I am planning with the possibilities to create dummy data in excel, but here I am not sure the relationships between the tables.
Found in Google that SQL profiler will provide the order of execution, but waiting for the access to analyze this.
One more thing I found in Google is red-gate tool can be used.
Is there any script or any other solution to perform this tasks in simple way.
I am very sorry if this is a common question, I am working first time in SQL real time scenario. but I have the knowledge on SQL.
Why You don't generate those records in SQL Server. Here is a script to generate table with 1000000 rows:
DECLARE #values TABLE (DataValue int, RandValue INT)
;WITH mycte AS
(
SELECT 1 DataValue
UNION all
SELECT DataValue + 1
FROM mycte
WHERE DataValue + 1 <= 1000000
)
INSERT INTO #values(DataValue,RandValue)
SELECT
DataValue,
convert(int, convert (varbinary(4), NEWID(), 1)) AS RandValue
FROM mycte m
OPTION (MAXRECURSION 0)
SELECT
v.DataValue,
v.RandValue,
(SELECT TOP 1 [User_ID] FROM tblUsers ORDER BY NEWID())
FROM #values v
In table #values You will have some random int value(column RandValue) which can be used to generate values for other columns. Also You have example of getting random foreign key.
Below is a simple procedure I wrote to insert millions of dummy records into the table, I know its not the most efficient one but serves the purpose for a million records it takes around 5 minutes. You need to pass the no of records you need to generate while executing the procedure.
IF EXISTS (SELECT 1 FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[DUMMY_INSERT]') AND type in (N'P', N'PC'))
BEGIN
DROP PROCEDURE DUMMY_INSERT
END
GO
CREATE PROCEDURE DUMMY_INSERT (
#noOfRecords INT
)
AS
BEGIN
DECLARE #count int
SET #count = 1;
WHILE (#count < #noOfRecords)
BEGIN
INSERT INTO [dbo].[LogTable] ([UserId],[UserName],[Priority],[CmdName],[Message],[Success],[StartTime],[EndTime],[RemoteAddress],[TId])
VALUES(1,'user_'+CAST(#count AS VARCHAR(256)),1,'dummy command','dummy message.',0,convert(varchar(50),dateadd(D,Round(RAND() * 1000,1),getdate()),121),convert(varchar(50),dateadd(D,Round(RAND() * 1000,1),getdate()),121),'160.200.45.1',1);
SET #count = #count + 1;
END
END
you can use the cursor for repeat data:
for example this simple code:
Declare #SYMBOL nchar(255), --sample V
#SY_ID int --sample V
Declare R2 Cursor
For SELECT [ColumnsName]
FROM [TableName]
For Read Only;
Open R2
Fetch Next From R2 INTO #SYMBOL,#SY_ID
While (##FETCH_STATUS <>-1 )
Begin
Insert INTO [TableName] ([ColumnsName])
Values (#SYMBOL,#SY_ID)
Fetch Next From R2 INTO #SYMBOL,#SY_ID
End
Close R2
Deallocate R2
/*wait a ... moment*/
SELECT COUNT(*) --check result
FROM [TableName]
I have a stored procedure in Oracle as shown below:
CREATE PROCEDURE MY_TEST_PROC(
CUR OUT SYS_REFCURSOR,
PARAM_THAT_WILL_BE _USED_INSIDE_WHERE_IN
)
AS
BEGIN
OPEN CUR FOR
SELECT *
FROM MY_TABLE
WHERE COL1 IN (here I want to put values received from C#)
END;
On the ASP.NET application side I have a select element with several options. I want to use these list items in my WHERE clause. I know that I can have a VARCHAR2 input parameter in my stored proc, make a comma separated string from the list items, send it to the procedure. There are two concerns with going this way:
I make my website vulnerable to SQL injections
In my stored proc I have to use EXECUTE ('SELECT ...') pattern which I would like to avoid.
How can I send these list items to the stored procedure and use them inside the WHERE IN clause? I'm using ODP.NET and have heard of UDT but don't know how to use it.
One way could be to use a VARRAY for the PARAM_THAT_WILL_BE _USED_INSIDE_WHERE_IN
parameter and use it as described here
I'm not sure, though, how to call it from c#.
Another way is to use varchar2 with a csv as you stated in your question but without dynamic sql, like this:
CREATE PROCEDURE MY_TEST_PROC(
CUR OUT SYS_REFCURSOR,
PARAM_THAT_WILL_BE varchar2)
AS
BEGIN
OPEN CUR FOR
SELECT *
FROM MY_TABLE
WHERE COL1 IN (
select regexp_substr(PARAM_THAT_WILL_BE, '[^,]+',1,level) p
from dual t
connect by level <= regexp_count(PARAM_THAT_WILL_BE, ',') + 1
)
END;
You can add this comma separated input parameter as a varchar() and use following where statement:
where (','||PARAM_THAT_WILL_BE||',' like '%,'||COL1||',%')
for example if PARAM_THAT_WILL_BE='2,3,4,5' and col1=3 we get:
where (',2,3,4,5,' like '%,3,%')
and it's TRUE if COL1 value is in this list.
Here you don't use a dynamic query so you avoid concerns 1) and 2).
For this scenario i used like this
CREATE PROCEDURE MY_TEST_PROC(CUR OUT SYS_REFCURSOR,A in VARCHAR2
)
AS
BEGIN
OPEN CUR FOR
SELECT *
FROM MY_TABLE
WHERE COL1 IN (SELECT REGEXP_SUBSTR(**A**,'[^,]+', 1, LEVEL)
FROM DUAL
CONNECT BY REGEXP_SUBSTR(**A**, '[^,]+', 1, LEVEL) IS NOT NULL)
END;
The A value should contain open and closed qutoes(').
EX: '512,456,4564'
if it one value '512' like this
How do I pass paramter values for a command where SQL Text contains IN.
i.e. My command SQL text is something like SELECT * FROM USERS WHERE USERID IN (1,2,3).
There are plenty of examples like MSDN has but coudn't find one to pass values for IN. Tried taking a variable and set values as a single string but SQL wont work that way.
You should have a UDF which converts a (for example) csv string into a table of values.
Then, your query can be some thing like:
select *
from yourTable
where yourField IN ( select yourColumn from dbo.yourUDF(yourCSV) )
The short answer is there's no good way to do this.
If you're indeed only working with a list of numeric values, you probably don't need to use parameters as this feature is mainly designed to prevent SQL injection attacks. However, if you have an array of ints, and use that to build your query, you're safe.
The only other way I can think of would be to add one parameter per item. For example:
SELECT * FROM USERS WHERE USERID IN (#p1, #p2, #p3)
Then, loop through your array in C# and create a parameter for each item.
Hope this helps!
This has been asked many, many times. Two ways spring to mind:
If you have SQL2008, you can pass a "table value parameter" (tvp), where your table parameter has the values for your "in" clause, then have a "where" clause that checks values are "in (select whatever from tvp)".
If you do not have SQL2008, you can use a "common table expression" (cte), such as...
declare #t table (xyz varchar(100))
insert into #t values ('hello')
insert into #t values ('there')
insert into #t values ('world')
insert into #t values ('10')
insert into #t values ('20')
insert into #t values ('30')
declare #a as varchar(120)
set #a = 'hello,10,30'
;with cte as
(
select cast(null as varchar(max)) as 'v', 1 as s, charindex(',' , #a, 0) as e
union all
select cast(substring(#a, s, e-s) as varchar(max)), e+1, charindex(',' , #a + ',', e+1)
from cte where e!=0
)
select *
from #t
where (xyz in (select v from cte where v is not null))
In this example (above), the string #a is the comma separated list of values you want for your "in" clause, and the (recursive) cte just strips out the values, one by one. Of course, this is just a quick example and would probably need checks for empty strings, successive commas, and the like.
Of course, the usual caveats apply with regard to using SQL to (effectively) do string manipulation.
Regards,
Ross