We Keep Coding

Stored procedure with parameters to return specific multiple columns, not all (*)

Currently, I have a stored procedure that returns all the columns (Select *). I am using this stored procedure to retrieve data from multiple tables, each table with different number of columns and obviously different column names, so Select * is convenient. However, now I need to retrieve only specific columns from each table so I want to pass the parameters something like this:
SELECT #param1, #param2, #param3, etc.
FROM #tableName
WHERE columnName = #tableId
The problem here is the number of parameters to be passed for the columns isn't set, as the table can have any number of columns. Is there a way to use some kind of loop or dynamic assignment so that I can pass any number of parameters as column names?
I know that I can filter out only the columns I want to use, and just leave out the rest, but that doesn't work in my case. I need the stored procedure to NOT return some specific columns with sensitive data.
I am using SQL Server 2008, ASP.NET MVC 4, and C# in my application.

If you are able to modify your stored procedure, you can easily put the required columns definitions as a parameter and use an auto-created temporary table:
CREATE PROCEDURE sp_GetDiffDataExample
#columnsStatement NVARCHAR(MAX) -- required columns statement (e.g. "field1, field2")
AS
BEGIN
DECLARE #query NVARCHAR(MAX)
SET #query = N'SELECT ' + #columnsStatement + N' INTO ##TempTable FROM dbo.TestTable'
EXEC sp_executeSql #query
SELECT * FROM ##TempTable
DROP TABLE ##TempTable
END
In this case you don't need to create a temp table manually - it is created automatically.
Hope this helps.

You can just pass one parameter that is a comma-delimited string of the columns you want to select and build a dynamic sql string.
#sql = 'SELECT ' + #param + ' FROM MyTable...';
EXECUTE (#sql);
If you use a dynamic sql solution, you should take care to guard against sql injection attacks.
You might also consider continuing to get all columns from the stored procedure, and showing only the columns the user wants in the front end.

Although I really do not like the stored procedure approach for this problem (I agree with #Gusman that building the query in C# is a better approach) you can make a stored procedure work without opening yourself to SQL Injection attacks. The following example is one simple way to do this:
Let's say that the table in question has the columns named COL1, COL2, and COL3. The stored procedure would accept a varchar(max) parameter named #IncludeCols and have code like:
SELECT CASE WHEN #IncludeCols LIKE '%#COL1#%' THEN COL1 ELSE '' END AS COL1,
CASE WHEN #IncludeCols LIKE '%#COL2#%' THEN COL2 ELSE '' END AS COL2,
CASE WHEN #IncludeCols LIKE '%#COL3#%' THEN COL3 ELSE '' END AS COL3
FROM <Table name>
WHERE <Where clause>
Yes, a column of every name will be returned, but data will only come from columns whose names are in the parameter. For example, if you wanted COL1 and COL3, the parameter value would be #COL1#COL3#. The # is important and must be on each side of every column name or any LIKE clause could get a false positive match.

If this is to export data to Excel then the best approach is likely to have a stored procedure that returns all of your columns minus the ones that should never be exported (passwords, other protected data, timestamps perhaps) and then filter out any additional unwanted columns in your front end.
You should never be using SELECT * for several reasons. Define what your application needs and program to that.

Adding column to temporary table with select statments

I have a stored procedure I'm trying to create to fill a temporary table. But I need to make several passes at adding data based upon some conditions and capture those conditions with an additional field added to the temp table.
I start out like this:
select top 0
into #mytable
from UserTable
This I have found simply copies the basic structure with the same columns and types. Then I need to add a field:
alter table #mytable ADD reasontype varchar
The I make several passes at examining the table, here is one of them:
insert into #mytable
select distinct a.*, 'Annual'
from UserTable a
where (a.EnrollmentDate < DATEADD(year, -1, getdate())
This is to select those that require an annual review. The procedure compiles without an error but when I try to fill a datatable I get the error that string or binary data would be truncated. What am I doing wrong?

alter table #mytable ADD reasontype varchar(max)
If that works, either use "max" or a value that won't truncate your values...or use a LEFT statement where the length matches the longest value of reasontype
example for left:
alter table #mytable ADD reasontype varchar(3)
insert into #mytable
select distinct a.*, LEFT('Annual',3)
from UserTable a
where (a.EnrollmentDate < DATEADD(year, -1, getdate())
but you probably just want this:
alter table #mytable ADD reasontype varchar(6) /* where 6 is the length of the string "Annual" */

Use a length on your varchar, 'Annual' is being truncated to 'A'. If you're planning on indexing the reasontype field, don't use varchar(max), it exceeds the maximum width of an indexable field

Building a SQL stored procedure query with list inputs

How do I pass a list of names to a SQL parameter so I can do WHERE IN (#myList)?
I'm making an "advanced search" for our webpage where the user can list, separated by commas, things they would like to find for several inputs. For example there could be 5 input's (First Name, account number, Last name, ...) and for each of those inputs they can list several and separate them by commas.
My approach so far is to take in this string for each input, and format it so I can use the "IN" clause on that string. Is this possible?
IE the user for "First Name" enters "Bob, Joe,Fred", I would transform that input into
"'Bob', 'Joe', 'Fred'"
and then send that as a parameter to a stored procedure.
Declare #firstNames
Select * from Users
where User.FirstName in firstNames
OR should I put all these names into a DataTable in C# and pass that to SQL? If I should go this route, some examples would help a lot!
Thanks!

Whenever my query gets a little complicated like this, I prefer to choose either of these ways:
Create a query dynamically in C#, based on string concatenation for better readability rather than LINQ expression trees, and send it to SQL Server, or
Create a SP, or a Table-Valued Function in SQL Server, and then call it from C#, passing arguments to it.
In case of creating the query in C#, it's not extensible, and every time you want to change the logic, you should recompile your application.
In case of SP, because it's going to be a dynamic query and where clause should be created based on input parameters, I use exec (#query)

Assuming you are using SQL Server, you can create a tabular function like to split comma seperated values into table
CREATE FUNCTION dbo.Split(#String nvarchar(4000), #Delimiter char(1))
RETURNS #Results TABLE (Items nvarchar(4000))
AS
BEGIN
DECLARE #INDEX INT
DECLARE #SLICE nvarchar(4000)
-- HAVE TO SET TO 1 SO IT DOESNT EQUAL Z
-- ERO FIRST TIME IN LOOP
SELECT #INDEX = 1
WHILE #INDEX !=0
BEGIN
-- GET THE INDEX OF THE FIRST OCCURENCE OF THE SPLIT CHARACTER
SELECT #INDEX = CHARINDEX(#Delimiter,#STRING)
-- NOW PUSH EVERYTHING TO THE LEFT OF IT INTO THE SLICE VARIABLE
IF #INDEX !=0
SELECT #SLICE = LEFT(#STRING,#INDEX - 1)
ELSE
SELECT #SLICE = #STRING
-- PUT THE ITEM INTO THE RESULTS SET
INSERT INTO #Results(Items) VALUES(#SLICE)
-- CHOP THE ITEM REMOVED OFF THE MAIN STRING
SELECT #STRING = RIGHT(#STRING,LEN(#STRING) - #INDEX)
-- BREAK OUT IF WE ARE DONE
IF LEN(#STRING) = 0 BREAK
END
RETURN
END
Then you can call the function like this in your SP
Select * from Users
where User.FirstName in (
SELECT items FROM [dbo].[Split] (#firstNames, ','))

In C# you can add a list of parameters to your SQLCommand.CommandText. Assuming customerName is a string of "'Bob', 'Joe', 'Fred'" you do something like this:
Dim command As New SqlCommand(commandText, connection)
' Add FirstName parameter for WHERE clause.
command.Parameters.Add("#FirstName", SqlDbType.nvarchar)
command.Parameters("#FirstName").Value = FirstName
In Query Analyzer you can't have a list in a #parameter, which is a nuisance, but you can pass one in from another source; for example your C# calling code. In your WHERE clause you do WHERE IN (#Name). In my testing I create a temp table and do WHERE in (SELECT FirstName FROM #MyCustomerTempTable), then when hooking it up replace the sub-query with the singleton parameter.
Source: http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlcommand.parameters(v=vs.110).aspx
Another approach (which I used most frequently) to adding parameters to the command is:
' Add the input parameter and set its properties.
Dim parameter As New SqlParameter()
parameter.ParameterName = "#FirstName"
parameter.SqlDbType = SqlDbType.NVarChar
parameter.Value = firstName
' Add the parameter to the Parameters collection.
command.Parameters.Add(parameter)
Source (scroll down to the example):
http://msdn.microsoft.com/en-us/library/yy6y35y8(v=vs.110).aspx
You can also dynamically build your query in C# or do a string.replace and replace #parameter with 'my list of names', but neither of these methods are preferred to adding parameter objects to the SQL Command object. I would suggest getting a solid understanding of the SQL Command object so you can build that instead of manipulating strings.

Sending an array of values to Oracle procedure to use in WHERE IN clause

I have a stored procedure in Oracle as shown below:
CREATE PROCEDURE MY_TEST_PROC(
CUR OUT SYS_REFCURSOR,
PARAM_THAT_WILL_BE _USED_INSIDE_WHERE_IN
)
AS
BEGIN
OPEN CUR FOR
SELECT *
FROM MY_TABLE
WHERE COL1 IN (here I want to put values received from C#)
END;
On the ASP.NET application side I have a select element with several options. I want to use these list items in my WHERE clause. I know that I can have a VARCHAR2 input parameter in my stored proc, make a comma separated string from the list items, send it to the procedure. There are two concerns with going this way:
I make my website vulnerable to SQL injections
In my stored proc I have to use EXECUTE ('SELECT ...') pattern which I would like to avoid.
How can I send these list items to the stored procedure and use them inside the WHERE IN clause? I'm using ODP.NET and have heard of UDT but don't know how to use it.

One way could be to use a VARRAY for the PARAM_THAT_WILL_BE _USED_INSIDE_WHERE_IN
parameter and use it as described here
I'm not sure, though, how to call it from c#.
Another way is to use varchar2 with a csv as you stated in your question but without dynamic sql, like this:
CREATE PROCEDURE MY_TEST_PROC(
CUR OUT SYS_REFCURSOR,
PARAM_THAT_WILL_BE varchar2)
AS
BEGIN
OPEN CUR FOR
SELECT *
FROM MY_TABLE
WHERE COL1 IN (
select regexp_substr(PARAM_THAT_WILL_BE, '[^,]+',1,level) p
from dual t
connect by level <= regexp_count(PARAM_THAT_WILL_BE, ',') + 1
)
END;

You can add this comma separated input parameter as a varchar() and use following where statement:
where (','||PARAM_THAT_WILL_BE||',' like '%,'||COL1||',%')
for example if PARAM_THAT_WILL_BE='2,3,4,5' and col1=3 we get:
where (',2,3,4,5,' like '%,3,%')
and it's TRUE if COL1 value is in this list.
Here you don't use a dynamic query so you avoid concerns 1) and 2).

For this scenario i used like this
CREATE PROCEDURE MY_TEST_PROC(CUR OUT SYS_REFCURSOR,A in VARCHAR2
)
AS
BEGIN
OPEN CUR FOR
SELECT *
FROM MY_TABLE
WHERE COL1 IN (SELECT REGEXP_SUBSTR(**A**,'[^,]+', 1, LEVEL)
FROM DUAL
CONNECT BY REGEXP_SUBSTR(**A**, '[^,]+', 1, LEVEL) IS NOT NULL)
END;
The A value should contain open and closed qutoes(').
EX: '512,456,4564'
if it one value '512' like this

Creating a SQL table from a comma concatenated list

I am running SQL Server and I have a stored procedure. I want do a select statement with a WHERE IN clause. I don't know how long the list will be so right now I have tried something as follows
SELECT * FROM table1 WHERE id IN (#idList)
in this solution #idList is a varChar(max). but this doesn't work. I heard about passing in table values, but I am confused about how to do that. Any help would be great

I would suggest using a function to split the incoming list (use the link that Martin put in his comment).
Store the results of the split function in a temporary table or table variable and join it in your query instead of the WHERE clause
select * into #ids from dbo.Split(',', #idList)
select t.*
from table1 t
join #ids i
on t.id = i.s

The most efficient way would be to pass in a table valued parameter (if you're on SQL Server 2008), or an XML parameter (if you're on SQL Server 2005/2000). If your list is small (and you're on SQL Server 2005/2000), passing in your list as a comma (or otherwise) delimited list and using a split function to divide the values out into rows in a temporary table is also an option.
Whichever option you use, you would then join this table (either the table parameter, the table resulting from the XML select, or the temporary table created by the values from the split) to your main query.

Here is a table valued function that takes a nvarchar and returns a table to join on:
Create function [ReturnValues]
(
#Values nvarchar(4000)
)
Returns #ValueTable table(Value nvarchar(2000))
As
Begin
Declare #Start int
Declare #End int
Set #Start = 1
Set #End = 1
While #Start <= len(#Values)
Begin
Set #End = charindex(',', #Values, #Start)
If #End = 0
Set #End = len(#Values) + 1
Insert into #ValueTable
Select rtrim(ltrim(substring(#Values, #Start, #End - #Start)))
Set #Start = #End + 1
End
Return
End
GO

Binding an #idList parameter as you suggested is not possible with SQL.
The best would be bulk inserting the ids into a separated table and than query that table by using an subselect, or joining the IDs.
e.g.
INSERT INTO idTable (id, context) values (#idValue, 1);
INSERT INTO idTable (id, context) values (#idValue, 1);
INSERT INTO idTable (id, context) values (#idValue, 1); // as often as you like
SELECT * FROM table1, idTable WHERE table1.id == idTable.id and idTable.context = 1
The context must be a unique value that identifies the Id Range. That is important for running the stored proc parallel. Without the context information, running the stored procecure in parallel would mix the values from different selections.

If the number of parameters are reasonably small (< 100) you can use several parameters
SELECT * FROM table1 WHERE IN id IN (#id1, #id2, #id3)
If it is longer, look for a split function.