Related
I have a List containing ids that I want to insert into a temp table using Dapper in order to avoid the SQL limit on parameters in the 'IN' clause.
So currently my code looks like this:
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
return db.Query<int>(
#"SELECT a.animalID
FROM
dbo.animalTypes [at]
INNER JOIN animals [a] on a.animalTypeId = at.animalTypeId
INNER JOIN edibleAnimals e on e.animalID = a.animalID
WHERE
at.animalId in #animalIds", new { animalIds }).ToList();
}
}
The problem I need to solve is that when there are more than 2100 ids in the animalIds list then I get a SQL error "The incoming request has too many parameters. The server supports a maximum of 2100 parameters".
So now I would like to create a temp table populated with the animalIds passed into the method. Then I can join the animals table on the temp table and avoid having a huge "IN" clause.
I have tried various combinations of syntax but not got anywhere.
This is where I am now:
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
db.Execute(#"SELECT INTO #tempAnmialIds #animalIds");
return db.Query<int>(
#"SELECT a.animalID
FROM
dbo.animalTypes [at]
INNER JOIN animals [a] on a.animalTypeId = at.animalTypeId
INNER JOIN edibleAnimals e on e.animalID = a.animalID
INNER JOIN #tempAnmialIds tmp on tmp.animalID = a.animalID).ToList();
}
}
I can't get the SELECT INTO working with the list of IDs. Am I going about this the wrong way maybe there is a better way to avoid the "IN" clause limit.
I do have a backup solution in that I can split the incoming list of animalIDs into blocks of 1000 but I've read that the large "IN" clause sufferes a performance hit and joining a temp table will be more efficient and it also means I don;t need extra 'splitting' code to batch up the ids in to blocks of 1000.
Ok, here's the version you want. I'm adding this as a separate answer, as my first answer using SP/TVP utilizes a different concept.
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
// This Open() call is vital! If you don't open the connection, Dapper will
// open/close it automagically, which means that you'll loose the created
// temp table directly after the statement completes.
db.Open();
// This temp table is created having a primary key. So make sure you don't pass
// any duplicate IDs
db.Execute("CREATE TABLE #tempAnimalIds(animalId int not null primary key);");
while (animalIds.Any())
{
// Build the statements to insert the Ids. For this, we need to split animalIDs
// into chunks of 1000, as this flavour of INSERT INTO is limited to 1000 values
// at a time.
var ids2Insert = animalIds.Take(1000);
animalIds = animalIds.Skip(1000).ToList();
StringBuilder stmt = new StringBuilder("INSERT INTO #tempAnimalIds VALUES (");
stmt.Append(string.Join("),(", ids2Insert));
stmt.Append(");");
db.Execute(stmt.ToString());
}
return db.Query<int>(#"SELECT animalID FROM #tempAnimalIds").ToList();
}
}
To test:
var ids = LoadAnimalTypeIdsFromAnimalIds(Enumerable.Range(1, 2500).ToList());
You just need to amend your select statement to what it originally was. As I don't have all your tables in my environment, I just selected from the created temp table to prove it works the way it should.
Pitfalls, see comments:
Open the connection at the beginning, otherwise the temp table will
be gone after dapper automatically closes the connection right after
creating the table.
This particular flavour of INSERT INTO is limited
to 1000 values at a time, so the passed IDs need to be split into
chunks accordingly.
Don't pass duplicate keys, as the primary key on the temp table will not allow that.
Edit
It seems Dapper supports a set-based operation which will make this work too:
public IList<int> LoadAnimalTypeIdsFromAnimalIdsV2(IList<int> animalIds)
{
// This creates an IEnumerable of an anonymous type containing an Id property. This seems
// to be necessary to be able to grab the Id by it's name via Dapper.
var namedIDs = animalIds.Select(i => new {Id = i});
using (var db = new SqlConnection(this.connectionString))
{
// This is vital! If you don't open the connection, Dapper will open/close it
// automagically, which means that you'll loose the created temp table directly
// after the statement completes.
db.Open();
// This temp table is created having a primary key. So make sure you don't pass
// any duplicate IDs
db.Execute("CREATE TABLE #tempAnimalIds(animalId int not null primary key);");
// Using one of Dapper's convenient features, the INSERT becomes:
db.Execute("INSERT INTO #tempAnimalIds VALUES(#Id);", namedIDs);
return db.Query<int>(#"SELECT animalID FROM #tempAnimalIds").ToList();
}
}
I don't know how well this will perform compared to the previous version (ie. 2500 single inserts instead of three inserts with 1000, 1000, 500 values each). But the doc suggests that it performs better if used together with async, MARS and Pipelining.
In your example, what I can't see is how your list of animalIds is actually passed to the query to be inserted into the #tempAnimalIDs table.
There is a way to do it without using a temp table, utilizing a stored procedure with a table value parameter.
SQL:
CREATE TYPE [dbo].[udtKeys] AS TABLE([i] [int] NOT NULL)
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[myProc](#data as dbo.udtKeys readonly)AS
BEGIN
select i from #data;
END
GO
This will create a user defined table type called udtKeys which contains just one int column named i, and a stored procedure that expects a parameter of that type. The proc does nothing else but to select the IDs you passed, but you can of course join other tables to it. For a hint regarding the syntax, see here.
C#:
var dataTable = new DataTable();
dataTable.Columns.Add("i", typeof(int));
foreach (var animalId in animalIds)
dataTable.Rows.Add(animalId);
using(SqlConnection conn = new SqlConnection("connectionString goes here"))
{
var r=conn.Query("myProc", new {data=dataTable},commandType: CommandType.StoredProcedure);
// r contains your results
}
The parameter within the procedure gets populated by passing a DataTable, and that DataTable's structure must match the one of the table type you created.
If you really need to pass more that 2100 values, you may want to consider indexing your table type to increase performance. You can actually give it a primary key if you don't pass any duplicate keys, like this:
CREATE TYPE [dbo].[udtKeys] AS TABLE(
[i] [int] NOT NULL,
PRIMARY KEY CLUSTERED
(
[i] ASC
)WITH (IGNORE_DUP_KEY = OFF)
)
GO
You may also need to assign execute permissions for the type to the database user you execute this with, like so:
GRANT EXEC ON TYPE::[dbo].[udtKeys] TO [User]
GO
See also here and here.
For me, the best way I was able to come up with was turning the list into a comma separated list in C# then using string_split in SQL to insert the data into a temp table. There are probably upper limits to this, but in my case I was only dealing with 6,000 records and it worked really fast.
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
return db.Query<int>(
#" --Created a temp table to join to later. An index on this would probably be good too.
CREATE TABLE #tempAnimals (Id INT)
INSERT INTO #tempAnimals (ID)
SELECT value FROM string_split(#animalIdStrings)
SELECT at.animalTypeID
FROM dbo.animalTypes [at]
JOIN animals [a] ON a.animalTypeId = at.animalTypeId
JOIN #tempAnimals temp ON temp.ID = a.animalID -- <-- added this
JOIN edibleAnimals e ON e.animalID = a.animalID",
new { animalIdStrings = string.Join(",", animalIds) }).ToList();
}
}
It might be worth noting that string_split is only available in SQL Server 2016 or higher or if using Azure SQL then compatibility mode 130 or higher. https://learn.microsoft.com/en-us/sql/t-sql/functions/string-split-transact-sql?view=sql-server-ver15
I am currently working on a solution in C# to copy database tables across from Oracle to PostgreSQL. Everything is working great apart from one thing. When I get to copying one of my table which contains sql statements in one of its fields it falls over due to having two single quotes back to back. The SQL statements MUST remain in the table as it is being used to make another program database agnostic.
Is there a way to write the following SQL but without having the two single quotes back to back as seen near the 'TRUE' value near the end of the line. I have also included my code below to show how the statement is built up in C#. The column is a varchar2 in Oracle and a TEXT column in PostgreSQL.
EDIT: The sql example shown is the actual INSERT statement generated by my C# code which will then be run on the postgresql database to add a record to a table. It will be used to insert a text field, among others, that contains an sql statement in the form of a string.
INSERT INTO SQL_FACTORY_TEST (SQL_FACTORY_TEST_ID,SQL_FACTORY_DIALECT,SQL_FACTORY_QUERY_NAME,SQL_FACTORY_SQL_COMMAND,USER_NAME)
VALUES (21, 'ORACLE', 'GET_CLUSTERS', 'SELECT CLUSTER_ID, NUM_POINTS, FEATURE_PK, A.CELL_CENTROID.SDO_POINT.X, A.CELL_CENTROID.SDO_POINT.Y, A.CLUSTER_CENTROID.SDO_POINT.X, A.CLUSTER_CENTROID.SDO_POINT.Y, TO_CHAR (A.CLUSTER_EXTENT.GET_WKT ()), TO_CHAR (A.CELL_GEOM.GET_WKT ()), A.CLUSTER_EXTENT.SDO_SRID FROM (SELECT CLUSTER_ID, NUM_POINTS, FEATURE_PK, SDO_CS.transform (CLUSTER_CENTROID, 4326) cluster_centroid, CLUSTER_EXTENT, SDO_CS.transform (CELL_CENTROID, 4326) cell_centroid, CELL_GEOM FROM :0) a where sdo_filter( A.CELL_GEOM, SDO_CS.transform(mdsys.sdo_geometry(2003, :1, NULL, mdsys.sdo_elem_info_array(1,1003,3),mdsys.sdo_ordinate_array(:2, :3, :4, :5)),81989)) = 'TRUE'', 'PUBLIC')
Code sample:
oleDataBaseConnection.OleExecutePureSqlQuery("SELECT * FROM " + tableName);
if (oleDataBaseConnection.HasRows())
{
while (oleDataBaseConnection.NextRecord())
{
Dictionary<string, string> postgreSQLQueries = TypeConversion.GetQueryDictionary("POSTGRESQL");
string postgreSQLInsertQuery;
postgreSQLQueries.TryGetValue("INSERT", out postgreSQLInsertQuery);
postgreSQLInsertQuery = postgreSQLInsertQuery.Replace("{0}", tableName);
StringBuilder postgresQuery = new StringBuilder();
postgresQuery.Append(postgreSQLInsertQuery);
postgresQuery.Append("(");
int columnCounter = 0;
//add a column parameter to query for each of our columns
foreach (KeyValuePair<string, string> t in columnData)
{
postgresQuery.Append(t.Key + ",");
columnCounter++;
}
postgresQuery = postgresQuery.Remove(postgresQuery.Length - 1, 1);
postgresQuery.Append(") ");
postgresQuery.Append("VALUES (");
//Loop through values and
for (int i = 0; i < columnCounter; i++)
{
string[] foo = new string[columnData.Count];
columnData.Values.CopyTo(foo, 0);
if (foo[i].ToUpper() == "TEXT")
{
postgresQuery.Append("'" + oleDataBaseConnection.GetFieldById(i) + "', ");
}
else
{
postgresQuery.Append(oleDataBaseConnection.GetFieldById(i) + ", ");
}
}
postgresQuery = postgresQuery.Remove(postgresQuery.Length - 2, 2);
postgresQuery.Append(") ");
postgresSQLDBConnection.PostgreSQLExecutePureSqlNonQuery(postgresQuery.ToString());
}
}
The best solution is to use a PreparedStatement where you don't pass literals directly. I don't know C#, so I can't give you an example for that.
However if you have to keep the statement like that, you can use a "dollar quoted" string literal in PostgreSQL.
INSERT INTO SQL_FACTORY_TEST (SQL_FACTORY_TEST_ID,SQL_FACTORY_DIALECT,SQL_FACTORY_QUERY_NAME,SQL_FACTORY_SQL_COMMAND,USER_NAME)
VALUES (21, 'ORACLE', 'GET_CLUSTERS', $$ ...... 'TRUE'$$, 'PUBLIC')
The $$ replaces the single quote(s) around the literal. If there is a chance that your value contains $$ you can also add some unique identifier to it, e.g.
$42$String with embedded single quotes ''' and two $$ dollar characters$42$
I think the best approach is to use a parameterised insert query. so you are actually building a query like:
INSERT INTO [Table] VALUES (#Column1, #Column2, #Column3)
then passing the variable into the query. Something along the lines of:
List<OleDbParameter> parameters = new List<OleDbParameter>();
for (int i = 0; i < columnCounter; i++)
{
postgresQuery.Append(string.Format("#Column{0}, ", i));
parameters.Add(new OleDbParameter(string.Format("#Column{0}, ", i), oleDataBaseConnection.GetFieldById(i));
}
Then pass paremeters.ToArray() to you OleDbCommand when executing. This means c# will do all the escaping, and work out the data type for you. It may need a bit of tweaking to suit your needs but the general gist of it is there.
You should always properly escape and unescape values stored in the database.
The PostgreSQL client library provides the PQescapeLiteral function for this purpose.
(Disclaimer: I changed/obfuscated some of the variable/table/column names here for security reasons. Please forgive me if something looks a little off.)
I am building a front-end to an Oracle 10g database, and I'm trying to get paged data. Aside from paging, the following SubSonic 2.2 code gives me what I want, in the order I want it:
var q = new Select()
.From(AppDb.MyTable.Schema)
.Where(AppDb.MyTable.DisabledDateColumn).IsNull()
.OrderDesc(AppDb.MyTable.CreatedDateColumn.ColumnName)
System.Console.Out.Writeline(q.BuildStatement());
This yields the following SQL:
SELECT
MYSCHEMA.MYTABLE.ID,
MYSCHEMA.MYTABLE.DISABLED_DATE
FROM
MYSCHEMA.MYTABLE
WHERE
MYSCHEMA.MYTABLE.DISABLED_DATE IS NULL
ORDER BY
CREATED_DATE DESC
Then I try to introduce paging:
var q = new Select()
.From(AppDb.MyTable.Schema)
.Where(AppDb.MyTable.DisabledDateColumn).IsNull()
.OrderDesc(AppDb.MyTable.CreatedDateColumn.ColumnName)
.Paged(0, 10);
And it wipes out my WHERE and ORDER BY clauses:
SELECT * FROM (
SELECT
MYSCHEMA.MYTABLE.ID,
MYSCHEMA.MYTABLE.DISABLED_DATE,
ROWNUM as row_number
FROM
MYSCHEMA.MYTABLE
)
WHERE
row_number BETWEEN 1 AND 10
I'm brand new to SubSonic, so I don't know if that's a bug, or I'm just not doing it the preferred way, or if Oracle demands that this be done in a different, more Oracle-centric way. (Oracle seems to demand that of everything.)
What I want, in case it isn't obvious, is each succeeding page to include the 10 next most-recently-created, non-disabled records. How can I do this in SubSonic 2.2?
It looks like your query is not generating the same SQL as the current Oracle Data Provider for SubSonic should. Here is the relevant code (starting at line 852):
if(qry.PageIndex < 0)
query = String.Format("{0} {1} FROM {2}.{3} {4} {5}",select ,columns, table.SchemaName, table.Name, where, order);
else
{
int start = qry.PageIndex * qry.PageSize;
int end = (qry.PageIndex + 1) * qry.PageSize;
const string cteFormat =
"WITH pagedtable AS (SELECT {0}, ROW_NUMBER () OVER ({1}) AS rowindex FROM {2}.{3} {4}) SELECT {5}, rowindex FROM pagedtable WHERE rowindex >= {6} AND rowindex < {7} ORDER BY rowindex";
query = string.Format(cteFormat, columns, order,table.SchemaName, table.Name, where, columns.Replace(table.Name + ".", String.Empty), start, end);
}
return query;
Perhaps an update to current source would do the trick. If there is something wrong with the actual provider, you can compare it to the SqlDataProvider for hints as to what might be the problem.
I'm in an environment using NET 2.0 and Oracle 9i R2 and I ran into the same exact problem. I'm using SubSonic 2.2.
I downloaded the source from GitHub and this is what I found:
The code quoted by #ranomore (OracleDataProvider.GetSelectSql()) is only called when using the SubSonic.Query object. Since the OP and myself use the Select object which is derived from the newer and more powerful SubSonic.SqlQuery object, OracleDataProvider.GetSelectSql() never gets called. Instead, OracleGenerator.BuildPagedSelectStatement() gets called and generates the SQL you see posted by the OP. This code is buggy as it never adds the WHERE and ORDER BY clauses to the pagination query it ultimately generates.
I replaced the contents of BuildPagedSelectStatement() with something based on ANSISqlGenerator.BuildSelectStatement():
public override string BuildPagedSelectStatement()
{
int startnum = query.PageSize * query.CurrentPage + 1;
int endnum = query.PageSize * query.CurrentPage + query.PageSize;
string orderBy = String.Empty;
if (this.query.OrderBys.Count > 0)
orderBy = GenerateOrderBy();
//The ROW_NUMBER() function in Oracle requires an ORDER BY clause.
//In case one is not specified, we need to halt and inform the caller.
if(orderBy.Equals(String.Empty))
throw new ArgumentException("There is no column specified for the ORDER BY clause", "OrderBys");
System.Text.StringBuilder sql = new System.Text.StringBuilder();
//Build the command string
sql.Append("WITH pagedtable AS (");
sql.Append(GenerateCommandLine());
//Since this class is for Oracle-specific SQL, we can add a hint
//which should help pagination queries return rows more quickly.
//AFAIK, this is only valid for Oracle 9i or newer.
sql.Replace("SELECT", "SELECT /*+ first_rows('" + query.PageSize + "') */");
sql.Append(", ROW_NUMBER () OVER (");
sql.Append(orderBy);
sql.Append(") AS rowindex ");
sql.Append(Environment.NewLine);
sql.Append(GenerateFromList());
sql.Append(GenerateJoins());
sql.Append(GenerateWhere());
if (query.Aggregates.Count > 0)
{
sql.Append(GenerateGroupBy());
sql.Append(Environment.NewLine);
sql.Append(GenerateHaving());
}
sql.Append(") SELECT * FROM pagedtable WHERE rowindex >= ");
sql.Append(startnum);
sql.Append(" AND rowindex < ");
sql.Append(endnum);
sql.Append(" ORDER BY rowindex");
return sql.ToString();
}
The above worked for me. Hope this helps others!
Oracle would be fine with the Top-N query if the inner SQL would consist of the first statement (including where and order by).
So, i'd say there is no Oracle specific reason to omit them.
Never used subsonic, don't know if you need to do it different there.
Performance wise, an index on DISABLED_DATE, CREATED_DATE should do the trick (see: http://blog.fatalmind.com/2010/07/30/analytic-top-n-queries/).
Suppose there is a fully populated array of data String[n][3] myData.
I want to do this:
for (String[] row : myData)
{
SQL = "update mytable set col3 = row[2]
where col1 = row[0] and col2=row[1];"
}
Obviously I've left a lot out, but I want to express the idea as succinctly as possible.
Is there a simple way of doing this in a single DB command?
How about a not so simple way?
EDITS:
Data is not coming from another table (it's a web form submission - Multiple Choice exam)
Seeing as the app is web facing, It's got to be injection proof. Parameterized Queries are my preferred way of going.
I'm using MS-SQL Server 2005
EDIT:Closing, and re-asking as Multiple DB Updates:
EDIT: Re-opened, as this appears to be a popular question
It depends on what database you are using. If you're using SQL Server 2008, you can use stored procedure TABLE parameters. This allows you to pass all your values into the stored procedure in a single table, then you can do:
update mytable set mytable.col1 = #tbl.col1
from mytable
inner join #tbl on mytable.col2 = #tbl.col2
If you're using SQL Server 2005, you can use XML. Format your values as XML, then use XQuery statements (ie, 'nodes' and 'value') to parse out the XML. This can also be done in a single SQL statement, and it doesn't require a stored procedure.
If you are using Sql Server you can use SqlBulkCopy. You would first have to put your data in a DataTable, which would be pretty easy since you already have it in a string array.
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopy.aspx
You can make a big string like:
for (String[] row : myData)
{
SQL += "update mytable set col3 = row[2]
where col1 = row[0] and col2=row[1];"
}
sqlDriver.doInsertQuery(SQL); // change this to your way of inserting into the db
And just commit it all at once. I'm not very good with SQL so that's how i would do it.
The sql engine will just split it by the ';' and do separate inserts on its own. It's ok to add it all in a string though. It's kind the same as if u copy a big string with multiple updates/inserts into the sql prompt
This may not be the answer you want, but opening a transaction, executing your statements and then committing the transaction would, from a database point of view, do what you describe.
The state of the database does not change for other users of the database until the transaction has been completed, and that probably is the preferred effect.
If you are using Enterprise Library in your data access layer, you can create the transaction in .Net, iterate through your procedure calls, then commit/rollback all from .Net.
DbTransaction transaction = connection.BeginTransaction();
try
{
for (String[] row : myData)
{
ListDictionary params = new Specialized.ListDictionary();
params.add("#col3", row[2]);
params.add("#col1", row[0]);
params.add("#col2", row[1]);
executeNonQuery("myUpdateProcedure", params);
}
transaction.commit();
}
catch(Exception ex)
{
transaction.rollback();
throw ex;
}
finally
{
connection.close();
}
If for whatever reason you can't perform the update using one of the methods suggested above, the highly inefficient approach below would probably work for you.
SQL = "Update myTable Set Col3 = Case "
for (String[] row : myData)
{
SQL += "When Col1 = " + Row[0] + " and Col2 = " + Row[1] + " then " + row[2] + " "
}
SQL + = "Else Col3 end"
That looks like you want to make an update A, over rows that has coditions B and C. (A, B, C) are stored as tuples (rows) in myData. Isn't it?
Maybe (if you're using Microsoft SQL Server... I don't know if it exists in Oracle, could be) you can use a JOIN with an UPDATE. You can declare an update over a table joined with another one. If myData comes from another table then you could do (it's not the correct syntax) :
UPDATE whatchanges wc INNER JOIN changes c ON <yourcondition>
SET wc.col1 = c.newvalue
WHERE ....
(if you want to apply all changes in "changes" table you don't have to use WHERE of course, the INNER JOIN already has selected the correct rows).
Of course there are limitations to this kind of update. And it's MS SQL proprietary. So if it's your case I'd suggest to look for it on MS web (keywords: UPDATE and JOIN)
Not really. You could create the string with the same loop, then pass your values as parameters, but that will still be multiple database commands.
for each whatever
sql += "UPDATE ... ;"
end for
execute (sql)
I suspect you will need to use multiple SQL statements. You may find a wrapper to handle the minutiae but underneath I'd imagine it'd iteratively run a SQL statement for each UPDATE.
emit an update that goes against a values table:
UPDATE myTable SET col3=c FROM myTable JOIN (
SELECT 1 as a, 2 as b, 'value1' as c UNION ALL
SELECT 3 as a, 4 as b, 'value2' as c -- etc...
) x ON myTable.col1=x.a AND myTable.col2=x.b
so you just put this together like this:
// make one of these for each row
String.Format("SELECT {0} as a, {1} as b, '{2}' as c",
row[0], row[1], row[2].Replace("'","''"))
// put it together
string expr = "UPDATE myTable SET col3=c FROM myTable JOIN (" +
String.Join(stringformatarray, " UNION ALL ") +
") x ON myTable.col1=x.a AND myTable.col2=x.b"
or you can use StringBuilder to put this together.
and then, of course, you execute this one string.
Working with a SqlCommand in C# I've created a query that contains a IN (list...) part in the where clause. Instead of looping through my string list generating the list I need for the query (dangerous if you think in sqlInjection). I thought that I could create a parameter like:
SELECT blahblahblah WHERE blahblahblah IN #LISTOFWORDS
Then in the code I try to add a parameter like this:
DataTable dt = new DataTable();
dt.Columns.Add("word", typeof(string));
foreach (String word in listOfWords)
{
dt.Rows.Add(word);
}
comm.Parameters.Add("LISTOFWORDS", System.Data.SqlDbType.Structured).Value = dt;
But this doesn't work.
Questions:
Am I trying something impossible?
Did I took the wrong approach?
Do I have mistakes in this approach?
Thanks for your time :)
What you are trying to do is possible but not using your current approach. This is a very common problem with all possible solutions prior to SQL Server 2008 having trade offs related to performance, security and memory usage.
This link shows some approaches for SQL Server 2000/2005
SQL Server 2008 supports passing a table value parameter.
I hope this helps.
You want to think about where that list comes from. Generally that information is in the database somewhere. For example, instead of this:
SELECT * FROM [Table] WHERE ID IN (1,2,3)
You could use a subquery like this:
SELECT * FROM [Table] WHERE ID IN ( SELECT TableID FROM [OtherTable] WHERE OtherTableID= #OtherTableID )
If I understand right, you're trying to pass a list as a SQL parameter.
Some folks have attempted this before with limited success:
Passing Arrays to Stored Procedures
Arrays and Lists in SQL 2005
Passing Array of Values to SQL Server without String Manipulation
Using MS SQL 2005's XML capabilities to pass a list of values to a command
Am I trying something impossible?
No, it isn't impossible.
Did I took the wrong approach?
Your approach is not working (at least in .net 2)
Do I have mistakes in this approach?
I would try "Joel Coehoorn" solution (2nd answers) if it is possible.
Otherwise, another option is to send a "string" parameter with all values delimited by an separator. Write a dynamic query (build it based on values from string) and execute it using "exec".
Another solution will be o build the query directly from code. Somthing like this:
StringBuilder sb = new StringBuilder();
for (int i=0; i< listOfWords.Count; i++)
{
sb.AppendFormat("p{0},",i);
comm.Parameters.AddWithValue("p"+i.ToString(), listOfWords[i]);
}
comm.CommandText = string.Format(""SELECT blahblahblah WHERE blahblahblah IN ({0})",
sb.ToString().TrimEnd(','));
The command should look like:
SELECT blah WHERE blah IN (p0,p1,p2,p3...)...p0='aaa',p1='bbb'
In MsSql2005, "IN" is working only with 256 values.
I would recommend setting the parameter as a comma delimited string of values and use a Split function in SQL to turn that into a single column table of values and then you can use the IN feature.
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=50648 - Split Functions
If you want to pass the list as a string in a parameter, you could just build the query dynamically.
DECLARE #query varchar(500)
SET #query = 'SELECT blah blah WHERE blahblah in (' + #list + ')'
EXECUTE(#query)
I used to have the same problem, I think there is now way to do this directly over the ADO.NET API.
You might consider inserting the words into a temptable (plus a queryid or something) and then refering to that temptable from the query. Or dynamically creating the query string and avoid sql injection by other measures (e.g. regex checks).
This is an old question but I've come up with an elegant solution for this that I love to reuse and I think everyone else will find it useful.
First of all you need to create a FUNCTION in SqlServer that takes a delimited input and returns a table with the items split into records.
Here is the following code for this:
ALTER FUNCTION [dbo].[Split]
(
#RowData nvarchar(max),
#SplitOn nvarchar(5) = ','
)
RETURNS #RtnValue table
(
Id int identity(1,1),
Data nvarchar(100)
)
AS
BEGIN
Declare #Cnt int
Set #Cnt = 1
While (Charindex(#SplitOn,#RowData)>0)
Begin
Insert Into #RtnValue (data)
Select
Data = ltrim(rtrim(Substring(#RowData,1,Charindex(#SplitOn,#RowData)-1)))
Set #RowData = Substring(#RowData,Charindex(#SplitOn,#RowData)+1,len(#RowData))
Set #Cnt = #Cnt + 1
End
Insert Into #RtnValue (data)
Select Data = ltrim(rtrim(#RowData))
Return
END
You can now do something like this:
Select Id, Data from dbo.Split('123,234,345,456',',')
And fear not, this can't be susceptible to Sql injection attacks.
Next write a stored procedure that takes your comma delimited data and then you can write a sql statement that uses this Split function:
CREATE PROCEDURE [dbo].[findDuplicates]
#ids nvarchar(max)
as
begin
select ID
from SomeTable with (nolock)
where ID in (select Data from dbo.Split(#ids,','))
end
Now you can write a C# wrapper around it:
public void SomeFunction(List<int> ids)
{
var idsAsDelimitedString = string.Join(",", ids.Select(id => id.ToString()).ToArray());
// ... or however you make your connection
var con = GetConnection();
try
{
con.Open();
var cmd = new SqlCommand("findDuplicates", con);
cmd.Parameters.Add(new SqlParameter("#ids", idsAsDelimitedString));
var reader = cmd.ExecuteReader();
// .... do something here.
}
catch (Exception)
{
// catch an exception?
}
finally
{
con.Close();
}
}