I use raw SQL with parameters in Dapper. Query speed is normal.
Something like this:
string sql = "SELECT * FROM SomeTable WHERE messages IN ('Volvo', 'BMW', 'Ford', 'Mazda')"
var results = conn.Query(sql);
When I change parameters with #messages, the speed is too slow:
string sql = "SELECT * FROM SomeTable WHERE messages IN (#cars)"
string[] cars = { "Volvo", "BMW", "Ford", "Mazda" };
var results = conn.Query(sql, new {cars});
PS: string[] size is almost 300.
If you pass string as parameter, Dapper treats it as nvarchar type.
So if your messages is varchar type and the SomeTable is huge, even if it's indexed, the speed could be really slow. DbString may solve this problem.
cars.ToList().Select(x => new DbString { Value = x, IsFixedLength = false, IsAnsi = true });
(would be a mess as a comment)
(I think there might be some syntax error in second one and there shouldn't be parentheses.)
The two queries would generate different SQL sent to backend. ie: with MS SQL server they would look like:
First one:
SELECT * FROM SomeTable WHERE messages IN ('Volvo', 'BMW', 'Ford', 'Mazda')
Second one:
exec sp_executesql N'SELECT * FROM SomeTable WHERE messages IN (#cars1, #cars2, #cars3, #cars4)',
N'#cars1 nvarchar(4000),#cars2 nvarchar(4000),#cars3 nvarchar(4000),#cars4 nvarchar(4000)',
#cars1=N'Volvo',#cars2=N'BMW',#cars3=N'Ford',#cars4=N'Mazda'
IMHO there are better ways of doing IN queries depending on your backend.
Related
I have a table with a lot of employees in it, every person has a Name column with their full name.
I then want to do a query similar to this when searching for people:
SELECT * FROM Employees WHERE Name LIKE '%' + #value1 + '%' AND Name LIKE '%' + #value2 +'%' AND so forth...
for an arbitrary array of values.
My Dapper code would look something like this:
public IEnumerable<Employee> Search(string[] words)
{
using var connection = CreateConnection();
connection.Query<Employee>("SELECT * etc.", words);
}
Is there ANY way to do this with SQL without resorting to string concatenation, and the risk of SQL Injection attacks that follows?
Caveat: I don't know how Dapper actually passes an array to the query, which limits my creative ideas for working around this :-D
And also: Changing the Table structure is, unfortunately, out of the question. And I'd rather avoid fetching every single person into .Net memory and doing the filtering there.
Is there ANY way to do this with SQL without resorting to string concatenation, and the risk of SQL Injection attacks that follows?
Because the set of where conditions is not fixed you will need to build the query dynamically. But that does not mean you cannot parameterise the query, you just build the parameter list alongside building the query. Each time a word from the list add to the condition and add a parameter.
As Dapper doesn't directly include anything that takes a collection of DbParameter, consider using ADO.NET to get an IDataReader and then Dappter's
IEnumerable<T> Parse<T>(this IDataReader reader)
for the mapping.
Such a builder would be very roughly
var n = 0;
for (criterion in cirteria) {
var cond = $"{crition.column} like #p{n}";
var p = new SqlPatameter($"#p{n}", $"%{crition.value}%";
conditions.Add(cond);
cmd.Parameters.Add(p);
}
var sql = "select whetever from table where " + String.Join(" and ", conditions);
cmd.CommandText = sql;
var reader = await cmd.ExecuteReaderAsync();
var res = reader.Parse<TResult>();
For performance reasons, it's much better to do this as a set-based operation.
You can pass through a datatable as a Table-Value Parameter, then join on that with LIKE as the condition. In this case you want all values to match, so you need a little bit of relational division.
First create your table type:
CREATE TYPE dbo.StringList AS TABLE (str varchar(100) NOT NULL);
Your SQL is as follows:
SELECT *
FROM Employees e
WHERE NOT EXISTS (SELECT 1
FROM #words w
WHERE e.Name NOT LIKE '%' + w.str + '%' ESCAPE '/' -- if you want to escape wildcards you need to add ESCAPE
);
Then you pass through the list as follows:
public IEnumerable<Employee> Search(string[] words)
{
var table = new DataTable{ Columns = {
{"str", typeof(string)},
} };
foreach (var word in words)
table.Rows.Add(SqlLikeEscape(word)); // make a function that escapes wildcards
using var connection = CreateConnection();
return connection.Query<Employee>(yourQueryHere, new
{
words = table.AsTableValuedParameter("dbo.StringList"),
});
}
So I have this query:
var retval = db.v_AViewOfJoinedTables.SqlQuery("SELECT * FROM dbo.v_AViewOfJoinedTables WHERE #filter = '#value';",
new SqlParameter("#filter", columnname),
new SqlParameter("#value", value)).ToList();
But I'm not getting any result. If I run this query through SSMS, it returns data. I know if I use a table instead that it works. So can SqlQuery only work on tables or is something wrong my query or code?
UPDATED CODE:
SqlCommandBuilder scb = new SqlCommandBuilder();
string columnname = userNameFilterType;
string escapedColumnName = scb.QuoteIdentifier(columnname);
string sqlQuery = String.Format("SELECT * FROM dbo.v_AViewOfJoinedTables WHERE {0} = '#username';", escapedColumnName);
var retval = db.v_AViewOfJoinedTables.SqlQuery(sqlQuery, new SqlParameter("username", currentUser)).ToList();
return retval.AsQueryable();
The problem isn't the view. You can't use SQL parameters for table or column names.
... where #filter = ...
That won't work. It must be text. Just use string interpolation to insert it:
var retval = db.v_AViewOfJoinedTables.SqlQuery($"SELECT * FROM dbo.v_AViewOfJoinedTables WHERE {columnname} = #value;",
new SqlParameter("#value", value)).ToList();
The SQL Execution Engine will cache the execution plan of queries you run, but it can't do that if the query itself changes. The plan might change depending on the column name, for instance. So, SQL parameters will only ever work as values supplied to the query.
You also need to remove the single quotes around #value. The SQL Server provider will handle the quote insertion for you.
I'm running the following query
cmd = new SqlCommand("SELECT * FROM addresses WHERE identifier NOT IN(#notIn)", _connector.getMsConnection());
When I view the value notIn and copy this query I get an empty result on my database (which I'm expecting). However when I'm running this code I get 6 results. The content of string notIN is for example
string notIn = "'201619011124027899693E8M2S3WOCKT9G6KHE11' ,'201619011124027899693E8M2S3WOCKT9G6KHE12'"
which combined with
SELECT *
FROM addresses
WHERE identifier NOT IN(#notIn)
Should create
SELECT *
FROM addresses
WHERE identifier NOT IN ('201619011124027899693E8M2S3WOCKT9G6KHE11',
'201619011124027899693E8M2S3WOCKT9G6KHE12' )
which runs as expected.
it should be like this:
cmd = new SqlCommand(string.Format("SELECT * FROM addresses WHERE identifier NOT IN({0})", notIn), _connector.getMsConnection());
This way the value of notIn will be concat to your string query.
Contrary to what the other answers say, concatenating the string to build the SQL is a bad idea, especially since the input values are strings. You open yourself up to SQL injection attacks.
You should be generating multiple parameters for each item in your list.
For example, if you have the input input:
var notIn = new[] { "A1", "B2", "C3" }
You'd want something like
for(var i = 0; i < notIn.Length; i++)
command.AddParamWithValue("p"+i, notIn);
And then you can build the SQL with concatenation (note that we are not concatenating an input here)
var sql = "SELECT * FROM addresses WHERE identifier NOT IN(" + string.Join(",", notIn.Select(i,v) => { "#p" + i; }) + ")";
Which then would look like:
SELECT * FROM addresses WHERE identifier NOT IN (#p0,#p1,#p2)
Alternatively, you could dump the values into a temporary table and do a join.
Note that the above is pseudocode, and may not compile verbatim, but should give you the right idea about how to procede.
It's because, you passed the #notIn as a whole string, which means, the SQL server see it as:
SELECT * FROM addresses WHERE identifier NOT IN('''201619011124027899693E8M2S3WOCKT9G6KHE11'',''201619011124027899693E8M2S3WOCKT9G6KHE12''')
So you got empty result
Try changing the "not in" to where clause and generate the where with C#:
string selectStatement = "SELECT * FROM addresses WHERE";
selectStatement += " identifier != '201619011124027899693E8M2S3WOCKT9G6KHE11' and identifier != '201619011124027899693E8M2S3WOCKT9G6KHE12'";
Or if you really want to use parameterized SQL, try doing it in stored procedure instead.
I'm trying to parameterize a raw SQL query for an Oracle synonym (non-entity) in EF 4 and I am having some problems. Currently I am doing something like the code below, based on some examples that I saw:
string term="foo";
OracleParameter p = new OracleParameter("#param1", term);
object[] parameters = new object[] { p };
var model = db.Database.SqlQuery<ProjectTask>("SELECT * FROM (SELECT * FROM web_project_task_vw WHERE project_num like '%#param1%') WHERE rownum<=100", parameters).ToList();
Running this doesn't return any results. If I replace the parameter with something like
"SELECT * FROM web_project_task_vw WHERE project_num like '%"+term+"%'"
it returns the results I expect, but this is obviously a SQL injection risk.
Can anyone point me in the right direction for how parameters are supposed to work in EF 4 for an Oracle DB?
Thanks.
First, like Mohammed wrote, you need to prefix the parameter with ':', but not as you define it, just in the query.
Second, you are currently searching not for the value of the parameter but rather strings that contains the string #param1. So surround the value of the parameter with % and you should get a result.
So it should look something like this:
string term="foo";
OracleParameter p = new OracleParameter("param1", term);
object[] parameters = new object[] { p };
var model = db.Database.SqlQuery<ProjectTask>("SELECT * FROM (SELECT * FROM web_project_task_vw WHERE project_num like '%'||:param1||'%') WHERE rownum<=100", parameters).ToList();
Your p might have an incorrect parameter name; the name should be param1, not #param1. Your query is also incorrect; replace '%#param1%' with '%:param1%'.
Working with a SqlCommand in C# I've created a query that contains a IN (list...) part in the where clause. Instead of looping through my string list generating the list I need for the query (dangerous if you think in sqlInjection). I thought that I could create a parameter like:
SELECT blahblahblah WHERE blahblahblah IN #LISTOFWORDS
Then in the code I try to add a parameter like this:
DataTable dt = new DataTable();
dt.Columns.Add("word", typeof(string));
foreach (String word in listOfWords)
{
dt.Rows.Add(word);
}
comm.Parameters.Add("LISTOFWORDS", System.Data.SqlDbType.Structured).Value = dt;
But this doesn't work.
Questions:
Am I trying something impossible?
Did I took the wrong approach?
Do I have mistakes in this approach?
Thanks for your time :)
What you are trying to do is possible but not using your current approach. This is a very common problem with all possible solutions prior to SQL Server 2008 having trade offs related to performance, security and memory usage.
This link shows some approaches for SQL Server 2000/2005
SQL Server 2008 supports passing a table value parameter.
I hope this helps.
You want to think about where that list comes from. Generally that information is in the database somewhere. For example, instead of this:
SELECT * FROM [Table] WHERE ID IN (1,2,3)
You could use a subquery like this:
SELECT * FROM [Table] WHERE ID IN ( SELECT TableID FROM [OtherTable] WHERE OtherTableID= #OtherTableID )
If I understand right, you're trying to pass a list as a SQL parameter.
Some folks have attempted this before with limited success:
Passing Arrays to Stored Procedures
Arrays and Lists in SQL 2005
Passing Array of Values to SQL Server without String Manipulation
Using MS SQL 2005's XML capabilities to pass a list of values to a command
Am I trying something impossible?
No, it isn't impossible.
Did I took the wrong approach?
Your approach is not working (at least in .net 2)
Do I have mistakes in this approach?
I would try "Joel Coehoorn" solution (2nd answers) if it is possible.
Otherwise, another option is to send a "string" parameter with all values delimited by an separator. Write a dynamic query (build it based on values from string) and execute it using "exec".
Another solution will be o build the query directly from code. Somthing like this:
StringBuilder sb = new StringBuilder();
for (int i=0; i< listOfWords.Count; i++)
{
sb.AppendFormat("p{0},",i);
comm.Parameters.AddWithValue("p"+i.ToString(), listOfWords[i]);
}
comm.CommandText = string.Format(""SELECT blahblahblah WHERE blahblahblah IN ({0})",
sb.ToString().TrimEnd(','));
The command should look like:
SELECT blah WHERE blah IN (p0,p1,p2,p3...)...p0='aaa',p1='bbb'
In MsSql2005, "IN" is working only with 256 values.
I would recommend setting the parameter as a comma delimited string of values and use a Split function in SQL to turn that into a single column table of values and then you can use the IN feature.
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=50648 - Split Functions
If you want to pass the list as a string in a parameter, you could just build the query dynamically.
DECLARE #query varchar(500)
SET #query = 'SELECT blah blah WHERE blahblah in (' + #list + ')'
EXECUTE(#query)
I used to have the same problem, I think there is now way to do this directly over the ADO.NET API.
You might consider inserting the words into a temptable (plus a queryid or something) and then refering to that temptable from the query. Or dynamically creating the query string and avoid sql injection by other measures (e.g. regex checks).
This is an old question but I've come up with an elegant solution for this that I love to reuse and I think everyone else will find it useful.
First of all you need to create a FUNCTION in SqlServer that takes a delimited input and returns a table with the items split into records.
Here is the following code for this:
ALTER FUNCTION [dbo].[Split]
(
#RowData nvarchar(max),
#SplitOn nvarchar(5) = ','
)
RETURNS #RtnValue table
(
Id int identity(1,1),
Data nvarchar(100)
)
AS
BEGIN
Declare #Cnt int
Set #Cnt = 1
While (Charindex(#SplitOn,#RowData)>0)
Begin
Insert Into #RtnValue (data)
Select
Data = ltrim(rtrim(Substring(#RowData,1,Charindex(#SplitOn,#RowData)-1)))
Set #RowData = Substring(#RowData,Charindex(#SplitOn,#RowData)+1,len(#RowData))
Set #Cnt = #Cnt + 1
End
Insert Into #RtnValue (data)
Select Data = ltrim(rtrim(#RowData))
Return
END
You can now do something like this:
Select Id, Data from dbo.Split('123,234,345,456',',')
And fear not, this can't be susceptible to Sql injection attacks.
Next write a stored procedure that takes your comma delimited data and then you can write a sql statement that uses this Split function:
CREATE PROCEDURE [dbo].[findDuplicates]
#ids nvarchar(max)
as
begin
select ID
from SomeTable with (nolock)
where ID in (select Data from dbo.Split(#ids,','))
end
Now you can write a C# wrapper around it:
public void SomeFunction(List<int> ids)
{
var idsAsDelimitedString = string.Join(",", ids.Select(id => id.ToString()).ToArray());
// ... or however you make your connection
var con = GetConnection();
try
{
con.Open();
var cmd = new SqlCommand("findDuplicates", con);
cmd.Parameters.Add(new SqlParameter("#ids", idsAsDelimitedString));
var reader = cmd.ExecuteReader();
// .... do something here.
}
catch (Exception)
{
// catch an exception?
}
finally
{
con.Close();
}
}