Programatically inserting rows where field value is SQL statement - c#

I am currently working on a solution in C# to copy database tables across from Oracle to PostgreSQL. Everything is working great apart from one thing. When I get to copying one of my table which contains sql statements in one of its fields it falls over due to having two single quotes back to back. The SQL statements MUST remain in the table as it is being used to make another program database agnostic.
Is there a way to write the following SQL but without having the two single quotes back to back as seen near the 'TRUE' value near the end of the line. I have also included my code below to show how the statement is built up in C#. The column is a varchar2 in Oracle and a TEXT column in PostgreSQL.
EDIT: The sql example shown is the actual INSERT statement generated by my C# code which will then be run on the postgresql database to add a record to a table. It will be used to insert a text field, among others, that contains an sql statement in the form of a string.
INSERT INTO SQL_FACTORY_TEST (SQL_FACTORY_TEST_ID,SQL_FACTORY_DIALECT,SQL_FACTORY_QUERY_NAME,SQL_FACTORY_SQL_COMMAND,USER_NAME)
VALUES (21, 'ORACLE', 'GET_CLUSTERS', 'SELECT CLUSTER_ID, NUM_POINTS, FEATURE_PK, A.CELL_CENTROID.SDO_POINT.X, A.CELL_CENTROID.SDO_POINT.Y, A.CLUSTER_CENTROID.SDO_POINT.X, A.CLUSTER_CENTROID.SDO_POINT.Y, TO_CHAR (A.CLUSTER_EXTENT.GET_WKT ()), TO_CHAR (A.CELL_GEOM.GET_WKT ()), A.CLUSTER_EXTENT.SDO_SRID FROM (SELECT CLUSTER_ID, NUM_POINTS, FEATURE_PK, SDO_CS.transform (CLUSTER_CENTROID, 4326) cluster_centroid, CLUSTER_EXTENT, SDO_CS.transform (CELL_CENTROID, 4326) cell_centroid, CELL_GEOM FROM :0) a where sdo_filter( A.CELL_GEOM, SDO_CS.transform(mdsys.sdo_geometry(2003, :1, NULL, mdsys.sdo_elem_info_array(1,1003,3),mdsys.sdo_ordinate_array(:2, :3, :4, :5)),81989)) = 'TRUE'', 'PUBLIC')
Code sample:
oleDataBaseConnection.OleExecutePureSqlQuery("SELECT * FROM " + tableName);
if (oleDataBaseConnection.HasRows())
{
while (oleDataBaseConnection.NextRecord())
{
Dictionary<string, string> postgreSQLQueries = TypeConversion.GetQueryDictionary("POSTGRESQL");
string postgreSQLInsertQuery;
postgreSQLQueries.TryGetValue("INSERT", out postgreSQLInsertQuery);
postgreSQLInsertQuery = postgreSQLInsertQuery.Replace("{0}", tableName);
StringBuilder postgresQuery = new StringBuilder();
postgresQuery.Append(postgreSQLInsertQuery);
postgresQuery.Append("(");
int columnCounter = 0;
//add a column parameter to query for each of our columns
foreach (KeyValuePair<string, string> t in columnData)
{
postgresQuery.Append(t.Key + ",");
columnCounter++;
}
postgresQuery = postgresQuery.Remove(postgresQuery.Length - 1, 1);
postgresQuery.Append(") ");
postgresQuery.Append("VALUES (");
//Loop through values and
for (int i = 0; i < columnCounter; i++)
{
string[] foo = new string[columnData.Count];
columnData.Values.CopyTo(foo, 0);
if (foo[i].ToUpper() == "TEXT")
{
postgresQuery.Append("'" + oleDataBaseConnection.GetFieldById(i) + "', ");
}
else
{
postgresQuery.Append(oleDataBaseConnection.GetFieldById(i) + ", ");
}
}
postgresQuery = postgresQuery.Remove(postgresQuery.Length - 2, 2);
postgresQuery.Append(") ");
postgresSQLDBConnection.PostgreSQLExecutePureSqlNonQuery(postgresQuery.ToString());
}
}

The best solution is to use a PreparedStatement where you don't pass literals directly. I don't know C#, so I can't give you an example for that.
However if you have to keep the statement like that, you can use a "dollar quoted" string literal in PostgreSQL.
INSERT INTO SQL_FACTORY_TEST (SQL_FACTORY_TEST_ID,SQL_FACTORY_DIALECT,SQL_FACTORY_QUERY_NAME,SQL_FACTORY_SQL_COMMAND,USER_NAME)
VALUES (21, 'ORACLE', 'GET_CLUSTERS', $$ ...... 'TRUE'$$, 'PUBLIC')
The $$ replaces the single quote(s) around the literal. If there is a chance that your value contains $$ you can also add some unique identifier to it, e.g.
$42$String with embedded single quotes ''' and two $$ dollar characters$42$

I think the best approach is to use a parameterised insert query. so you are actually building a query like:
INSERT INTO [Table] VALUES (#Column1, #Column2, #Column3)
then passing the variable into the query. Something along the lines of:
List<OleDbParameter> parameters = new List<OleDbParameter>();
for (int i = 0; i < columnCounter; i++)
{
postgresQuery.Append(string.Format("#Column{0}, ", i));
parameters.Add(new OleDbParameter(string.Format("#Column{0}, ", i), oleDataBaseConnection.GetFieldById(i));
}
Then pass paremeters.ToArray() to you OleDbCommand when executing. This means c# will do all the escaping, and work out the data type for you. It may need a bit of tweaking to suit your needs but the general gist of it is there.

You should always properly escape and unescape values stored in the database.
The PostgreSQL client library provides the PQescapeLiteral function for this purpose.

Related

String or binary data would be truncated exception when inserting data [duplicate]

I am running data.bat file with the following lines:
Rem Tis batch file will populate tables
cd\program files\Microsoft SQL Server\MSSQL
osql -U sa -P Password -d MyBusiness -i c:\data.sql
The contents of the data.sql file is:
insert Customers
(CustomerID, CompanyName, Phone)
Values('101','Southwinds','19126602729')
There are 8 more similar lines for adding records.
When I run this with start > run > cmd > c:\data.bat, I get this error message:
1>2>3>4>5>....<1 row affected>
Msg 8152, Level 16, State 4, Server SP1001, Line 1
string or binary data would be truncated.
<1 row affected>
<1 row affected>
<1 row affected>
<1 row affected>
<1 row affected>
<1 row affected>
Also, I am a newbie obviously, but what do Level #, and state # mean, and how do I look up error messages such as the one above: 8152?
From #gmmastros's answer
Whenever you see the message....
string or binary data would be truncated
Think to yourself... The field is NOT big enough to hold my data.
Check the table structure for the customers table. I think you'll find that the length of one or more fields is NOT big enough to hold the data you are trying to insert. For example, if the Phone field is a varchar(8) field, and you try to put 11 characters in to it, you will get this error.
I had this issue although data length was shorter than the field length.
It turned out that the problem was having another log table (for audit trail), filled by a trigger on the main table, where the column size also had to be changed.
In one of the INSERT statements you are attempting to insert a too long string into a string (varchar or nvarchar) column.
If it's not obvious which INSERT is the offender by a mere look at the script, you could count the <1 row affected> lines that occur before the error message. The obtained number plus one gives you the statement number. In your case it seems to be the second INSERT that produces the error.
Just want to contribute with additional information: I had the same issue and it was because of the field wasn't big enough for the incoming data and this thread helped me to solve it (the top answer clarifies it all).
BUT it is very important to know what are the possible reasons that may cause it.
In my case i was creating the table with a field like this:
Select '' as Period, * From Transactions Into #NewTable
Therefore the field "Period" had a length of Zero and causing the Insert operations to fail. I changed it to "XXXXXX" that is the length of the incoming data and it now worked properly (because field now had a lentgh of 6).
I hope this help anyone with same issue :)
Some of your data cannot fit into your database column (small). It is not easy to find what is wrong. If you use C# and Linq2Sql, you can list the field which would be truncated:
First create helper class:
public class SqlTruncationExceptionWithDetails : ArgumentOutOfRangeException
{
public SqlTruncationExceptionWithDetails(System.Data.SqlClient.SqlException inner, DataContext context)
: base(inner.Message + " " + GetSqlTruncationExceptionWithDetailsString(context))
{
}
/// <summary>
/// PArt of code from following link
/// http://stackoverflow.com/questions/3666954/string-or-binary-data-would-be-truncated-linq-exception-cant-find-which-fiel
/// </summary>
/// <param name="context"></param>
/// <returns></returns>
static string GetSqlTruncationExceptionWithDetailsString(DataContext context)
{
StringBuilder sb = new StringBuilder();
foreach (object update in context.GetChangeSet().Updates)
{
FindLongStrings(update, sb);
}
foreach (object insert in context.GetChangeSet().Inserts)
{
FindLongStrings(insert, sb);
}
return sb.ToString();
}
public static void FindLongStrings(object testObject, StringBuilder sb)
{
foreach (var propInfo in testObject.GetType().GetProperties())
{
foreach (System.Data.Linq.Mapping.ColumnAttribute attribute in propInfo.GetCustomAttributes(typeof(System.Data.Linq.Mapping.ColumnAttribute), true))
{
if (attribute.DbType.ToLower().Contains("varchar"))
{
string dbType = attribute.DbType.ToLower();
int numberStartIndex = dbType.IndexOf("varchar(") + 8;
int numberEndIndex = dbType.IndexOf(")", numberStartIndex);
string lengthString = dbType.Substring(numberStartIndex, (numberEndIndex - numberStartIndex));
int maxLength = 0;
int.TryParse(lengthString, out maxLength);
string currentValue = (string)propInfo.GetValue(testObject, null);
if (!string.IsNullOrEmpty(currentValue) && maxLength != 0 && currentValue.Length > maxLength)
{
//string is too long
sb.AppendLine(testObject.GetType().Name + "." + propInfo.Name + " " + currentValue + " Max: " + maxLength);
}
}
}
}
}
}
Then prepare the wrapper for SubmitChanges:
public static class DataContextExtensions
{
public static void SubmitChangesWithDetailException(this DataContext dataContext)
{
//http://stackoverflow.com/questions/3666954/string-or-binary-data-would-be-truncated-linq-exception-cant-find-which-fiel
try
{
//this can failed on data truncation
dataContext.SubmitChanges();
}
catch (SqlException sqlException) //when (sqlException.Message == "String or binary data would be truncated.")
{
if (sqlException.Message == "String or binary data would be truncated.") //only for EN windows - if you are running different window language, invoke the sqlException.getMessage on thread with EN culture
throw new SqlTruncationExceptionWithDetails(sqlException, dataContext);
else
throw;
}
}
}
Prepare global exception handler and log truncation details:
protected void Application_Error(object sender, EventArgs e)
{
Exception ex = Server.GetLastError();
string message = ex.Message;
//TODO - log to file
}
Finally use the code:
Datamodel.SubmitChangesWithDetailException();
Another situation in which you can get this error is the following:
I had the same error and the reason was that in an INSERT statement that received data from an UNION, the order of the columns was different from the original table. If you change the order in #table3 to a, b, c, you will fix the error.
select a, b, c into #table1
from #table0
insert into #table1
select a, b, c from #table2
union
select a, c, b from #table3
on sql server you can use SET ANSI_WARNINGS OFF like this:
using (SqlConnection conn = new SqlConnection("Data Source=XRAYGOAT\\SQLEXPRESS;Initial Catalog='Healthy Care';Integrated Security=True"))
{
conn.Open();
using (var trans = conn.BeginTransaction())
{
try
{
using cmd = new SqlCommand("", conn, trans))
{
cmd.CommandText = "SET ANSI_WARNINGS OFF";
cmd.ExecuteNonQuery();
cmd.CommandText = "YOUR INSERT HERE";
cmd.ExecuteNonQuery();
cmd.Parameters.Clear();
cmd.CommandText = "SET ANSI_WARNINGS ON";
cmd.ExecuteNonQuery();
trans.Commit();
}
}
catch (Exception)
{
trans.Rollback();
}
}
conn.Close();
}
I had the same issue. The length of my column was too short.
What you can do is either increase the length or shorten the text you want to put in the database.
Also had this problem occurring on the web application surface.
Eventually found out that the same error message comes from the SQL update statement in the specific table.
Finally then figured out that the column definition in the relating history table(s) did not map the original table column length of nvarchar types in some specific cases.
I had the same problem, even after increasing the size of the problematic columns in the table.
tl;dr: The length of the matching columns in corresponding Table Types may also need to be increased.
In my case, the error was coming from the Data Export service in Microsoft Dynamics CRM, which allows CRM data to be synced to an SQL Server DB or Azure SQL DB.
After a lengthy investigation, I concluded that the Data Export service must be using Table-Valued Parameters:
You can use table-valued parameters to send multiple rows of data to a Transact-SQL statement or a routine, such as a stored procedure or function, without creating a temporary table or many parameters.
As you can see in the documentation above, Table Types are used to create the data ingestion procedure:
CREATE TYPE LocationTableType AS TABLE (...);
CREATE PROCEDURE dbo.usp_InsertProductionLocation
#TVP LocationTableType READONLY
Unfortunately, there is no way to alter a Table Type, so it has to be dropped & recreated entirely. Since my table has over 300 fields (😱), I created a query to facilitate the creation of the corresponding Table Type based on the table's columns definition (just replace [table_name] with your table's name):
SELECT 'CREATE TYPE [table_name]Type AS TABLE (' + STRING_AGG(CAST(field AS VARCHAR(max)), ',' + CHAR(10)) + ');' AS create_type
FROM (
SELECT TOP 5000 COLUMN_NAME + ' ' + DATA_TYPE
+ IIF(CHARACTER_MAXIMUM_LENGTH IS NULL, '', CONCAT('(', IIF(CHARACTER_MAXIMUM_LENGTH = -1, 'max', CONCAT(CHARACTER_MAXIMUM_LENGTH,'')), ')'))
+ IIF(DATA_TYPE = 'decimal', CONCAT('(', NUMERIC_PRECISION, ',', NUMERIC_SCALE, ')'), '')
AS field
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = '[table_name]'
ORDER BY ORDINAL_POSITION) AS T;
After updating the Table Type, the Data Export service started functioning properly once again! :)
When I tried to execute my stored procedure I had the same problem because the size of the column that I need to add some data is shorter than the data I want to add.
You can increase the size of the column data type or reduce the length of your data.
A 2016/2017 update will show you the bad value and column.
A new trace flag will swap the old error for a new 2628 error and will print out the column and offending value. Traceflag 460 is available in the latest cumulative update for 2016 and 2017:
https://support.microsoft.com/en-sg/help/4468101/optional-replacement-for-string-or-binary-data-would-be-truncated
Just make sure that after you've installed the CU that you enable the trace flag, either globally/permanently on the server:
...or with DBCC TRACEON:
https://learn.microsoft.com/en-us/sql/t-sql/database-console-commands/dbcc-traceon-trace-flags-transact-sql?view=sql-server-ver15
Another situation, in which this error may occur is in
SQL Server Management Studio. If you have "text" or "ntext" fields in your table,
no matter what kind of field you are updating (for example bit or integer).
Seems that the Studio does not load entire "ntext" fields and also updates ALL fields instead of the modified one.
To solve the problem, exclude "text" or "ntext" fields from the query in Management Studio
This Error Comes only When any of your field length is greater than the field length specified in sql server database table structure.
To overcome this issue you have to reduce the length of the field Value .
Or to increase the length of database table field .
If someone is encountering this error in a C# application, I have created a simple way of finding offending fields by:
Getting the column width of all the columns of a table where we're trying to make this insert/ update. (I'm getting this info directly from the database.)
Comparing the column widths to the width of the values we're trying to insert/ update.
Assumptions/ Limitations:
The column names of the table in the database match with the C# entity fields. For eg: If you have a column like this in database:
You need to have your Entity with the same column name:
public class SomeTable
{
// Other fields
public string SourceData { get; set; }
}
You're inserting/ updating 1 entity at a time. It'll be clearer in the demo code below. (If you're doing bulk inserts/ updates, you might want to either modify it or use some other solution.)
Step 1:
Get the column width of all the columns directly from the database:
// For this, I took help from Microsoft docs website:
// https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlconnection.getschema?view=netframework-4.7.2#System_Data_SqlClient_SqlConnection_GetSchema_System_String_System_String___
private static Dictionary<string, int> GetColumnSizesOfTableFromDatabase(string tableName, string connectionString)
{
var columnSizes = new Dictionary<string, int>();
using (var connection = new SqlConnection(connectionString))
{
// Connect to the database then retrieve the schema information.
connection.Open();
// You can specify the Catalog, Schema, Table Name, Column Name to get the specified column(s).
// You can use four restrictions for Column, so you should create a 4 members array.
String[] columnRestrictions = new String[4];
// For the array, 0-member represents Catalog; 1-member represents Schema;
// 2-member represents Table Name; 3-member represents Column Name.
// Now we specify the Table_Name and Column_Name of the columns what we want to get schema information.
columnRestrictions[2] = tableName;
DataTable allColumnsSchemaTable = connection.GetSchema("Columns", columnRestrictions);
foreach (DataRow row in allColumnsSchemaTable.Rows)
{
var columnName = row.Field<string>("COLUMN_NAME");
//var dataType = row.Field<string>("DATA_TYPE");
var characterMaxLength = row.Field<int?>("CHARACTER_MAXIMUM_LENGTH");
// I'm only capturing columns whose Datatype is "varchar" or "char", i.e. their CHARACTER_MAXIMUM_LENGTH won't be null.
if(characterMaxLength != null)
{
columnSizes.Add(columnName, characterMaxLength.Value);
}
}
connection.Close();
}
return columnSizes;
}
Step 2:
Compare the column widths with the width of the values we're trying to insert/ update:
public static Dictionary<string, string> FindLongBinaryOrStringFields<T>(T entity, string connectionString)
{
var tableName = typeof(T).Name;
Dictionary<string, string> longFields = new Dictionary<string, string>();
var objectProperties = GetProperties(entity);
//var fieldNames = objectProperties.Select(p => p.Name).ToList();
var actualDatabaseColumnSizes = GetColumnSizesOfTableFromDatabase(tableName, connectionString);
foreach (var dbColumn in actualDatabaseColumnSizes)
{
var maxLengthOfThisColumn = dbColumn.Value;
var currentValueOfThisField = objectProperties.Where(f => f.Name == dbColumn.Key).First()?.GetValue(entity, null)?.ToString();
if (!string.IsNullOrEmpty(currentValueOfThisField) && currentValueOfThisField.Length > maxLengthOfThisColumn)
{
longFields.Add(dbColumn.Key, $"'{dbColumn.Key}' column cannot take the value of '{currentValueOfThisField}' because the max length it can take is {maxLengthOfThisColumn}.");
}
}
return longFields;
}
public static List<PropertyInfo> GetProperties<T>(T entity)
{
//The DeclaredOnly flag makes sure you only get properties of the object, not from the classes it derives from.
var properties = entity.GetType()
.GetProperties(System.Reflection.BindingFlags.Public
| System.Reflection.BindingFlags.Instance
| System.Reflection.BindingFlags.DeclaredOnly)
.ToList();
return properties;
}
Demo:
Let's say we're trying to insert someTableEntity of SomeTable class that is modeled in our app like so:
public class SomeTable
{
[Key]
public long TicketID { get; set; }
public string SourceData { get; set; }
}
And it's inside our SomeDbContext like so:
public class SomeDbContext : DbContext
{
public DbSet<SomeTable> SomeTables { get; set; }
}
This table in Db has SourceData field as varchar(16) like so:
Now we'll try to insert value that is longer than 16 characters into this field and capture this information:
public void SaveSomeTableEntity()
{
var connectionString = "server=SERVER_NAME;database=DB_NAME;User ID=SOME_ID;Password=SOME_PASSWORD;Connection Timeout=200";
using (var context = new SomeDbContext(connectionString))
{
var someTableEntity = new SomeTable()
{
SourceData = "Blah-Blah-Blah-Blah-Blah-Blah"
};
context.SomeTables.Add(someTableEntity);
try
{
context.SaveChanges();
}
catch (Exception ex)
{
if (ex.GetBaseException().Message == "String or binary data would be truncated.\r\nThe statement has been terminated.")
{
var badFieldsReport = "";
List<string> badFields = new List<string>();
// YOU GOT YOUR FIELDS RIGHT HERE:
var longFields = FindLongBinaryOrStringFields(someTableEntity, connectionString);
foreach (var longField in longFields)
{
badFields.Add(longField.Key);
badFieldsReport += longField.Value + "\n";
}
}
else
throw;
}
}
}
The badFieldsReport will have this value:
'SourceData' column cannot take the value of
'Blah-Blah-Blah-Blah-Blah-Blah' because the max length it can take is
16.
Kevin Pope's comment under the accepted answer was what I needed.
The problem, in my case, was that I had triggers defined on my table that would insert update/insert transactions into an audit table, but the audit table had a data type mismatch where a column with VARCHAR(MAX) in the original table was stored as VARCHAR(1) in the audit table, so my triggers were failing when I would insert anything greater than VARCHAR(1) in the original table column and I would get this error message.
I used a different tactic, fields that are allocated 8K in some places. Here only about 50/100 are used.
declare #NVPN_list as table
nvpn varchar(50)
,nvpn_revision varchar(5)
,nvpn_iteration INT
,mpn_lifecycle varchar(30)
,mfr varchar(100)
,mpn varchar(50)
,mpn_revision varchar(5)
,mpn_iteration INT
-- ...
) INSERT INTO #NVPN_LIST
SELECT left(nvpn ,50) as nvpn
,left(nvpn_revision ,10) as nvpn_revision
,nvpn_iteration
,left(mpn_lifecycle ,30)
,left(mfr ,100)
,left(mpn ,50)
,left(mpn_revision ,5)
,mpn_iteration
,left(mfr_order_num ,50)
FROM [DASHBOARD].[dbo].[mpnAttributes] (NOLOCK) mpna
I wanted speed, since I have 1M total records, and load 28K of them.
This error may be due to less field size than your entered data.
For e.g. if you have data type nvarchar(7) and if your value is 'aaaaddddf' then error is shown as:
string or binary data would be truncated
You simply can't beat SQL Server on this.
You can insert into a new table like this:
select foo, bar
into tmp_new_table_to_dispose_later
from my_table
and compare the table definition with the real table you want to insert the data into.
Sometime it's helpful sometimes it's not.
If you try inserting in the final/real table from that temporary table it may just work (due to data conversion working differently than SSMS for example).
Another alternative is to insert the data in chunks, instead of inserting everything immediately you insert with top 1000 and you repeat the process, till you find a chunk with an error. At least you have better visibility on what's not fitting into the table.

Varying number of records

I am executing a SQL Server stored procedure from my C# code which essentially pulls some data from a database based on the supplied condition.
DataSet GetAllxxxxxByDate(string entityValue,string companyValue)
{
using (var sqlConn = new SqlConnection(GetConnectionString()))
{
using (var cmd = new SqlCommand())
{
var data = new DataSet();
cmd.CommandText = “myStoredprodecure”;
cmd.CommandType = CommandType.StoredProcedure;
cmd.Connection = sqlConn;
var eVal = string.IsNullOrWhiteSpace(entityValue) ? string.Empty : entityValue;
cmd.Parameters.AddWithValue("#entity_value", eVal);
var company = string.IsNullOrWhiteSpace(companyValue) ? string.Empty : companyValue;
cmd.Parameters.AddWithValue("#company", company);
var sqlDataAdaptor = new SqlDataAdapter(cmd);
sqlDataAdaptor.Fill(data);
return data;
}
}
}
Here entityValue, companyValue are comma separated strings, formed dynamically within C# code and pass it to stored procedure.
Eg:
’first’,’second’,’third’
And the stored procedure uses these values to fill the NOT IN condition defined within it.
The issue is that, I am getting inconsistent number of records when I execute the code.
Following is a quick screenshot where first WHERE clause return 3 records and second WHERE clause return 1 record. The input values for the first WHERE clause is been filled from c# code and the second is been filled manually to test.
The only difference, which I can spot is number of quotes.
Question: can someone help me to zero in the issue or the difference in these give WHERE clause ?
Well, you don't show what entity_value is in your results, but the difference between the two is you're adding single quotes around the literal values:
N'''FSEC''' in SQL is the literal valiue 'FSEC'
'FSEC' in SQL is just FSEC (without the quotes).
My guess is that records 2004981 and 2004982 have a value of FSEC (without the quotes) for entity_value.
If you're adding parameter values from C# code, don't add quotes around them like you would if you were building a string. SQL will treat the values as strings without needing string qualifiers.
EDIT
Okay, I just read this statement:
Here entityValue, companyValue are comma separated string
You can't just pass in a comma-delimited string to an IN clause. To search for multiple values there are a few options:
Add commas to each end and use LIKE:
Where (',' + #entity_value +',' LIKE '%,' + entity_value + ',%')
Parse the string into a temporary table, than use that table in your IN clause
Pass the values as a table-valued parameter and use that in your IN clause.
Build the SQL statement as a string and execute it with EXEC

Transferring parsed strings to sql server database

I have this program that is feeding me data. I take this data (string) and parse it so that the different fields can go into the respective db table column. I can parse the string but I can't find the right function or way to send them to the db. This is my second time working with sql server or database in general. I have done inserts this way
MyCommand.CommandType = System.Data.CommandType.Text;
MyCommand.CommandText = "INSERT INTO TimeStampTable(ID, TimeStamp) VALUES ('24', 'sep 13, 2009')";
From what I know, CommandType only allows either text or a stored procedure. In this case, I would want to insert the string that is being parsed.
string teststring = dtString;
string[] result = teststring.Split(',', ' ', ':', '=');
Console.WriteLine("The parsed string looks like this:");
foreach (string word in result)
{
Console.WriteLine(word);
}
This is my code that parses my incoming string. So I receive name address zip state, etc. I would like for name to go to col1, address to go to col2, etc. I think the ideal way to do this would be to convert my loop to something like this
foreach (string word in result)
{
SqlDatasource.InsertCommand=Insert into Tablename col1 col2 col3(word);
}
Does anyone have any suggestions?
I will answer your question directly, but there are many different ways that you could go about performing the same thing (I will list some at the end)
String insertQuery = "INSERT INTO TABLENAME (Col1, Col2, Col3...) VALUES (";
//This is also assuming that your data is in the same order as the columns
int isFirstLoop = true
foreach(string word in result)
{
if(!isFirstLoop)
insertQuery += ","
insertQuery += word;
isFirstLoop = false;
}
insertQuery += ")";
SqlDataSource.InsertCommand = insertQuery;
NOTE: this is very open to SQL Injection, so keep that in mind (do you trust your incoming source). There are ways to clean the data, but ultimately, I suggest some of the methods listed below
Alternatives:
Use a stored procedure over direct TSQL. Then you can map your data to SQLParameters, which (I believe) are built to scrub the data to protect against SQL Injection
Use a very basic ORM and/or LINQ so that you can work with objects directly. Then you only need to read the data into a POCO
I am sure there are other ways, however for some reason I am drawing a blank. I think that is because these are the most used alternatives :)

Adding Parameter SQL IN

I just say if i have a SQL like this;
SELECT B.HESAP_NO
FROM S_TEKLIF B
WHERE B.HESAP NO = 234
I can add some parameters this SQL programaticly like this C# code;
if (txtBoxText1 != "")
{
strQuery = strQuery + " AND A.ISL_TAR >= #S_TARIH_B";
dt_stb = DateTime.Parse(txtBoxText1);
myCommand.Parameters.AddWithValue("#S_TARIH_B", dt_stb);
}
With this code, i can add AND "A.ISL_TAR >= #S_TARIH_B" to i my SQL.
BUT, in SQL IN operators i don't know how can i add some parameters.
For example;
SELECT A.HESAP_NO
FROM S_TEKLIF A
WHERE A.HESAP_NO IN (SELECT A.HESAP FROM S_TEKLIF WHERE A.MUS_K_ISIM =
for (int counter = 0; counter < list.Items.Count; counter++)
{
if (list.Items[counter].Selected)
{
values.Add(list.Items[counter].Value);
}
})
How can i add a parameters to SQL IN ?
There are a number of ways to achieve this, and all of them are described in the Arrays and Lists in SQL Server article. It's a gread read for any SQL developer.
I'm not aware of any way to pass a list/array as a single parameter for use as part of an IN clause. Your best option may be to build up the IN clause yourself as a string.
Alternatively, you could do this in steps.
1. Create a temp table with just an ID field
2. Create a parameterised query to insert one value into that temp table
3. Loop through your values, calling the sql to insert values into the temp table
4. Exexute your main SQL query, joining on to the temp table instead of using IN
You just need to ensure you don't close and re-open you session each time, thus ensuring your temp table is persisted and retains the values you've been inserting.

String list in SqlCommand through Parameters in C#

Working with a SqlCommand in C# I've created a query that contains a IN (list...) part in the where clause. Instead of looping through my string list generating the list I need for the query (dangerous if you think in sqlInjection). I thought that I could create a parameter like:
SELECT blahblahblah WHERE blahblahblah IN #LISTOFWORDS
Then in the code I try to add a parameter like this:
DataTable dt = new DataTable();
dt.Columns.Add("word", typeof(string));
foreach (String word in listOfWords)
{
dt.Rows.Add(word);
}
comm.Parameters.Add("LISTOFWORDS", System.Data.SqlDbType.Structured).Value = dt;
But this doesn't work.
Questions:
Am I trying something impossible?
Did I took the wrong approach?
Do I have mistakes in this approach?
Thanks for your time :)
What you are trying to do is possible but not using your current approach. This is a very common problem with all possible solutions prior to SQL Server 2008 having trade offs related to performance, security and memory usage.
This link shows some approaches for SQL Server 2000/2005
SQL Server 2008 supports passing a table value parameter.
I hope this helps.
You want to think about where that list comes from. Generally that information is in the database somewhere. For example, instead of this:
SELECT * FROM [Table] WHERE ID IN (1,2,3)
You could use a subquery like this:
SELECT * FROM [Table] WHERE ID IN ( SELECT TableID FROM [OtherTable] WHERE OtherTableID= #OtherTableID )
If I understand right, you're trying to pass a list as a SQL parameter.
Some folks have attempted this before with limited success:
Passing Arrays to Stored Procedures
Arrays and Lists in SQL 2005
Passing Array of Values to SQL Server without String Manipulation
Using MS SQL 2005's XML capabilities to pass a list of values to a command
Am I trying something impossible?
No, it isn't impossible.
Did I took the wrong approach?
Your approach is not working (at least in .net 2)
Do I have mistakes in this approach?
I would try "Joel Coehoorn" solution (2nd answers) if it is possible.
Otherwise, another option is to send a "string" parameter with all values delimited by an separator. Write a dynamic query (build it based on values from string) and execute it using "exec".
Another solution will be o build the query directly from code. Somthing like this:
StringBuilder sb = new StringBuilder();
for (int i=0; i< listOfWords.Count; i++)
{
sb.AppendFormat("p{0},",i);
comm.Parameters.AddWithValue("p"+i.ToString(), listOfWords[i]);
}
comm.CommandText = string.Format(""SELECT blahblahblah WHERE blahblahblah IN ({0})",
sb.ToString().TrimEnd(','));
The command should look like:
SELECT blah WHERE blah IN (p0,p1,p2,p3...)...p0='aaa',p1='bbb'
In MsSql2005, "IN" is working only with 256 values.
I would recommend setting the parameter as a comma delimited string of values and use a Split function in SQL to turn that into a single column table of values and then you can use the IN feature.
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=50648 - Split Functions
If you want to pass the list as a string in a parameter, you could just build the query dynamically.
DECLARE #query varchar(500)
SET #query = 'SELECT blah blah WHERE blahblah in (' + #list + ')'
EXECUTE(#query)
I used to have the same problem, I think there is now way to do this directly over the ADO.NET API.
You might consider inserting the words into a temptable (plus a queryid or something) and then refering to that temptable from the query. Or dynamically creating the query string and avoid sql injection by other measures (e.g. regex checks).
This is an old question but I've come up with an elegant solution for this that I love to reuse and I think everyone else will find it useful.
First of all you need to create a FUNCTION in SqlServer that takes a delimited input and returns a table with the items split into records.
Here is the following code for this:
ALTER FUNCTION [dbo].[Split]
(
#RowData nvarchar(max),
#SplitOn nvarchar(5) = ','
)
RETURNS #RtnValue table
(
Id int identity(1,1),
Data nvarchar(100)
)
AS
BEGIN
Declare #Cnt int
Set #Cnt = 1
While (Charindex(#SplitOn,#RowData)>0)
Begin
Insert Into #RtnValue (data)
Select
Data = ltrim(rtrim(Substring(#RowData,1,Charindex(#SplitOn,#RowData)-1)))
Set #RowData = Substring(#RowData,Charindex(#SplitOn,#RowData)+1,len(#RowData))
Set #Cnt = #Cnt + 1
End
Insert Into #RtnValue (data)
Select Data = ltrim(rtrim(#RowData))
Return
END
You can now do something like this:
Select Id, Data from dbo.Split('123,234,345,456',',')
And fear not, this can't be susceptible to Sql injection attacks.
Next write a stored procedure that takes your comma delimited data and then you can write a sql statement that uses this Split function:
CREATE PROCEDURE [dbo].[findDuplicates]
#ids nvarchar(max)
as
begin
select ID
from SomeTable with (nolock)
where ID in (select Data from dbo.Split(#ids,','))
end
Now you can write a C# wrapper around it:
public void SomeFunction(List<int> ids)
{
var idsAsDelimitedString = string.Join(",", ids.Select(id => id.ToString()).ToArray());
// ... or however you make your connection
var con = GetConnection();
try
{
con.Open();
var cmd = new SqlCommand("findDuplicates", con);
cmd.Parameters.Add(new SqlParameter("#ids", idsAsDelimitedString));
var reader = cmd.ExecuteReader();
// .... do something here.
}
catch (Exception)
{
// catch an exception?
}
finally
{
con.Close();
}
}

Categories