We Keep Coding

SQL paramaterisation in C#, using SQL declared variables

I have the following SQL query which I am sending from a C# program:
DECLARE #id as int
SELECT #id = max(fault_catagory_ident) FROM fault_catagory_list
INSERT INTO fault_catagory_list (fault_catagory_ident, fault_catagory)
VALUES (#id + 1, 'TEST')
SELECT #id + 1
The 'fault_catagory' value is coming from my program, but the ident value needs to be the next number in line (primary key) from the existing table in the database. My C# code is parameterising values for security.
I have two problems:
How can I get the #id + 1 value returned to my program (executeNonQuery doesn't return anything)?
How can I get #id as a parametarised value for the insert command?
I am wondering if my primary key could be automated in some way?
I want to carry all this out in one single query, as there will be a risk of multiple logins running this same query. If any happened to run simultainiously, the #id value may get duplicated and one would fail.
Apologies if there isn't enough info here, I'm on a learning curve!
Any advice would be greatly appreciated. Thanks in advance.

I think that you will find everything you need in this example provided in MSDN:
https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlcommand.executescalar?view=dotnet-plat-ext-5.0
in short:
to return a single parameter from a query ull use ExecuteScalar()
parameters are added to a query through the SqlCommand class provided in System.Data.SqlClient.
cheers!

Thanks for the advice, think I have found the solution...
Firstly I need to recreate the table with the primary key column using the IDENTITY constraint (makes sense to use this now I know it exists!). Found a guide here (though it will mean some rebuilding of primary/foreign key links) https://www.datameer.com/blog/how-to-add-an-identity-to-an-existing-column-in-sql/
Then in C# program, use SqlCommand.executeScalar to return the identity value
Int identReturn = 0;
identReturn = (Int32)cmd.ExecuteScalar();
Thanks again for the responses.

While I don't agree with the logic in which you are choosing identity row values for the inserted rows, you could certainly acheive that using IDENTITY() column attribute in the SQL table definition.
In case you have multiple SELECTs in your SQL command,
everytime you have a SELECT in your sql command, a new table is added to the passed in dataset to the data adapter.
string sql = "
DECLARE #id as int
SELECT #id = max(fault_catagory_ident) FROM fault_catagory_list
INSERT INTO fault_catagory_list (fault_catagory_ident, fault_catagory)
VALUES (#id + 1, 'TEST')
SELECT #id + 1";
SqlCommand cmd = new SqlCommand(sql, conn); // assuming you already set connection
SqlDataAdapter da = new SqlDataAdapter(cmd);
DataSet ds = new DataSet();
da.Fill(ds);
Console.WriteLine( ds.Tables[0].Rows[0][0]); // this will print #id
Console.WriteLine( ds.Tables[1].Rows[0][0]); // this will print #id + 1

Map identity value to object after merge statement

I have a table called People with the following schema:
Id INT NOT NULL IDENTITY(1, 1)
FirstName NVARCHAR(64) NOT NULL
LastName NVARCHAR964) NOT NULL
I am using a query like this one to perform inserts and updates in one query:
MERGE INTO People AS TARGET
USING ( VALUES
(#id0, #firstName0, #lastname0),
(#id1, #firstName1, #lastname1)
...
) AS SOURCE ([Id],[FirstName],[LastName])
ON TARGET.[Id] = SOURCE.[Id]
WHEN MATCHED BY TARGET THEN
UPDATE SET
[FirstName] = SOURCE.[FirstName],
[LastName] = SOURCE.[LastName]
WHEN NOT MATCHED BY TARGET THEN
INSERT ([FirstName],[LastName])
VALUES ([FirstName],[LastName])
WHEN NOT MATCHED BY SOURCE THEN
DELETE
OUTPUT $action, INSERTED.*;
My application is structured such that the client calls back to the server to load the existing state of the app. The client then creates/modifies/deletes entities locally and pushes those changes to the server in one bunch.
Here's an example of what my "SaveEntities" code currently looks like:
public void SavePeople(IEnumerable<Person> people)
{
// Returns the query I mentioned above
var query = GetMergeStatement(people);
using(var command = new SqlCommand(query))
{
using(var reader = command.ExecuteReader())
{
while(reader.Read())
{
// how do I tie these records back to
// the objects in the people collection?
}
}
}
}
I can use the value in the $action column to filter down to just INSERTED records. INSERTED.* returns all of the columns in TARGET for the inserted record. The problem is I have no way of distinctly linking those results back to the collection of objects passed into this method.
The only solution I could think of was to add a writable GUID column to the table and allow the MERGE statement to specify that value so I could link back to these objects in code using that and assign the ID value from there, but that seems like it defeats the purpose of having an automatic identity column and feels convoluted.
I'm really curious how this can work because I know Entity Framework does something to mitigate this problem (to be clear, I believe I'd have to same problem were I to be using a pure INSERT statement instead of a MERGE). In EF I can add objects to the model and call Entity.SaveChanges() and have the entity's ID property auto-update using magic. I guess it's that kind of magic I'm looking to understand more.
Also, I know I could structure my saves to insert one record at a time and cascade the changes appropriately (by returning SCOPE_IDENTITY for every insert) but this would be terribly inefficient.

One of the things I love about the MERGE statement is that the source data is in scope in the OUTPUT clause.
OUTPUT $action, SOURCE.Id, INSERTED.Id;
On insert, this will give you three columns: 'INSERT' in the first, the values of #id0 and #id1 in the second, and the matching, newly inserted Id values in the third.
In your C# code, just read the rows as you normally would.
while (reader.Read())
{
string action = reader.GetString(0);
if (action == "INSERT")
{
int oldId = reader.GetInt32(1);
int newId = reader.GetInt32(2);
// Now do what you want with them.
}
}
You can check for "DELETE" and "UPDATE" too, but keep in mind that ordinal 2 will be NULL on "DELETE" so you need to make sure you check for that before calling reader.GetInt32 in that case.
I've used this, in combination with table variables (OUTPUT SOURCE.Id, INSERTED.Id INTO #PersonMap ([OldId], [NewId])), to copy hierarchies 4 and 5 tables deep, all with identity columns.

Identity column in SQL Server exposes strange behaviour

I have an identity (auto increment integer) column in my data table in a SQL Server database.
When I start my program and add new record to this table identity column always equals -1. For the next record it becomes -2 and so on. I add new record this way:
http://msdn.microsoft.com/en-us/library/5ycd1034.aspx
However when I restart my program all identity values are reordered (become 1, 2, ...).
Any ideas why this happens? It would be no issue if i could delete these records without restarting. I use SQL Server 2008.
Also is there any way to specify MAX size for column data type through GUI interface (when adding table in visual studio 2012 server explorer) ?

Why they are negative I don't know. However, when you reload the application those records already exist in the database and have id's that were assigned when they were committed to the database; that's why they have real values on restart.
But, you don't need to delete the records from the DataTable, you just need to refresh that row after committing it to the database. There are a number of ways to do this, and would depend significantly on exactly how you're accessing your data now, but you can do things like tack on the SELECT SCOPE_IDENTITY() command with the INSERT command and then use ExecuteScalar to commit the row, like this:
var insertCmd = "INSERT INTO tbl (fld1, fld2) VALUES (#fld1, #fld2); SELECT SCOPE_IDENTITY()";
using (var c = new SqlConnection(connString))
using (var cmd = new SqlCommand(insertCmd, c))
{
cmd.AddParameterWithValue("#fld1", fld1Value);
cmd.AddParameterWithValue("#fld2", fld2Value);
var result = cmd.ExecuteScalar();
int id;
if (int.TryParse(result, out id))
{
// update the DataTable row here
dataTable.Rows[index]["id_column"] = id;
dataTable.AcceptChanges();
}
}
You could even choose to reload the entire DataTable after performing the update.

Possible to insert with a Table Parameter, and also retrieve identity values?

I'm trying to insert records using a high performance table parameter method ( http://www.altdevblogaday.com/2012/05/16/sql-server-high-performance-inserts/ ), and I'm curious if it's possible to retrieve back the identity values for each record I insert.
At the moment, the answer appears to be no - I insert the data, then retrieve back the identity values, and they don't match. Specifically, they don't match about 75% of the time, and they don't match in unpredictable ways. Here's some code that replicates this issue:
// Create a datatable with 100k rows
DataTable dt = new DataTable();
dt.Columns.Add(new DataColumn("item_id", typeof(int)));
dt.Columns.Add(new DataColumn("comment", typeof(string)));
for (int i = 0; i < 100000; i++) {
dt.Rows.Add(new object[] { 0, i.ToString() });
}
// Insert these records and retrieve back the identity
using (SqlConnection conn = new SqlConnection("Data Source=localhost;Initial Catalog=testdb;Integrated Security=True")) {
conn.Open();
using (SqlCommand cmd = new SqlCommand("proc_bulk_insert_test", conn)) {
cmd.CommandType = CommandType.StoredProcedure;
// Adding a "structured" parameter allows you to insert tons of data with low overhead
SqlParameter param = new SqlParameter("#mytable", SqlDbType.Structured);
param.Value = dt;
cmd.Parameters.Add(param);
SqlDataReader dr = cmd.ExecuteReader();
// Set all the records' identity values
int i = 0;
while (dr.Read()) {
dt.Rows[i].ItemArray = new object[] { dr.GetInt32(0), dt.Rows[i].ItemArray[1] };
i++;
}
dr.Close();
}
// Do all the records' ID numbers match what I received back from the database?
using (SqlCommand cmd = new SqlCommand("SELECT * FROM bulk_insert_test WHERE item_id >= #base_identity ORDER BY item_id ASC", conn)) {
cmd.Parameters.AddWithValue("#base_identity", (int)dt.Rows[0].ItemArray[0]);
SqlDataReader dr = cmd.ExecuteReader();
DataTable dtresult = new DataTable();
dtresult.Load(dr);
}
}
The database is defined using this SQL server script:
CREATE TABLE bulk_insert_test (
item_id int IDENTITY (1, 1) NOT NULL PRIMARY KEY,
comment varchar(20)
)
GO
CREATE TYPE bulk_insert_table_type AS TABLE ( item_id int, comment varchar(20) )
GO
CREATE PROCEDURE proc_bulk_insert_test
#mytable bulk_insert_table_type READONLY
AS
DECLARE #TableOfIdentities TABLE (IdentValue INT)
INSERT INTO bulk_insert_test (comment)
OUTPUT Inserted.item_id INTO #TableOfIdentities(IdentValue)
SELECT comment FROM #mytable
SELECT * FROM #TableOfIdentities
Here's the problem: the values returned from proc_bulk_insert_test are not in the same order as the original records were inserted. Therefore, I can't programmatically assign each record the item_id value I received back from the OUTPUT statement.
It seems like the only valid solution is to SELECT back the entire list of records I just inserted, but frankly I'd prefer any solution that would reduce the amount of data piped across my SQL Server's network card. Does anyone have better solutions for large inserts while still retrieving identity values?
EDIT: Let me try clarifying the question a bit more. The problem is that I would like my C# program to learn what identity values SQL Server assigned to the data that I just inserted. The order isn't essential; but I would like to be able to take an arbitrary set of records within C#, insert them using the fast table parameter method, and then assign their auto-generated ID numbers in C# without having to requery the entire table back into memory.
Given that this is an artificial test set, I attempted to condense it into as small of a readable bit of code as possible. Let me describe what methods I have used to resolve this issue:
In my original code, in the application this example came from, I would insert about 15 million rows using 15 million individual insert statements, retrieving back the identity value after each insert. This worked but was slow.
I revised the code using high performance table parameters for insertion. I would then dispose of all of the objects in C#, and read back from the database the entire objects. However, the original records had dozens of columns with lots of varchar and decimal values, so this method was very network traffic intensive, although it was fast and it worked.
I now began research to figure out whether it was possible to use the table parameter insert, while asking SQL Server to just report back the identity values. I tried scope_identity() and OUTPUT but haven't been successful so far on either.
Basically, this problem would be solved if SQL Server would always insert the records in exactly the order I provided them. Is it possible to make SQL server insert records in exactly the order they are provided in a table value parameter insert?
EDIT2: This approach seems very similar to what Cade Roux cites below:
http://www.sqlteam.com/article/using-the-output-clause-to-capture-identity-values-on-multi-row-inserts
However, in the article, the author uses a magic unique value, "ProductNumber", to connect the inserted information from the "output" value to the original table value parameter. I'm trying to figure out how to do this if my table doesn't have a magic unique value.

Your TVP is an unordered set, just like a regular table. It only has order when you specify as such. Not only do you not have any way to indicate actual order here, you're also just doing a SELECT * at the end with no ORDER BY. What order do you expect here? You've told SQL Server, effectively, that you don't care. That said, I implemented your code and had no problems getting the rows back in the right order. I modified the procedure slightly so that you can actually tell which identity value belongs to which comment:
DECLARE #TableOfIdentities TABLE (IdentValue INT, comment varchar(20))
INSERT INTO bulk_insert_test (comment)
OUTPUT Inserted.item_id, Inserted.comment
INTO #TableOfIdentities(IdentValue, comment)
SELECT comment FROM #mytable
SELECT * FROM #TableOfIdentities
Then I called it using this code (we don't need all the C# for this):
DECLARE #t bulk_insert_table_type;
INSERT #t VALUES(5,'foo'),(2,'bar'),(3,'zzz');
SELECT * FROM #t;
EXEC dbo.proc_bulk_insert_test #t;
Results:
1 foo
2 bar
3 zzz
If you want to make sure the output is in the order of identity assignment (which isn't necessarily the same "order" that your unordered TVP has), you can add ORDER BY item_id to the last select in your procedure.
If you want to insert into the destination table so that your identity values are in an order that is important to you, then you have a couple of options:
add a column to your TVP and insert the order into that column, then use a cursor to iterate over the rows in that order, and insert one at a time. Still more efficient than calling the entire procedure for each row, IMHO.
add a column to your TVP that indicates order, and use an ORDER BY on the insert. This isn't guaranteed, but is relatively reliable, particularly if you eliminate parallelism issues using MAXDOP 1.
In any case, you seem to be placing a lot of relevance on ORDER. What does your order actually mean? If you want to place some meaning on order, you shouldn't be doing so using an IDENTITY column.

You specify no ORDER BY on this: SELECT * FROM #TableOfIdentities so there's no guarantee of order. If you want them in the same order they were sent, do an INNER JOIN in that to the data that was inserted with an ORDER BY which matches the order the rows were sent in.

why sql delete query is not working

I have a table table1 with fields id(int), name(nchar), grade(real).
The following code isn't working. There are no errors or warnings. The code executes well but the number of affected rows = 0.
MsSql Server
sqlConnection1.Open();
SqlCommand cmd = new SqlCommand("Delete from [table1] where [id] = 1", sqlConnection1);
int c = cmd.ExecuteNonQuery();
sqlConnection1.Close();
All other queries are working well.

A slight expansion of what others have already asked. Are you certain that there are records to be deleted in your target table? Moreover, are you certain you are getting the table from the right database? It's possible the default is tempdb, for instance, and that just happens to have a table with the target name and with an id column.
First do a select from the SQL prompt to insure there are items of the type you are looking for:
SELECT TOP 10 * FROM [database].[schema].[table1] WHERE [id] = 1
If that provides results, try changing your command to explicitly state the database and schema as well:
DELETE FROM [database].[schema].[table1] WHERE [id] = 1

Thoughts:
is there a row with [id] 1
do you have a trigger that is firing?
my guess would be the second... the number is after triggers have been taken into account, and is the number of rows from the last operation.

I know this might sound silly, but is there data in the call with an ID of 1? Can you see it executing via SQL Profiler? What happens if you execute it via SSMS?

Does the query work when you run it in Sql Management studio?

Apart from Marc Gravell's comment about triggers you should also check, if there are foreign key constraints with ON DELETE RESTRICT` in place (and the error message somehow disappears before getting to you ...)