We Keep Coding

Select and read first record of table and delete record after that, in one stored procedure

I want to read records one-by-one and delete them after read. Table is a temp table, a multi thread program will use data of table. I need to read each record just once and not by multiple thread.
is there any solution by stored procedures to create this thread safe program(delete record just after read by first thread)?

First, I feel like I have to warn you that it's probably not the best idea to do it in SQL Server - relational databases works best with a set based approach, not on a row-by-row basis.
Reading and deleting each row individually will have a very poor performance.
Having said that, here's one way to delete a row, get it back to the client using the output clause, and (thanks to the rowlock hint) do it in a thread safe manner:
DELETE TOP(1)
FROM #tempTable WITH (ROWLOCK)
OUTPUT deleted.*
ORDER BY id

This should be your Stored Procedure code:
Create Procedure DeleteButOne
As
Begin
Select TOP 1 * From "Your Table Name"
DELETE FROM "Your Table Name" WHERE Id NOT IN (SELECT TOP 1 ID FROM "Your Table
Name")
End
And then you can execute procedure :
Execute DeleteButOne

Multiple SQL commands executed via "ExecuteReader" in a system I can only manipulate SQL statements

I have to configure a system which provides me with an area to input SQL statements.
It is important to notice that we cannot modify the system we are configuring.
I believe the system was built in C# (.net for sure but C# is a guess).
Any way, I'm trying to create a script that would:
create a temporary table
create a temporary procedure (which inserts into the table created)
call the temporary procedure 4 times
Read the temp table as a response to the system's call.
Something like:
CREATE Procedure #myTempProcedure(
#param1 nvarchar(max)
) as
begin
insert #tempTable (col1, col2) select aCol, bCol from table2 where col2 = #param1;
end;
CREATE TABLE #tempTable
(col1 nvarchar(512),
(col2 nvarchar(512));
EXEC #myTempProcedure N'val1';
EXEC #myTempProcedure N'val2';
EXEC #myTempProcedure N'val3';
EXEC #myTempProcedure N'val4';
select col1, col2 from #tempTable;
The system is very likely executing my script via C# SqlCommand.ExecuteReader() method. As I can simulate the problem in a simple C# application I created.
The problem is that when executing this script, the system (or SQL Server) assumes the procedure body to be the entire script and seems to disregard my ; in line 6 of the example above. My intention with this ; was to flag the end of the procedure creation.
Executing this script in Management studio requires a GO to be placed in line 7 of the example above, otherwise the same problem reported by the system would happen in Management Studio.
Is there a GO equivalent I could use in this script to get it to work??
Or is there a better way to script this??
I have a background in Oracle, and I'm still leaning SQL server usual tricks... The System accepts multiple commands apart from the create procedure here, So I'm inclined to believe there is a SQL Server trick I could use here.
Thank you in advance!

The problem is that syntactically there is no way to create a procedure and then do something after it in the same batch. The compiler doesn't know where it ends, and things like semi-colon don't fix it (because semi-colon only terminates a statement, not a batch).
Using dynamic SQL, (and fixing one syntax error) this works:
EXEC('
CREATE Procedure ##myTempProcedure(
#param1 nvarchar(max)
) as
begin
insert #tempTable (col1, col2) select aCol, bCol from table2 where col2 = #param1;
end;
');
CREATE TABLE #tempTable
(
col1 nvarchar(512),
col2 nvarchar(512)
);
EXEC ##myTempProcedure N'val1';
EXEC ##myTempProcedure N'val2';
EXEC ##myTempProcedure N'val3';
EXEC ##myTempProcedure N'val4';
select col1, col2 from #tempTable;
EXEC('DROP PROC ##myTempProcedure;');

1) please look at the rights on the server you many have some issues with if you cannot change or add anything to the system.
ie., Create procedure statements.
2) you could do a small exercise
open a connection object using the SqlConnection()
keep the connection open till you execute all you statements
ie., a) create your #table
b) execute your insert statement.
c) select * from your #table
this should get you back the data you are intending to get back from your temp table note i skipped the entire proc here.

Instead of creating stored procedure, you can execute sql statements delimited by semi colon. You can execute multiple statements this way. Also if you want to create a temp table and load it with data, you can use the same connection with multiple sql commands.

Given that the proc definition doesn't change, and that there is no real harm in the proc existing beyond the end of this particular process, it could just as easily be a regular (i.e. non-temporary) Stored Procedure that just happens to exist in tempdb. The benefit of using a regular Stored Procedure created in tempdb is that you do not need to worry about potential name collisions when using global temporary stored procedures. The script simply needs to ensure that the stored procedure exists. But there is no need to remove the Stored Procedure manually or have it automatically cleaned up.
The following code is adapted from #RBarryYoung's answer:
IF (OBJECT_ID(N'tempdb.dbo.myTempProcedure') IS NULL)
BEGIN
USE [tempdb];
EXEC('
CREATE PROCEDURE dbo.myTempProcedure(
#param1 NVARCHAR(MAX)
) AS
BEGIN
INSERT INTO #tempTable (col1, col2)
SELECT aCol, bCol
FROM table2
WHERE col2 = #param1;
END;
');
END;
CREATE TABLE #tempTable
(
col1 NVARCHAR(512),
col2 NVARCHAR(512)
);
EXEC tempdb.dbo.myTempProcedure N'val1';
EXEC tempdb.dbo.myTempProcedure N'val2';
EXEC tempdb.dbo.myTempProcedure N'val3';
EXEC tempdb.dbo.myTempProcedure N'val4';
SELECT col1, col2 FROM #tempTable;
The only difference here is that a non-temporary Stored Procedure does not execute in the context of the current database, but instead, like any other non-temporary Stored Procedure, runs within the context of the database where it exists, which in this case is tempdb. So the table that is being selected from (i.e. table2) needs to be fully-qualified. This means that if the proc needs to run in multiple databases and reference objects local to each of them, then this approach is probably not an option.

Pitfalls of using SQL transactions vs .NET transactions

I've been playing around with using transaction in SQL server and in C#. Consider a store procedure which inserts a row into a three column table
alter proc spInsertItem
#itemId int
,#itemDescription varchar(50)
,#itemCost decimal
as
begin
if(#itemCost < 0)
begin
raiserror('cost cannot be less than 0',16,1)
end
else
begin
begin try
begin tran
insert into Items(itemid, [description],itemCost)
values (#itemid, #itemdescription,#itemCost)
commit tran
end try
begin catch
rollback tran
select ERROR_LINE()as errorLine
,ERROR_MESSAGE() as errorMessage
,ERROR_STATE() as errorState
,ERROR_PROCEDURE() as errorProcedure
,ERROR_NUMBER() as errorNumber
end catch
end
end
vs
create proc spInsertItem2
#itemid int
,#itemDescription varchar(50)
,#itemCost decimal
as
begin
insert into Items(ItemId,[Description],ItemCost)
values (#itemid, #itemDescription,#itemCost)
end
In the first example the user is notified that they are unable to enter in an item cost less than 0, and the rest is pretty self explanatory. This got me to thinking, if you're going to want to disallow a certain values you should need a check constraint, so I added the following constraint
alter table items
add constraint chkItemCost
check (ItemCost > 0)
Now the two stored procedures function the same in code, and the SQL is much shorter and in my opinion, easier to read in the second, shorter version. Granted, this is a very rudimentary example, but to me it seems that if you see the try/catch in code when you call the stored procedure, you can be sure of the database not being put in an inconsistent state. So, what am I missing that I shouldn't rely on C# to create transactions?

This is usually a design decision; where the logic of application housed. If you decide upon concentrating your business logic in the application code, for each atomic application logic which involves multiple trips to the database, you need to wrap that logic using transaction in C#.
Whereas, if you house business logic in the database with the help of SPs, you do not need transaction in C#.
A common scenario is:
You create one or more records in the database.
You do some post processing on this data in C#.
You update another table with the data you just processed .
The requirement is that if step 2 or 3 fails, step 1 (created records) should be rolled back. For this you need transactions. You may argue that you can put all the three steps in an SP and wrap it with a transaction; it should be possible and is generally a matter of preference of where you put your application logic.

Insert multiple sql rows via stored proc

I have looked a some related topics but my question isn't quite answered:
C# - Inserting multiple rows using a stored procedure
Insert Update stored proc on SQL Server
Efficient Multiple SQL insertion
I have the following kind of setup when running my stored procedure in the code behind for my web application. The thing is I am now faced with the possibility of inserting multiple products and I would like to do it all in one ExecuteNonQuery rather than do a foreach loop and run it n number of times.
I am not sure how to do this, or if it can be, with my current setup.
The code should be somewhat self explanatory but if clarification is needed let me know. Thanks.
SqlDatabase database = new SqlDatabase(transMangr.ConnectionString);
DbCommand commandWrapper = StoredProcedureProvider.GetCommandWrapper(database, "proc_name", useStoredProc);
database.AddInParameter(commandWrapper, "#ProductID", DbType.Int32, entity._productID);
database.AddInParameter(commandWrapper, "#ProductDesc", DbType.String, entity._desc);
...more parameters...
Utility.ExecuteNonQuery(transMangr, commandWrapper);
Proc
ALTER PROCEDURE [dbo].[Products_Insert]
-- Add the parameters for the stored procedure here
#ProductID int,
#Link varchar(max)
#ProductDesc varchar(max)
#Date DateTime
AS BEGIN
SET NOCOUNT ON;
INSERT INTO [dbo].[Prodcuts]
(
[CategoryID],
[Link],
[Desc],
[Date]
)
VALUES
(
#ProductID,
#Link,
#ProductDesc,
#Date
)
END

You should be fine running your stored procedure in a loop. Just make sure that you commit rarely, not after every insert.
For alternatives, you have already found the discussion about loading data.
Personally, I like SQL bulk insert of the form insert into myTable (select *, literalValue from someOtherTable);
But that will probably not do in your case.

You could pass all your data as a table value parameter - MSDN has a pretty good write up about it here
Something along the lines of the following should work
CREATE TABLE dbo.tSegments
(
SegmentID BIGINT NOT NULL CONSTRAINT pkSegment PRIMARY KEY CLUSTERED,
SegCount BIGINT NOT NULL
);
CREATE TYPE dbo.SegmentTableType AS TABLE
(
SegmentID BIGINT NOT NULL
);
CREATE PROCEDURE dbo.sp_addSegments
#Segments dbo.SegmentTableType READONLY
AS
BEGIN
MERGE INTO dbo.tSegments AS tSeg
USING #Segments AS S
ON tSeg.SegmentID = S.SegmentID
WHEN MATCHED THEN UPDATE SET T.SegCount = T.SegCount + 1
WHEN NOT MATCHED THEN INSERT VALUES(tSeg.SegmentID, 1);
END

Define the commandWrapper and parameters for the command outside of the loop and then with in the loop you just assign parameter values and execute the proc.
SqlDatabase database = new SqlDatabase(transMangr.ConnectionString);
DbCommand commandWrapper = StoredProcedureProvider.GetCommandWrapper(database, "proc_name", useStoredProc);
database.AddInParameter(commandWrapper, "#ProductID", DbType.Int32 );
database.AddInParameter(commandWrapper, "#ProductDesc", DbType.String);
...more parameters...
foreach (var entity in entitties)
{
database.SetParameterValue(commandWrapper, "#ProductID",entity._productID);
database.SetParameterValue(commandWrapper, "#ProductDesc",entity._desc);
//..more parameters...
Utility.ExecuteNonQuery(transMangr, commandWrapper);
}

Not ideal from a purist way of doing things, but sometimes one is limited by frameworks and libraries, and that you are forced to call stored procedures in a certain way, bind parameters in a certain way, and that connections are managed by pools as part of your framework.
In such circumstances, a method we have found to work is to simply write your stored procedure with a lot of parameters, usually a name followed by a number, e.g. #ProductId1, #ProductDesc1, #ProductId2, #ProductDesc2 up to a number you decide, possibly say 32.
You can use some form of scripting language to produce the lines for this.
You can get the stored procedure to insert all the values first into a table parameter that allows nulls, then do bulk inserts / merges on this data in a way similar to Johnv2020's answer. You might remove the null rows first.
It will usually be more efficient than doing it one at a time (partly because of the database operations itself, and partly because of your framework's overheads in getting the connection to call the procedure etc.)

fastest Import of a csv to a database table

I have implemented an import functionality which takes data from a csv file in an Asp.Net appication. The file of the size can vary from a few kb's to a max of 10 MB.
However when an import occurs and if the file size is > 50000 it takes around 20 MINS .
Which is way too much of a time. I need to perform an import for around 300000 records within a timespan of 2-3 Mins .
I know that the import to a database also depends on the physical memory of the db server .I create insert scripts in bulk and execute . I also know using SqlBulkCopy would also be another option but in my case its just not the inserting of product's that take place but also update and delete that is a field called "FUNCTION CODE" which decides whether to Insert,Update Or Delete.
Any suggestions regarding as to how to go about this would be greatly appreciated.
One approach towards this would be to implement multiple threads which carry out processes simultaneosly ,but i have never implemented threading till date and hence am not aware of the complication i would incur by implementing the same.
Thanks & Regards,
Francis P.

SqlBulkCopy is definitely going to be fastest. I would approach this by inserting the data into a temporary table on the database. Once the data is in the temp table, you could use SQL to merge/insert/delete accordingly.

I guess you are using SQL Server...
If you are using 2005/2008 consider using SSIS to process the file. Technet
Importing huge amount of data within the asp.net process is not the best thing you can do. You might upload the file and start a process that is doing the magic for you.

If this is a repeated process and the file is uploaded via asp.net plus you are doing some decision making on the data to decide insert/update or delete, then try out http://www.codeproject.com/KB/database/CsvReader.aspx it is this fast csv reader. Its quite quick and economical with memory

You are doing all your database queries with 1 connection sequentially. So for every insert/update/delete you are sending the command through the wire, wait for the db to do it's thing, and then wake up again when something is sent back.
Databases are optimized for heavy parallel access. So there are 2 easy routes for a significant speedup:
Open X connections to the database (where you have to tweak X but just start with 5) and either: spin up 5 threads who each do a chunk of the same work you were doing.
or: use asynchronous calls and when a callback arrives shoot in the next query.

I suggest using the XML functionality in SQL Server 2005/2008, which will allow you to bulk insert and bulk update. I'd take the following approach:
Process the entire file into an in-memory data structure.
Create a single XML document from this structure to pass to a stored proc.
Create a stored proc to load data from the XML document into a temporary table, then perform the inserts and updates. See below for guidance on creating the stored proc.
There are numerous advantages to this approach:
The entire operation is completed in one database call, although if your dataset is really large you may want to batch it.
You can wrap all the database writes into a single transaction easily, and roll back if anything fails.
You are not using any dynamic SQL, which could have posed a security risk.
You can return the IDs of the inserted, updated and/or deleted records using the OUTPUT clause.
In terms of the stored proc you will need, something like the following should work:
CREATE PROCEDURE MyBulkUpdater
(
#p_XmlData VARCHAR(MAX)
)
AS
DECLARE #hDoc INT
EXEC sp_xml_preparedocument #hDoc OUTPUT, #p_XmlData
-- Temporary table, should contain the same schema as the table you want to update
CREATE TABLE #MyTempTable
(
-- ...
)
INSERT INTO #MyTempTable
(
[Field1],
[Field2]
)
SELECT
XMLData.Field1,
XMLData.Field2
FROM OPENXML (#hdoc, 'ROOT/MyRealTable', 1)
WITH
(
[Field1] int,
[Field2] varchar(50),
[__ORDERBY] int
) AS XMLData
EXEC sp_xml_removedocument #hDoc
Now you can simply insert, update and delete your real table from your temporary table as required eg
INSERT INTO MyRealTable (Field1, Field2)
SELECT Field1, Field2
FROM #MyTempTable
WHERE ...
UPDATE MyRealTable
SET rt.Field2 = tt.Field2
FROM MyRealTable rt
JOIN MyTempTable tt ON tt.Field1 = MyRealTable.Field1
WHERE ...
For an example of the XML you need to pass in, you can do:
SELECT TOP 1 *, 0 AS __ORDERBY FROM MyRealTable AS MyRealTable FOR XML AUTO, ROOT('ROOT')
For more info, see OPENXML, sp_xml_preparedocument and sp_xml_removedocument.