Can i ignore the duplicate records of a data already present in sql database from a datatable which i am passing to SqlBulkCopy. If Yes then How and also explain me if No and other option.
No, that's not built-in. You need to clean the data on the client or insert into a staging table first.
As previous poster said, this is not built in. I achieve similar using the following:
SQL Stored Procedure that accepts a TableValuedParameter with the data you require.
In the stored proc, I then INSERT all records into a temp table. Once you have it there, you can use SQLs MERGE statement in your stored proc to insert data where it doesn't already exist.
So, let us assume that our data is simply people's names stored in a table people. We hold only an ID and a name. I also assume this table is called 'people'.
Here's how I create my Table Valued Parameter type (created in SQL Server)
CREATE TYPE udt_person AS TABLE(
[id] [INT] NOT NULL,
[name] [nvarchar(50)] NULL
)
GO
I now create the stored procedure:
CREATE PROCEDURE SaveNewPeople #pPeople udt_Person
AS
BEGIN
-- Create Temp table
CREATE TABLE #tmpPeople (id INT, name VARCHAR 50)
-- We will stage all data passed in into temp table
INSERT INTO #tmpPeople
SELECT id, name FROM #pPeople
-- NB: you will need to think about locking strategy a bit here
MERGE people AS p
USING #tmpPeople AS t
ON p.id = t.id
WHEN NOT MATCHED BY TARGET THEN
-- We want to insert new person
INSERT (id, name) VALUES (t.id, t.name)
WHEN MATCHED THEN
-- you may not need this, assume updating name for example
UPDATE SET p.name = t.name
END
Now we have the SQL in place.
Let us create the bulk of data in C#:
DataTable ppl = new DataTable();
ppl.Columns.Add("id", typeof(int));
ppl.Columns.Add("name", typeof(string));
// table is created, let's add some people
var bob = ppl.NewRow();
bob["id"] = 1;
bob["name"] = "Bob";
ppl.Rows.Add(bob);
var jim = ppl.NewRow();
jim["id"] = 2;
jim["name"] = "Jim";
ppl.Rows.Add(jim);
// that's enough people for now, let's call the stored procedure
using(var conn = new SqlConnection("YouConnStringHere"))
{
using(var cmd = new SqlCommand("SaveNewPeople", conn))
{
cmd.CommandType = CommandType.StoredProcedure;
var tvp = new SqlParameter
{
ParameterName = "#pPeople",
SqlDbType = SqlDbType.Structured,
Value = ppl,
TypeName = "udt_person"
}
cmd.Parameters.Add(tvp);
conn.Open();
cmd.ExecuteNonQuery();
}
}
Hopefully this gives you the idea. If you then modified the C# datatable, you should see rows inserted, updated or ignored.
Good luck.
Another way to do it is to create a database trigger to replace the inserts initiated by SqlBulkCopy. The performance will be impeded, depending on, among other things, the size of the batch, but it works nonetheless.
CREATE TABLE [dbo].[TempTable] (
[Id] INT IDENTITY PRIMARY KEY,
[Val] NVARCHAR(20)
)
GO
CREATE OR ALTER TRIGGER [IgnoreDuplicates] ON [dbo].[TempTable]
INSTEAD OF INSERT
AS
BEGIN
SET NOCOUNT ON
INSERT INTO [dbo].[TempTable]([Val])
SELECT [Val] FROM [INSERTED] WHERE [Val] NOT IN (
SELECT [Val] FROM [dbo].[TempTable]
)
END
GO
Related
I just came across a situation where i had to insert multiple records into the database. The records i had were in generic list and now i have found two ways to insert multiple records.
i just do it as usual using ExecuteNonQuery and i use foreach loop to insert every records one by one(i don't think that this is the best way to do it since this way makes multiple trips to the database to insert the record).
using (SqlCommand cmd = new SqlCommand()
{
using (SqlConnection conn = new SqlConnection()
{
cmd.Connection = conn;
foreach (var entry in entries)
{
cmd.CommandText = "INSERT INTO Employees(id, name) VALUES
(#id,#name);";
cmd.Parameters.AddWithValue("#id", entry.Id);
cmd.Parameters.AddWithValue("#name", entry.Name);
cmd.ExecuteNonQuery();
}
}
}
i tried using SqlBulkCopy class i converted the Generic List to DataTable and used SqlBulkCopy.
using (SqlConnection con = new SqlConnection(consString))
{
using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy(con))
{
sqlBulkCopy.DestinationTableName = "Employees";
con.Open();
sqlBulkCopy.WriteToServer(dt);
con.Close();
}
}
Both the methods do the intended work. But i want to know which is the best way. If there are any other better way to do this i am open to suggesstions.
Use Table-Valued Parameters (TVPs) ,which were introduced in SQL Server 2008. Before SQL Server 2008, it was not possible to pass a table variable in stored procedure as a parameter, after SQL Server now we can pass Table-Valued Parameter to send multiple rows of data to a stored procedure or a function without creating a temporary table or passing so many parameters. have a look at below example
CREATE TYPE UT_Employee AS TABLE
(
Emp_Id int NOT NULL,
EmployeeName nvarchar(MAX),
EmpSalary varchar(50),
StateId varchar(50),
CityId varchar(50)
)
CREATE PROCEDURE USP_Insert_Employee_Info
(
#Employee_Details [UT_Employee] READONLY
)
AS
BEGIN
INSERT INTO dbo.Employee
(
Emp_Id,
EmployeeName,
EmpSalary,
StateId,
CityId
)
SELECT * FROM #Employee_Details
END
Inserting multiple values into stored procedure as below
DECLARE #Employee_Details AS UT_Employee
INSERT INTO #Employee_Details
SELECT 1001,'Abcd',10000,101,21 UNION ALL
SELECT 1002,'dggg',20000,121,15 UNION ALL
SELECT 1003,'esse',22222,122,35 UNION ALL
SELECT 1004,'uyyy',44333,121,32 UNION ALL
SELECT 1005,'dghd',13233,656,87
EXEC dbo.USP_Insert_Employee_Info #Employee_Details = #Employee_Details
I'd normally use a Table-Valued Parameter for this. The explanation of how to do that is a bit long to put here, but check out Table-Valued Parameters on Microsoft Docs for how to do this.
I want to use a SELECT statement into a table based on multiple values from ListBox and get multiple IDs and then run an INSERT statement and store the multiple IDs into a different table from INSERT statement.
My code below is not working as I am getting "NULL" in single row instead of multiple IDs in multiple rows.
I am using a stored procedure for all the SQL statements.
Please see my code below:
Code-behind of my ASPX web page:
string listboxvalue = "";
foreach (ListItem item in listbox.Items)
{
if (item.Selected)
{
listboxvalue += item.Text + ',';
}
}
listboxvalue = listboxvalue.Substring(0, listboxvalue.Length - 1);
cmd.Parameters.AddWithValue("spselectvalue", listboxvalue);
Stored procedure:
#spselectvalue nvarchar(MAX),
// Select multiple Ids based on multiple items from list box
DECLARE #Dis TABLE (DisID int)
INSERT INTO #Dis
SELECT DId
FROM [table name]
WHERE [COLUMN] IN ('+#spselectvalue +')
EXEC sp_executesql #Dis
// Insert multiple Ids (from above select statement) into different table
INSERT INTO [dbo].[DifferentTable] ([SelectedIds])
VALUES
(
(SELECT DisID from #Dis)
)
Seems like your problem is that you don't know how to get a list of ID's into a stored procedure for processing. Hopefully this example will help. If you do it this way, you won't have to mess with comma-delimited strings or dynamic SQL.
First, define a SQL type that can contain your list of IDs:
CREATE TYPE dbo.MyList
AS TABLE
(
ID VarChar(50)
);
Then, write a stored procedure that accepts this type as its input parameter:
CREATE PROCEDURE dbo.InsertAList
#List AS dbo.MyList READONLY
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO [dbo].[DifferentTable]
SELECT ID FROM #List
END
Now to bind your data to the stored procedure from the c# end. To bind it, you have to store the data in a DataTable first, which is relatively easy:
var table = new DataTable();
table.Columns.Add("ID", typeof(string));
foreach (ListItem item in listBox.Items.Where(i => i.Selected))
{
table.Rows.Add(item.Text);
}
Then submit the table to the stored procedure like so:
var cmd = new SqlCommand()
{
CommandType = CommandType.StoredProcedure,
CommandText = "InsertAList",
Connection = myConnection
};
cmd.Parameters.Add(new SqlParameter("#List", SqlDbType.Structured)
{
TypeName = "dbo.MyList",
Value = table
});
cmd.ExecuteNonQuery();
That doesn't seem like a valid (or logical) SQL. The exec sp_executesql #Dis shouldn't be there or #Dis shouldn't be a table.
When using sp_executesql the parameter should be a string containing the statement to execute (documentation).
Detailed answer can be found here.
Also: the INSERT INTO #Dis ... line won't work as you expect when written that way, for the reasons mentioned in the linked question (and Erland Sommarskog blog mentioned there).
Selected values considered as numeric.
DECLARE #Sqltext nvarchar(max) =
'INSERT INTO [dbo].[DifferentTable] ([SelectedIds])
SELECT DId
FROM [table name]
WHERE [COLUMN] IN (' + #spselectvalue + ')'
EXEC sp_executesql #Sqltext
I have a table SupplierMaster in a SQL Server database with a column SUPPLIERNAME.
I want to edit saved supplier name using stored procedure with below query
ALTER PROCEDURE [dbo].[usp_SupplierMasterUpdateDetails]
(
#SUPPLIERNAME NVARCHAR(50)
)
AS
BEGIN
UPDATE [dbo].[SupplierMaster]
SET [SUPPLIERNAME] = #SUPPLIERNAME
WHERE [SUPPLIERNAME] = #SUPPLIERNAME
END
and I run the BELOW code through "UPDATE BUTTON" to update the data.
string connString = ConfigurationManager.ConnectionStrings["dbx"].ConnectionString;
using (SqlConnection conn = new SqlConnection(connString))
{
using (SqlCommand cmd = new SqlCommand("usp_SupplierMasterUpdateDetails", conn))
{
cmd.CommandType = CommandType.StoredProcedure;
// Parameter
cmd.Parameters.AddWithValue("SUPPLIERNAME", AddSupplierTextBox.Text);
// Open Connection
conn.Open();
// ExecuteReader (Select Statement)
// ExecuteScalar (Select Statement)
// ExecuteNonQuery (Insert, Update or Delete)
cmd.ExecuteNonQuery();
MessageBox.Show("SUCCESSFULLY UPDATED", "Successful", MessageBoxButtons.OK, MessageBoxIcon.Information);
}
}
But its not updating the selected data.
Please advice and assist me to correct the code for proper work.
You have multiple issues there.
First you need to fix your update query just as Thomas Levesque suggested.
a SQL Server table needs a primary key to be able to uniquely identify a record, for updates for example.
The easiest thing you could do is set that primary key to be identity of type int and make it self generating. Your supplier table could look like this :
SupplierID int, Primary Key, identity
SupplierName nvarchar(100)
Now, when you do an update, you would do it like this:
Update SupplierMaster
Set SupplierName = #supplierName
Where SupplierID = #suplierID
Such a SQL statement will return an int value. This return value will tell you how many SQL rows this update statement has changed. If it says 0 then it means that the SQL statement could not find that id you passed through and nothing changed. If it says 1, then the record was found and updated, if you get more than 1 you have an issue with the SQL statement and multiple rows were updated.
In your code check for this return value and that's how you determine if your update statement was successful or not.
I created a user-defined table type in SQL Server:
CREATE TYPE dbo.TestType AS TABLE
(
ColumnA int,
ColumnB nvarchar(500)
)
And I'm using a stored procedure to insert records into the database:
create procedure [dbo].[sp_Test_CustomType]
#testing TestType READONLY
as
insert into [dbo].[myTable]
select ColumnA, ColumnB
from #testing
And I would like to use EF to execute this stored procedure, but here's the problem: how can I pass a user defined table to the stored procedure?
I tried adding the stored procedure to the model, but I'm unable to find the desired stored procedure in the updated context.
What I'm trying to do is to execute a bulk insert to a table, here's the method that I'm currently using:
List<items> itemToInsertToDB = //fetchItems;
foreach(items i in itemToInsertToDB)
{
context.sp_InsertToTable(i.ColumnA, i.ColumnB)
}
Currently, I use a foreach loop to loop through the list to insert item to DB, but if the list have a lot of items, then there will be a performance issue, so, I'm thinking of passing a list to the stored procedure and do the insert inside.
So how to solve this problem? or are there any better ways to do this?
Lets say you want to send a table with a single column of GUIDs.
First we need to create a structure using SqlMetaData which represents the schema of the table (columns).
The below code demonstrates one column named "Id" of the GUID is the SQL stored procedure parameter table type
var tableSchema = new List<SqlMetaData>(1)
{
new SqlMetaData("Id", SqlDbType.UniqueIdentifier)
}.ToArray();
Next you create a list of records that match the schema using SqlDataRecord.
The below code demonstrates how to add the items inside a list using the above created schema. Create a new SqlDataRecord for each of the items in the list. Replace SetGuid with the corresponding type and Replace Guid.NewGuid() as the corresponding value.
Repeat new SqlDataRecord for each item and add them to a List
var tableRow = new SqlDataRecord(tableSchema);
tableRow.SetGuid(0, Guid.NewGuid());
var table = new List<SqlDataRecord>(1)
{
tableRow
};
Then create the SqlParameter:
var parameter = new SqlParameter();
parameter.SqlDbType = SqlDbType.Structured;
parameter.ParameterName = "#UserIds"; //#UserIds is the stored procedure parameter name
parameter.TypeName = "{Your stored procedure type name}"
parameter.Value = table;
var parameters = new SqlParameter[1]
{
parameter
};
Then simply call the stored procedure by using the Database.SqlQuery.
IEnumerable<ReturnType> result;
using (var myContext = new DbContext())
{
result = myContext.Database.SqlQuery<User>("GetUsers #UserIds", parameters)
.ToList(); // calls the stored procedure
// ToListAsync(); // Async
{
In SQL Server, create your User-Defined Table Type (I suffix them with TTV, Table Typed Value):
CREATE TYPE [dbo].[UniqueidentifiersTTV] AS TABLE(
[Id] [uniqueidentifier] NOT NULL
)
GO
Then specify the type as a parameter (don't forget, Table Type Values have to be readonly!):
CREATE PROCEDURE [dbo].[GetUsers] (
#UserIds [UniqueidentifiersTTV] READONLY
) AS
BEGIN
SET NOCOUNT ON
SELECT u.* -- Just an example :P
FROM [dbo].[Users] u
INNER JOIN #UserIds ids On u.Id = ids.Id
END
I suggest you not using Stored Procedure to insert bulk data, but just rely to Entity Framework insert mechanism.
List<items> itemToInsertToDB = //fetchItems;
foreach(items i in itemToInsertToDB)
{
TestType t = new TestType() { ColumnA = i.ColumnA, ColumnB = i.ColumnB };
context.TestTypes.Add(t);
}
context.SaveChanges();
Entity framework will smartly perform those insertion in single transaction and (usually) in single query execution, which will almost equal to executing stored procedure. This is better rather than relying on stored procedure just to insert bulk of data.
When a user deletes a row in the database, I want to archive it into a separate table in the database rather than delete it or flag it in the current table. I figure I would need to do something like in this link:
How to copy a row from one SQL Server table to another
The thing is, the archive table has 1 extra column in it that does not match the original table (ArchiveTimeStamp). This ArchiveTimeStamp does not exist in the original table, instead I would use something like
archiveComm.Parameters.AddWithValue("ArchiveTimeStamp", Date.Time.Now);
This is what I have so far:
SqlCommand archiveComm = new SqlCommand("INSERT INTO Archive_Table SELECT * FROM Table WHERE RowID = #rowID", conn);
Is there a way for me to modify the SqlCommand to add another param that doesn't exist in the original Table?
Why not just handle this on the back end? You can create a trigger on the original table to insert into another table after every delete?
Your trigger will look like this:
CREATE TRIGGER onOriginalTableDelete
ON originalTable
FOR DELETE
AS
INSERT INTO anotherTable
SELECT * FROM deleted;
When a record is deleted on the original table, it will insert the deleted record into the other table. You might want to read on using the deleted table here.
Check this SQL Fiddle. Since you're inserting the timestamp in another column, you can just add this on the INSERT INTO SELECT statement:
INSERT INTO OtherTable
SELECT *, CURRENT_TIMESTAMP FROM MainTable;
This could be the query for your trigger:
CREATE TRIGGER onOriginalTableDelete
ON originalTable
FOR DELETE
AS
INSERT INTO anotherTable
SELECT *, CURRENT_TIMESTAMP FROM deleted;
Good question. I'd suggest (as Gian has also suggested) moving the logic you require to backup the deleted row into a trigger that gets fired on delete.
Triggers are events in a database associated to a table which get fired upon an action occurring i.e. insert / update / delete.
So in your scenario, if you create an ON DELETE trigger in the source table, it will get fired when a delete occurs. The SQL contained within the trigger can specify what to do with the deleted data, which in your scenario will be: insert the deleted info into the archive table with a timestamp.
So if you have:
Source_Table:
Col_1
Col_2
Col_3
Archive_Table:
Col_1
Col_2
Col_3
Time_Stamp
You'll need to create a FOR DELETE trigger against Source_Table (something like this):
CREATE TRIGGER SourceDeletedTrigger
ON database.dbo.Source_Table
FOR DELETE
AS
INSERT INTO Archive_Table(Col_1, Col_2, Col_3, Time_Stamp)
SELECT
DELETED.Col_1,
DELETED.Col_2,
DELETED.Col_3,
GETDATE()
FROM DELETED
GO
The above is some rough SQL which may contain a couple of syntax errors but the guts of the idea is conveyed.
You will have to use to explicit column list and values form of the INSERT statement:
INSERT INTO Archive_Table (
Col1
,Col2
,Col3 )
SELECT
Col1
,Col2
,Col3
FROM
Table
WHERE
Row_ID = #Row_ID
See Insert into ... values ( SELECT ... FROM ... )
I think you have to specify the columns with something like this
INSERT INTO tab1
(col1, col2)
SELECT col1, col2
FROM tab2
WHERE RowID = #rowID"
You need to specify the columns name in that case:
archiveComm.Parameters.AddWithValue("ArchiveTimeStamp", Date.Time.Now);
string SQL = "INSERT INTO Archive_Table (Col1,Col2,ArchiveTimeStamp) " & _
"SELECT Col1,Col2,#ArchiveTimeStamp FROM Table WHERE RowID = #rowID"
SqlCommand archiveComm = new SqlCommand(SQL, conn);
Here is my suggestion, you are forced to supply the column names or it won't let you run the query, however I understand you would prefer a generic solution that worked for any table so I suggest building the insert SQL dynamically, cache it on your application, and then just execute it with your extra archive column. Here is a c# example:
public class ArchiveTableRow
{
private static readonly IDictionary<string, string> _cachedInsertStatements = new Dictionary<string, string>();
public void Archive(string tableName, string rowId)
{
if (_cachedInsertStatements.ContainsKey(tableName) == false)
{
BuildInsertStatement(tableName);
}
var insertQuery = _cachedInsertStatements[tableName];
// ...
SqlCommand archiveComm = new SqlCommand(insertQuery, conn);
// ...
archiveComm.Parameters.AddWithValue("ArchiveTimeStamp", Date.Time.Now);
// ...
}
private void BuildInsertStatement(string tableName)
{
// Get the columns names:
var getColumnNamesQuery = #"
SELECT Table_Schema, Column_Name
FROM Information_Schema.Columns
WHERE Table_Name = '" + tableName + "'
Order By Ordinal_Position";
// Execute the query
SqlCommand archiveComm = new SqlCommand(getColumnNamesQuery, conn);
// Loop and build query and add your archive in the end
// Add to dictionary
}
}
You would use it with something like:
var archiveRow = new ArchiveTableRow();
archiveRow.Archive("TableName", rowId);