Build Where Clause Dynamically in Ado.net C# - c#

I will be taking in around 1000 records at a given time and I have to determine if they are existing records or new records.
If they are existing I have to update the records, if new then just insert them. I will not know if any of them will be existing or if they all will be existing.
I thought that it might be best to do one query to the database and try to find if any of them exist in the db and store them in memory and check that collection in memory and check that.
Originally I was told I 1 field would be enough to determine uniqueness. So I thought I could just do 1 big in clause against 1 field in the database but now I found out that is not the case and I need to use 3 fields to determine if the record is existing or now.
This is basically an and clause
select * from where columnA = "1" and ColumnB = "2" and ColumnC = "3"
How can I properly right this in C# ado.net?
I am guessing I going to need to have like some super where clause?
select * from where (columnA = "1" and ColumnB = "2" and ColumnC = "3") or (columnA = "4" and ColumnB = "5" and ColumnC = "6") or [....998 more conditional clauses)
I am open to better ideas if possible. I still think doing it in 1 shot is better than doing 1000 separate queries.

I can only help you to write query for your request
var recordCount = 1000;
var query = "SELECT * FROM TableName WHERE";
for (var i = 1; i < recordCount - 2; i += 3)
{
query += " (columnA = " + i + " and ColumnB = " + (i + 1) + " and ColumnC = " + (i + 2) + ") or ";
}

I feel kinda silly writing this answer, because I think that you should be able to put the pieces together a complete answer from other posts - but this is not an exact duplicate of either of the questions I have in mind.
There already are questions and answers in Stackoverflow dealing with this issue - however in my search I only found answers that are not thread safe, and most of them are using merge.
There are different questions and answers I can refer you to such as
my answer to Adding multiple parameterized variables to a database in c#
where you can see how to work with table valued parameters on c#, and to Aaron Bertrand's answer to Using a if condition in an insert SQL Server where you can see how to create a safe upsert - however I didn't find any answer that covers this completely - so here you go:
First you need to create a user defined table type in your database:
CERATE TYPE MyTableType AS TABLE
(
Column1 int NOT NULL,
Column2 int NOT NULL,
Column3 int NOT NULL,
-- rest of the columns in your table goes here
PRIMARY KEY (Column1, Column2, Column3)
)
Then, you create the stored procedure:
CREATE stp_UpsertMyTable
(
#MyTableType dbo.MyTableType readonly -- table valued parameters must be readonly
)
AS
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
BEGIN TRANSACTION;
UPDATE t
SET t.column4 = tvp.column4,
t.column5 = tvp.column5 -- and so on for all columns that are not part of the key
FROM dbo.MyTable AS t
INNER JOIN #MyTableType AS tvp
ON t.Column1 = tvp.Column1
AND t.Column2 = tvp.Column2
AND t.Column3 = tvp.Column3;
-- Note: <ColumnsList> should be replaced with the actual columns in the table
INSERT dbo.MyTable(<ColumnsList>)
SELECT <ColumnsList>
FROM #MyTableType AS tvp
WHERE NOT EXISTS
(
SELECT 1 FROM dbo.MyTable t
WHERE t.Column1 = tvp.Column1
AND t.Column2 = tvp.Column2
AND t.Column3 = tvp.Column3
);
COMMIT TRANSACTION;
GO
Then, the c# part is simple:
DataTable dt = new DataTable();
dt.Columns.Add("Column1", typeof(int));
dt.Columns.Add("Column2", typeof(int));
dt.Columns.Add("Column3", typeof(int));
dt.Columns.Add("Column4", typeof(string));
dt.Columns.Add("Column5", typeof(string));
// Fill your data table here
using (var con = new SqlConnection("ConnectionString"))
{
using(var cmd = new SqlCommand("stp_UpsertMyTable", con))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#MyTable", SqlDbType.Structured).Value = dt;
con.Open();
cmd.ExecuteNonQuery();
}
}
Now you have a complete and safe upsert using a table valued parameter with only one round trip between c# and sql server.

Related

Why does my SQL update for 20.000 records take over 5 minutes?

I have a piece of C# code, which updates two specific columns for ~1000x20 records in a database on the localhost. As I know (though I am really far from being a database expert), it should not take long, but it takes more than 5 minutes.
I tried SQL Transactions, with no luck. SqlBulkCopy seems a bit overkill, since it's a large table with dozens of columns, and I only have to update 1/2 column for a set of records, so I would like to keep it simple. Is there a better approach to improve efficiency?
The code itself:
public static bool UpdatePlayers(List<Match> matches)
{
using (var connection = new SqlConnection(Database.myConnectionString))
{
connection.Open();
SqlCommand cmd = connection.CreateCommand();
foreach (Match m in matches)
{
cmd.CommandText = "";
foreach (Player p in m.Players)
{
// Some player specific calculation, which takes almost no time.
p.Morale = SomeSpecificCalculationWhichMilisecond();
p.Condition = SomeSpecificCalculationWhichMilisecond();
cmd.CommandText += "UPDATE [Players] SET [Morale] = #morale, [Condition] = #condition WHERE [ID] = #id;";
cmd.Parameters.AddWithValue("#morale", p.Morale);
cmd.Parameters.AddWithValue("#condition", p.Condition);
cmd.Parameters.AddWithValue("#id", p.ID);
}
cmd.ExecuteNonQuery();
}
}
return true;
}
Updating 20,000 records one at a time is a slow process, so taking over 5 minutes is to be expected.
From your query, I would suggest putting the data into a temp table, then joining the temp table to the update. This way it only has to scan the table to update once, and update all values.
Note: it could still take a while to do the update if you have indexes on the fields you are updating and/or there is a large amount of data in the table.
Example update query:
UPDATE P
SET [Morale] = TT.[Morale], [Condition] = TT.[Condition]
FROM [Players] AS P
INNER JOIN #TempTable AS TT ON TT.[ID] = P.[ID];
Populating the temp table
How to get the data into the temp table is up to you. I suspect you could use SqlBulkCopy but you might have to put it into an actual table, then delete the table once you are done.
If possible, I recommend putting a Primary Key on the ID column in the temp table. This may speed up the update process by making it faster to find the related ID in the temp table.
Minor improvements;
use a string builder for the command text
ensure your parameter names are actually unique
clear your parameters for the next use
depending on how many players in each match, batch N commands together rather than 1 match.
Bigger improvement;
use a table value as a parameter and a merge sql statement. Which should look something like this (untested);
CREATE TYPE [MoraleUpdate] AS TABLE (
[Id] ...,
[Condition] ...,
[Morale] ...
)
GO
MERGE [dbo].[Players] AS [Target]
USING #Updates AS [Source]
ON [Target].[Id] = [Source].[Id]
WHEN MATCHED THEN
UPDATE SET SET [Morale] = [Source].[Morale],
[Condition] = [Source].[Condition]
DataTable dt = new DataTable();
dt.Columns.Add("Id", typeof(...));
dt.Columns.Add("Morale", typeof(...));
dt.Columns.Add("Condition", typeof(...));
foreach(...){
dt.Rows.Add(p.Id, p.Morale, p.Condition);
}
SqlParameter sqlParam = cmd.Parameters.AddWithValue("#Updates", dt);
sqlParam.SqlDbType = SqlDbType.Structured;
sqlParam.TypeName = "dbo.[MoraleUpdate]";
cmd.ExecuteNonQuery();
You could also implement a DbDatareader to stream the values to the server while you are calculating them.

SQL Server - Update table and return the Updated rows

I have a SQL Server database which has a lot of information inside.
I want to select top 50 rows in a single query (which I did, with no problem) but then I want to update a column from false to true, so next time I select I wont select the same, my code looks like this:
string Command = "UPDATE HubCommands SET [Alreadytaken] = 'true' FROM (SELECT TOP 50 [CommandId],[DeviceId],[Commandtext], [HashCommand],[UserId] FROM HubCommands) I WHERE [HubId] = '18353fe9-82fd-4ac2-a078-51c199d9072b'";
using (SqlConnection myConnection = new SqlConnection(SqlConnection))
{
using (SqlDataAdapter myDataAdapter = new SqlDataAdapter(Command, myConnection))
{
DataTable dtResult = new DataTable();
myDataAdapter.Fill(dtResult);
foreach (DataRow row in dtResult.Rows)
{
Guid CommandId, DeviceId, UserId;
Guid.TryParse(row["CommandId"].ToString(), out CommandId);
Guid.TryParse(row["DeviceId"].ToString(), out DeviceId);
Guid.TryParse(row["UserId"].ToString(), out UserId);
Console.WriteLine("CommandId" + CommandId);
}
}
}
This code does work, and it updates what I ask it to update, but I don't get nothing in the data table, its like it is always updating but not selecting.
If I do a normal select it does work and give information.
Does anyone have any idea how to update and get some data back, in a single query?
So your question is:
How can I update a table in SQL Server using C# and return the truly updated
rows as a DataTable ?
First You have multiple issues in your query.
You should use 1 and 0, not true or false. SQL-Server has a bit datatype and not a Boolean.
Second, this is how you should've constructed your query:
DECLARE #IDs TABLE
(
[CommandId] uniqueidentifier
);
INSERT INTO #IDs
SELECT [CommandId] FROM HubCommands
WHERE [HubId] = '18353fe9-82fd-4ac2-a078-51c199d9072b' AND [Alreadytaken] = 0;
UPDATE HubCommands
SET [Alreadytaken] = 1
WHERE CommandId IN
(
SELECT [CommandId] FROM #IDs
);
SELECT * FROM HubCommands
WHERE CommandId IN
(
SELECT [CommandId] FROM #IDs
);
Wrap all the above in a single string and use SqlDataReader. No need for an Adapter in you case (Since we're mixing commands unlike what the adapter usually does):
var sqlCommand = new SqlCommand(Command, myConnection);
SqlDataReader dataReader = sqlCommand.ExecuteReader();
DataTable dtResult = new DataTable();
dtResult.Load(dataReader);
I highly advise you to create a stored procedure accepting HubId as a parameter that does all the above work. It is neater and better for maintenance.

SQL Update one item at a time in order

I have this c# code that does an SQL Update that could be doing multiple updates at a time. Now the table I am updating has a column called SortOrder, so while I am doing these multiple updates, I would like to do the updates in order of the sortOrder column...is this even possible?
Here is my code:
public void PostScheduledTasks(List<CellModel> cells)
{
conn = new SqlConnection(connectionString);
cmd = new SqlCommand(
#"UPDATE ScheduleTasks_Copy
SET
ActualStart=#actualStart,
ActualFinish=#actualFinish,
ActualEndDate=#actualEndDate,
UserDate1=#userDateOne,
IsCompleted=#isCompleted
WHERE ScheduleTaskID = #scheduleTaskID");
cmd.Parameters.Add("#isCompleted", System.Data.SqlDbType.Bit);
cmd.Parameters.Add("#userDateOne", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#actualStart", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#actualFinish", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#actualEndDate", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#scheduleTaskID", System.Data.SqlDbType.Int);
cmd.Connection = conn;
conn.Open();
for (int i = 0; i < cells.Count; i++)
{
cmd.Parameters["#isCompleted"].Value = cmd.Parameters["#percentComplete"].Value = (cells[i].selected == true) ? 1 : 0;
cmd.Parameters["#userDateOne"].Value = !string.IsNullOrEmpty(cells[i].scheduledDate) ? cells[i].scheduledDate : (object)DBNull.Value;
cmd.Parameters["#actualStart"].Value = !string.IsNullOrEmpty(cells[i].actualDate) ? cells[i].actualDate : (object)DBNull.Value;
cmd.Parameters["#actualFinish"].Value = !string.IsNullOrEmpty(cells[i].finishedDate) ? cells[i].finishedDate : (object)DBNull.Value;
cmd.Parameters["#actualEndDate"].Value = !string.IsNullOrEmpty(cells[i].finishedDate) ? cells[i].finishedDate : (object)DBNull.Value;
cmd.Parameters["#scheduleTaskID"].Value = cells[i].scheduleTaskID;
cmd.ExecuteNonQuery();
}
conn.Close();
}
If "SortOrder" can be ascertained from the source object "cells", like other attributes:
Then the most efficient way would be to sort "cells" by SortOrder prior to iterating. The exact method to do so is beyond the scope of the question, since you have not told us what "cells" is, exactly (a list? an array? a custom object? a set?)
If SortOrder can only be ascertained by querying the database:
Then, not surprisingly, you'll need to query the database:
SELECT ScheduleTaskID, SortOrder FROM ScheduleTasks_Copy ORDER BY SortOrder
You iterate through that rowset, grabbing the ScheduleTaskID each time. For each ScheduleTaskID, iterate through "cells" until you find the matching task (cells[i].scheduleTaskID == TaskID), and THEN do the database update using the matching task from the source table.
Here is very rough code, I haven't written C# in a while:
using (connection)
{
SqlCommand command = new SqlCommand("SELECT ScheduleTaskID, SortOrder FROM ScheduleTasks_Copy ORDER BY SortOrder;", connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
while (reader.Read())
{
int taskid = reader.GetInt32(0);
for (int i = 0; i < cells.Count; i++)
{
if (cells[i].scheduleTaskID == taskid) {
cmd.Parameters["#isCompleted"].Value = cmd.Parameters["#percentComplete"].Value = (cells[i].selected == true) ? 1 : 0;
cmd.Parameters["#userDateOne"].Value = !string.IsNullOrEmpty(cells[i].scheduledDate) ? cells[i].scheduledDate : (object)DBNull.Value;
cmd.Parameters["#actualStart"].Value = !string.IsNullOrEmpty(cells[i].actualDate) ? cells[i].actualDate : (object)DBNull.Value;
cmd.Parameters["#actualFinish"].Value = !string.IsNullOrEmpty(cells[i].finishedDate) ? cells[i].finishedDate : (object)DBNull.Value;
cmd.Parameters["#actualEndDate"].Value = !string.IsNullOrEmpty(cells[i].finishedDate) ? cells[i].finishedDate : (object)DBNull.Value;
cmd.Parameters["#scheduleTaskID"].Value = cells[i].scheduleTaskID;
cmd.ExecuteNonQuery();
}
}
}
}
reader.Close();
}
Taking a step back and ignoring your actual question about running the updates concurrently, lets assume that your problem isn't "can I run a bunch of single updates concurrently?" and instead its "how do I update each of the items in my List<CellModel> cells as quickly/efficiently as possible?".
Obviously, running single updates sequentially doesn't work when cells.Count gets large, likely due to the fact that the cost of performing the update is dwarfed by the overheads incurred in performing multiple roundtrips across the network. It follows that running a bunch of single updates concurrently probably isn't the most efficient solution either.
Instead, lets move the whole cells list in one go. This means the server has everything it needs to process our request. If cells.Count is large, SqlBulkCopy (as suggested by Will) will be hard to beat. Alternatively, a Table-Valued Parameter could work fine for smaller sets.
Once the data is available to the server, all you need to do is perform a single update joining against the temp table/variable and you're done (MSDN has some good examples of this technique.) Do note, however, that if cells contains duplicate scheduleTaskID you may also have to perform a little extra pre-processing (ensuring that the highest sortOrder wins, etc.)
Example:
create table #temp(scheduleTaskID int, isCompleted bit, userDateOne datetime, actualStart datetime, actualFinish datetime, actualEndDate datetime)
--populate #temp via SqlBulkCopy, etc
update st
set st.ActualStart = t.actualStart
, st.ActualFinish = t.actualFinish
, st.ActualEndDate = t.actualEndDate
, st.UserDate1 = t.userDateOne
, st.IsCompleted = t.isCompleted
from dbo.ScheduleTasks_Copy st
inner join #temp t on st.ScheduleTaskID = t.scheduleTaskID
drop table #temp
The join syntax for updates isn't as well known as it probably should be - it becomes ridiculously useful once you get used to the slight awkwardness...
You would have to grab the values of the SortOrder column initially, and then iterate through them updating with SortOrder = X in the WHERE clause:
var dt = new DataTable();
Fill dt from query like: SELECT SortOrder FROM ScheduleTasks_Copy
var conn = new SqlConnection(connectionString);
foreach (DataRow dr in dt.Rows)
{
cmd = new SqlCommand(
#"UPDATE ScheduleTasks_Copy
SET
ActualStart=#actualStart,
ActualFinish=#actualFinish,
ActualEndDate=#actualEndDate,
UserDate1=#userDateOne,
IsCompleted=#isCompleted
WHERE ScheduleTaskID = #scheduleTaskID
AND SortOrder = #sortOrder");
cmd.Parameters.Add("#isCompleted", System.Data.SqlDbType.Bit);
cmd.Parameters.Add("#userDateOne", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#actualStart", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#actualFinish", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#actualEndDate", System.Data.SqlDbType.DateTime);
cmd.Parameters.Add("#scheduleTaskID", System.Data.SqlDbType.Int);
cmd.Parameters.Add("#sortOrder", dr["SortOrder"].ToString());
}
(I've not tested this code for syntax, but it gets across the general idea)
Though I guess it depends on the reason you want to do this, there seems to me a limited number of reasons that this would be required.
Does the CellModel has the sortOrder property? Because it should. And in this case you can add this line:
cells = cells.OrderBy(o => o.sortOrder); // add this -> order your cells first
for (int i = 0; i < cells.Count; i++) {
...
}
If you don't have the sortOrder property you should find another way to sort this collection.
Another solution would be to get the ScheduleTaskID's that you expect to update ordered by sortOrder using:
select ScheduleTaskID from ScheduleTasks_Copy order by SortOrder
Then you should go through all these items and if you find an item in cells having the same ScheduleTaskID then you should do the update.
I believe what you could do is upload your result into a temp table, then use a cursor server side to roll through and update all your rows in the specified order. This would probably be easier if you are able to use sql user defined types and stored procedures.
var conn = new SqlConnection(connectionString);
conn.Open();
SqlCommand cmd = new SqlCommand("create table ##Tasks(id int, isCompleted bit, userDateOne datetime, actualStart datetime, actualFinish datetime, actualEndDate datetime)", conn);
cmd.ExecuteNonQuery();
DataTable localTempTable = new DataTable("Tasks");
localTempTable.Columns.Add("id", typeof(int));
localTempTable.Columns.Add("isCompleted", typeof(bool));
localTempTable.Columns.Add("userDateOne", typeof(DateTime));
localTempTable.Columns.Add("actualStart", typeof(DateTime));
localTempTable.Columns.Add("actualFinish", typeof(DateTime));
localTempTable.Columns.Add("actualEndDate", typeof(DateTime));
for (int i = 0; i < cells.Count; i++)
{
var row = localTempTable.NewRow();
row["id"] = cells[i].scheduleTaskID;
row["isCompleted"] = (cells[i].selected == true);
row["userDateOne"] = !string.IsNullOrEmpty(cells[i].scheduledDate) ? cells[i].scheduledDate : (object)DBNull.Value;
row["actualStart"] = !string.IsNullOrEmpty(cells[i].actualDate) ? cells[i].actualDate : (object)DBNull.Value;
row["actualFinish"] = !string.IsNullOrEmpty(cells[i].finishedDate) ? cells[i].finishedDate : (object)DBNull.Value;
row["actualEndDate"] = !string.IsNullOrEmpty(cells[i].finishedDate) ? cells[i].finishedDate : (object)DBNull.Value;
}
localTempTable.AcceptChanges();
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection))
{
bulkCopy.DestinationTableName = "##Tasks";
bulkCopy.WriteToServer(localTempTable);
}
SqlCommand cmd = new SqlCommand(#"
declare orderedUpdate cursor
for
select tempTasks.* from ##tasks tempTasks
inner join ScheduleTasks tasks on tempTasks.id = tasks.ScheduleTaskID
order by tasks.sortOrder;
declare #id int, #isCompleted bit, #actualStart datetime, #actualFinish datetime, #actualEndDate datetime, #userDateOne datetime;
open orderedUpdate;
fetch next from orderedUpdate INTO #id, #isCompleted, #userDateOne, #actualStart, #actualFinish, #actualEndDate
while ##fetch_status = 0
begin
UPDATE ScheduleTasks_Copy
SET
ActualStart=#actualStart,
ActualFinish=#actualFinish,
ActualEndDate=#actualEndDate,
UserDate1=#userDateOne,
IsCompleted=#isCompleted
WHERE ScheduleTaskID = #id;
end
close orderedUpdate
deallocate orderedUpdate", conn);
cmd.ExecuteNonQuery();
While it is still unclear as to why there is a desire to update the set of records in any particular order (is there a trigger that causes some down-stream effect that would benefit from being processed in the order of the sortOrder field?), it should be pointed out that doing all of the updates in a single UPDATE statement renders the ordering irrelevant. It has been pointed out by a few others already that taking a set-based approach will be much faster (and it certainly will be), but being done in a single UPDATE means that transactionally there was no order (on a practical level).
Making this set-based via SqlBulkCopy has already been mentioned, but my preference is for using Table-Valued Parameters (TVPs) as they allow for streaming the Cells collection straight to the stored procedure that will do the UPDATE (well, technically via [tempdb], but close enough). Conversely, using SqlBulkCopy means needing to make a copy of the existing Cells collection in the form of a `DataTable.
Below is both the T-SQL and C# code for converting the existing process to using a TVP. It should be noted that if there is still some reason for doing this in a particular order, the only thing to change would be how the table variable in the stored procedure is handled.
T-SQL code
-- First: You need a User-Defined Table Type
CREATE TYPE dbo.ScheduleTasksImport AS TABLE
(
ScheduleTaskID INT NOT NULL, -- optionally mark this field as PRIMARY KEY
IsCompleted BIT NOT NULL,
ActualStart DATETIME NULL,
ActualFinish DATETIME NULL,
ActualEndDate DATETIME NULL,
UserDate1 DATETIME NULL
);
GO
GRANT EXECUTE ON TYPE::[dbo].[ScheduleTasksImport] TO [user_or_role];
GO
-- Second: Use the UDTT as an input param to an import proc.
-- Hence "Tabled-Valued Parameter" (TVP)
CREATE PROCEDURE dbo.ImportData (
#ImportTable dbo.ScheduleTasksImport READONLY
)
AS
SET NOCOUNT ON;
UPDATE stc
SET stc.ActualStart = imp.ActualStart,
stc.ActualFinish = imp.ActualFinish,
stc.ActualEndDate = imp.ActualEndDate,
stc.UserDate1 = imp.UserDate1,
stc.IsCompleted = imp.IsCompleted
FROM ScheduleTasks_Copy stc
INNER JOIN #ImportTable imp
ON imp.ScheduleTaskID = stc.ScheduleTaskID
GO
GRANT EXECUTE ON dbo.ImportData TO [user_or_role];
C# code
Part 1: Define the method that will take the collection and return it as IEnumerable<SqlDataRecord>
using System.Collections;
using System.Data;
using System.Data.SqlClient;
using System.IO;
using Microsoft.SqlServer.Server;
private static IEnumerable<SqlDataRecord> SendRows(List<CellModel> Cells)
{
SqlMetaData[] _TvpSchema = new SqlMetaData[] {
new SqlMetaData("ScheduleTaskID", SqlDbType.Int),
new SqlMetaData("IsCompleted", SqlDbType.Bit),
new SqlMetaData("ActualStart", SqlDbType.DateTime),
new SqlMetaData("ActualFinish", SqlDbType.DateTime),
new SqlMetaData("ActualEndDate", SqlDbType.DateTime),
new SqlMetaData("UserDate1", SqlDbType.DateTime)
};
SqlDataRecord _DataRecord = new SqlDataRecord(_TvpSchema);
// read a row, send a row
for (int _Index = 0; _Index < Cells.Count; _Index++)
{
// Unlike BCP and BULK INSERT, you have the option here to create an
// object, do manipulation(s) / validation(s) on the object, then pass
// the object to the DB or discard via "continue" if invalid.
_DataRecord.SetInt32(0, Cells[_Index].scheduleTaskID);
_DataRecord.SetBoolean(1, Cells[_Index].selected); // IsCompleted
_DataRecord.SetDatetime(2, Cells[_Index].actualDate); // ActualStart
_DataRecord.SetDatetime(3, Cells[_Index].finishedDate); // ActualFinish
_DataRecord.SetDatetime(4, Cells[_Index].finishedDate); // ActualEndDate
_DataRecord.SetDatetime(5, Cells[_Index].scheduledDate); // UserDate1
yield return _DataRecord;
}
}
Part 2: Replace your current PostScheduledTasks method with the following that just executes the ImportData stored procedure. When the stored procedure is executed, it will ask for the value of the #ImportTable input parameter which will start the process of streaming the records across.
public static void PostScheduledTasks(List<CellModel> Cells)
{
SqlConnection _Connection = new SqlConnection(connectionString);
SqlCommand _Command = new SqlCommand("ImportData", _Connection);
_Command.CommandType = CommandType.StoredProcedure;
SqlParameter _TVParam = new SqlParameter();
_TVParam.ParameterName = "#ImportTable";
_TVParam.SqlDbType = SqlDbType.Structured;
_TVParam.Value = SendRows(Cells); // method return value is streamed data
_Command.Parameters.Add(_TVParam);
try
{
_Connection.Open();
// Send the data and process the UPDATE
_Command.ExecuteNonQuery();
}
finally
{
_Connection.Close();
}
return;
}
Please note that the two C# listing really go in the same file but were broken apart here for readability. If they are put into separate .cs files, then the using statements will need to be copied over for the second listing.
Ignoring the obvious question about why you would want to do this, you can try this hack:
Insert the data into a temp table (using SqlBulkCopy), then use a MERGE with an ORDER BY clause in the source. In this example I'm using ScheduleTasks_Copy as the "temp" table and ScheduleTasks as the destination.
MERGE ScheduleTasks AS T
USING (SELECT TOP 99999999 C.*
FROM ScheduleTasks_Copy C
JOIN ScheduleTasks S
ON C.ID = S.ID
ORDER BY S.SortOrder ASC) AS S
ON S.ID = T.ID
WHEN MATCHED THEN
UPDATE SET T.ActualStart = S.ActualStart,
T.ActualFinish = S.ActualFinish,
T.ActualEndDate = S.ActualEndDate,
T.UserDate1 = S.UserDate1,
T.IsCompleted = S.IsCompleted,
T.UpdatedOn = dbo.GetDateValue();
Don't worry about the silly GetDateValue function and UpdatedOn column. I just included them so that you could see in which order the updates were performed. If you want to update the records in reverse order, just change the sort order to DESC.
http://sqlfiddle.com/#!6/dbc5f/16
Personally I suggest avoid things like this and redesigning your solution so that it avoids the issue altogether.

SQLBulkCopy Don't Copy Primary Keys

What is the best way to deal with the primary key violation errors when using SQLBulkCopy
Violation of PRIMARY KEY constraint 'email_k__'. Cannot insert duplicate key in object 'lntmuser.email'.
(i.e. if the row already exists in the destination table) ?
Is there a way to skip inserting duplicate rows or would this have to be checked and dealt with before hand ?
Here is the code I am currently using:
var conPro = tx_ProConStr.Text;
var conArc = tx_ArcConStr.Text;
var con = new SqlConnection {ConnectionString = conPro};
var cmd = new SqlCommand("SELECT * FROM dbo.email", con);
con.Open();
var rdr = cmd.ExecuteReader();
var sbc = new SqlBulkCopy(conArc) {DestinationTableName = "dbo.email"};
sbc.WriteToServer(rdr);
sbc.Close();
rdr.Close();
con.Close();
I usually end up performing a Bulk Copy operation to a temporary table, and then copy data from it to the target table using regular SQL. This allows me to perform 'bulk updates', as well as take care of special situations like this (although I haven't encountered this specific need).
There's a performance hit compared to straight bulk copy, but it's still a lot faster than performing INSERTs.
You could adjust the source query to exclude duplicates. For example:
select distinct * from dbo.email
Or to filter for the first col1 per pkcol:
select *
from (
select row_number() over (parition by pkcol order by col1) as rn
from dbo.email
) as SubQueryAlias
where rn = 1

Oracle/c#: How do i use bind variables with select statements to return multiple records?

I have a question regarding Oracle bind variables and select statements.
What I would like to achieve is do a select on a number different values for the primary key. I would like to pass these values via an array using bind values.
select * from tb_customers where cust_id = :1
int[] cust_id = { 11, 23, 31, 44 , 51 };
I then bind a DataReader to get the values into a table.
The problem is that the resulting table only contains a single record (for cust_id=51). Thus it seems that each statement is executed independently (as it should), but I would like the results to be available as a collective (single table).
A workaround is to create a temporary table, insert all the values of cust_id and then do a join against tb_customers. The problem with this approach is that I would require temporary tables for every different type of primary key, as I would like to use this against a number of tables (some even have combined primary keys).
Is there anything I am missing?
Not asking the question as to why you would want to do this to begin with. Shouldn't the sql statement be something like
select * from tb_customers where cust_id = 11 or 23 or ...
Edit:
I am limited in Oracle but when I look at the documentation I think that you might have to do something like this:
variable i number
exec :i := 11
select * from tb_customers where cust_id = :i
This would allow you to take advantage of binding. You will have to add each record return to your own collection since it will still only return one at a time.
I know this was asked a while ago but not a brilliant answer.
I would do something like this - please excuse the crude psudo code
string bindList = "";
for(int ii=0;ii<cust_id.count;++ii)
{
if(ii == 0)
{
bindList += ":" + ii;
}
else
{
bindList += ",:" + ii;
}
OracleParameter param = new OracleParameter();
param.dbType = types.int;
param.value = cust_id[ii];
command.Parameters.Add(param);
}
query = "select * from tb_customers where cust_id in(" + bindList + ")";
So then query ends up having in(:1,:2,:3,etc) and each of these are bound separately.
There is also a similar question here: OracleParameter and IN Clause

Categories