We Keep Coding

Dapper to insert multiple rows into two tables using stored procedure

I am using Dapper (https://github.com/StackExchange/Dapper) in asp.net core myweb api project. I have a requirement to create a master record and insert a set of rows into a child table.
Ex. OrdersMaster table
create an order id in this table
OrderDetails table
ordermasterid
order items
How can I do this using Dapper? Please share some code snippets if possible.

I would implement the insert operations as a transactional stored procedure, and then call that from your .NET application.
You may need a table-valued type to pass in a list of data, like this:
CREATE TYPE List_Of_Items AS TABLE (
ItemID INT NOT NULL,
Quantity INT NOT NULL
)
The procedure might look like this
CREATE PROC Insert_Order_With_Details (
#CustomerId INT,
#Items List_Of_Items
) AS
BEGIN
BEGIN TRANSACTION
INSERT INTO OrdersMaster (CustomerId) VALUES #CustomerId
DECLARE #OrderID INT
SET #OrderID = SCOPE_IDENTITY() --last assigned id
INSERT INTO OrderDetails (OrderId, CustomerId, ItemId, Quantity)
SELECT #OrderID, #CustomerID, ItemID, Quantity
FROM #Items
COMMIT
END
Then in C#, I would suggest creating methods for creating your TVP. It is not as simple as you might like. This requires the using Microsoft.SqlServer.Server and using Dapper.Tvp.
//This is a shell to create any kind of TVP
private static void AddTableCore<T>(
this DynamicParametersTvp dp,
string tvpTypeName,
Func<T, SqlDataRecord> valueProjection,
IEnumerable<T> values,
string parameterTableName)
{
var tvp = values
.Select(valueProjection)
.ToList();
//If you pass a TVP with 0 rows to SQL server it will error, you must pass null instead.
if (!tvp.Any()) tvp = null;
dp.Add(new TableValueParameter(parameterTableName, tvpTypeName, tvp));
}
//This will create your specific Items TVP
public static void AddItemsTable(this DynamicParametersTvp dp, IEnumerable<Item> items, string parameterTableName = "Items")
{
var columns = new[]
{
new SqlMetaData("ItemID", SqlDbType.Int)
new SqlMetaData("Quantity", SqlDbType.Int)
};
var projection = new Func<Item, SqlDataRecord>(item =>
{
var record = new SqlDataRecord(columns);
record.SetInt32(0, item.Id);
record.SetInt32(1, item.Quantity);
return record;
});
AddTableCore(dp, "Items", projection, items, parameterTableName);
}
and then where you need to query you might do:
using (var cn = new SqlConnection(myConnectionString))
{
var p = new DynampicParametersTvp(new {
CustomerId = myCustomerId
});
p.AddItemsTable(items);
cn.Execute("Insert_Order_With_Details", p, commandType: CommandType.StoredProcedure);
}
The commandType argument is super important. It defaults to plain SQL text and will error if you send the name of a proc.
If you want to put in multiple orders at once, you'll need to use table-valued parameters and the Dapper.Tvp package.
See this SO question Using Dapper.TVP TableValueParameter with other parameters as well as this documentation on TVP's from Microsoft https://learn.microsoft.com/en-us/sql/relational-databases/tables/use-table-valued-parameters-database-engine. I don't think all SQL vendors support TVPs.

Using stored procedure as mentioned in other answer is good solution. My answer implements the same without Stored Procedure.
You have to use transaction. This way, either all the changes will be committed or rolled back. Code below assumes you are using Identity as Primary Key. Please refer this question for discussion about ##IDENTITY that I have used in below code.
Though code is not complete, I have put detailed comments to explain steps.
using (var connection = new SqlCeConnection("connection_string"))
{
connection.Open();
//Begin the transaction
using (var transaction = connection.BeginTransaction())
{
//Create and fill-up master table data
var paramMaster = new DynamicParameters();
paramMaster.Add("#XXX", ...);
....
//Insert record in master table. Pass transaction parameter to Dapper.
var affectedRows = connection.Execute("insert into OrdersMaster....", paramMaster, transaction: transaction);
//Get the Id newly created for master table record.
//If this is not an Identity, use different method here
newId = Convert.ToInt64(connection.ExecuteScalar<object>("SELECT ##IDENTITY", null, transaction: transaction));
//Create and fill-up detail table data
//Use suitable loop as you want to insert multiple records.
//for(......)
foreach(OrderItem item in orderItems)
{
var paramDetails = new DynamicParameters();
paramDetails.Add("#OrderMasterId", newId);
paramDetails.Add("#YYY", ...);
....
//Insert record in detail table. Pass transaction parameter to Dapper.
var affectedRows = connection.Execute("insert into OrderDetails....", paramDetails, transaction: transaction);
}
//Commit transaction
transaction.Commit();
}
}

Cannot save a stored procedure into an array

I'm using asp.net to process a SQL query that returns a column from some table. Normally what I'd do is set a variable equal to the stored procedure function call and add .ToArray() at the end, which is what I want to do here but I'm getting an error message int does not contain a definition for toarray...
I'm confused because I followed the same syntax that I used in another part of the program for a similar thing. It worked fine before but I can't figure out why it wants to fight with me now.
Here's my SQL:
IF OBJECT_ID('#temp') IS NOT NULL
BEGIN
DROP TABLE #temp
END
--Create temp table to store data
CREATE TABLE #temp
(
EventID nvarchar(50),
RunDate date,
SectionCode nvarchar(50),
SectionMapCode nvarchar(50),
DispSort int,
Capacity int,
Attendance int,
PctCap int,
HeatColor nvarchar(50)
)
DECLARE #runDate date = GETDATE()
INSERT #temp Exec GamedayReporting.dbo.uspGetEventKillSheetDetailedReport #EventID, #runDate;
select Capacity from #temp;
This returns exactly what I want in SQL but when I call it in my Controller I get the error I posted above.
Here's my C# code:
public ActionResult Dropdown()
{
// add your code after post method is done
var selectedOption = Request["eventId"];
var date = DateTime.Today;
var myQuery = db.uspGetEventKillSheetDetailedReport(selectedOption, date).ToArray();
ViewData["query"] = myQuery;
System.Diagnostics.Debug.WriteLine(myQuery);
TempData["option"] = selectedOption;
return RedirectToAction("Map");
}
public ActionResult Map()
{
var secAttendance = db.uspGetSectionAttendance("option").ToArray();
var secCapacity = db.uspGetSecCapacity("option");
var secMapCode = db.uspGetSectionMapCode("option");
}
public JsonResult GetDropdownList()
{
var ids = db.uspGetAllEventIds().ToArray();
ViewData["ids"] = db.uspGetAllEventIds().ToArray();
return Json(new { data = ids }, JsonRequestBehavior.AllowGet);
}
So Dropdown() and GetDropdownList() work fine, but I'm getting the problem with Map().
Basically I want to take the column returned from my SP and store it into an array but it won't let me. Anybody able to help me work through this?
Update
I changed .ToArray() to .toString().toArray(), which got me past the compiler error, but upon logging it into the console I found it was returning char instead of string. So I changed the whole line to
string secAttendance = new string(db.uspGetSectionAttendance("option").ToString().ToArray());
and output the result into the console and found it returns 0.
0 comes from the Return Value in SQL, which I don't understand. It will fetch the correct column but will not send the correct data to ASP.
Here's a screenshot of the output from my SQL:

The error is correct. There must be only one row, one column value in the output. You are selecting
select Capacity from #temp;
which returns a single value as an int.
It cannot be directly cast into an array.
Instead, if you want an array, you can create a blank
Array<int> a = new Array<int>[1]
and then push this output to that array.

How can I use more than 2100 values in an IN clause using Dapper?

I have a List containing ids that I want to insert into a temp table using Dapper in order to avoid the SQL limit on parameters in the 'IN' clause.
So currently my code looks like this:
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
return db.Query<int>(
#"SELECT a.animalID
FROM
dbo.animalTypes [at]
INNER JOIN animals [a] on a.animalTypeId = at.animalTypeId
INNER JOIN edibleAnimals e on e.animalID = a.animalID
WHERE
at.animalId in #animalIds", new { animalIds }).ToList();
}
}
The problem I need to solve is that when there are more than 2100 ids in the animalIds list then I get a SQL error "The incoming request has too many parameters. The server supports a maximum of 2100 parameters".
So now I would like to create a temp table populated with the animalIds passed into the method. Then I can join the animals table on the temp table and avoid having a huge "IN" clause.
I have tried various combinations of syntax but not got anywhere.
This is where I am now:
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
db.Execute(#"SELECT INTO #tempAnmialIds #animalIds");
return db.Query<int>(
#"SELECT a.animalID
FROM
dbo.animalTypes [at]
INNER JOIN animals [a] on a.animalTypeId = at.animalTypeId
INNER JOIN edibleAnimals e on e.animalID = a.animalID
INNER JOIN #tempAnmialIds tmp on tmp.animalID = a.animalID).ToList();
}
}
I can't get the SELECT INTO working with the list of IDs. Am I going about this the wrong way maybe there is a better way to avoid the "IN" clause limit.
I do have a backup solution in that I can split the incoming list of animalIDs into blocks of 1000 but I've read that the large "IN" clause sufferes a performance hit and joining a temp table will be more efficient and it also means I don;t need extra 'splitting' code to batch up the ids in to blocks of 1000.

Ok, here's the version you want. I'm adding this as a separate answer, as my first answer using SP/TVP utilizes a different concept.
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
// This Open() call is vital! If you don't open the connection, Dapper will
// open/close it automagically, which means that you'll loose the created
// temp table directly after the statement completes.
db.Open();
// This temp table is created having a primary key. So make sure you don't pass
// any duplicate IDs
db.Execute("CREATE TABLE #tempAnimalIds(animalId int not null primary key);");
while (animalIds.Any())
{
// Build the statements to insert the Ids. For this, we need to split animalIDs
// into chunks of 1000, as this flavour of INSERT INTO is limited to 1000 values
// at a time.
var ids2Insert = animalIds.Take(1000);
animalIds = animalIds.Skip(1000).ToList();
StringBuilder stmt = new StringBuilder("INSERT INTO #tempAnimalIds VALUES (");
stmt.Append(string.Join("),(", ids2Insert));
stmt.Append(");");
db.Execute(stmt.ToString());
}
return db.Query<int>(#"SELECT animalID FROM #tempAnimalIds").ToList();
}
}
To test:
var ids = LoadAnimalTypeIdsFromAnimalIds(Enumerable.Range(1, 2500).ToList());
You just need to amend your select statement to what it originally was. As I don't have all your tables in my environment, I just selected from the created temp table to prove it works the way it should.
Pitfalls, see comments:
Open the connection at the beginning, otherwise the temp table will
be gone after dapper automatically closes the connection right after
creating the table.
This particular flavour of INSERT INTO is limited
to 1000 values at a time, so the passed IDs need to be split into
chunks accordingly.
Don't pass duplicate keys, as the primary key on the temp table will not allow that.
Edit
It seems Dapper supports a set-based operation which will make this work too:
public IList<int> LoadAnimalTypeIdsFromAnimalIdsV2(IList<int> animalIds)
{
// This creates an IEnumerable of an anonymous type containing an Id property. This seems
// to be necessary to be able to grab the Id by it's name via Dapper.
var namedIDs = animalIds.Select(i => new {Id = i});
using (var db = new SqlConnection(this.connectionString))
{
// This is vital! If you don't open the connection, Dapper will open/close it
// automagically, which means that you'll loose the created temp table directly
// after the statement completes.
db.Open();
// This temp table is created having a primary key. So make sure you don't pass
// any duplicate IDs
db.Execute("CREATE TABLE #tempAnimalIds(animalId int not null primary key);");
// Using one of Dapper's convenient features, the INSERT becomes:
db.Execute("INSERT INTO #tempAnimalIds VALUES(#Id);", namedIDs);
return db.Query<int>(#"SELECT animalID FROM #tempAnimalIds").ToList();
}
}
I don't know how well this will perform compared to the previous version (ie. 2500 single inserts instead of three inserts with 1000, 1000, 500 values each). But the doc suggests that it performs better if used together with async, MARS and Pipelining.

In your example, what I can't see is how your list of animalIds is actually passed to the query to be inserted into the #tempAnimalIDs table.
There is a way to do it without using a temp table, utilizing a stored procedure with a table value parameter.
SQL:
CREATE TYPE [dbo].[udtKeys] AS TABLE([i] [int] NOT NULL)
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[myProc](#data as dbo.udtKeys readonly)AS
BEGIN
select i from #data;
END
GO
This will create a user defined table type called udtKeys which contains just one int column named i, and a stored procedure that expects a parameter of that type. The proc does nothing else but to select the IDs you passed, but you can of course join other tables to it. For a hint regarding the syntax, see here.
C#:
var dataTable = new DataTable();
dataTable.Columns.Add("i", typeof(int));
foreach (var animalId in animalIds)
dataTable.Rows.Add(animalId);
using(SqlConnection conn = new SqlConnection("connectionString goes here"))
{
var r=conn.Query("myProc", new {data=dataTable},commandType: CommandType.StoredProcedure);
// r contains your results
}
The parameter within the procedure gets populated by passing a DataTable, and that DataTable's structure must match the one of the table type you created.
If you really need to pass more that 2100 values, you may want to consider indexing your table type to increase performance. You can actually give it a primary key if you don't pass any duplicate keys, like this:
CREATE TYPE [dbo].[udtKeys] AS TABLE(
[i] [int] NOT NULL,
PRIMARY KEY CLUSTERED
(
[i] ASC
)WITH (IGNORE_DUP_KEY = OFF)
)
GO
You may also need to assign execute permissions for the type to the database user you execute this with, like so:
GRANT EXEC ON TYPE::[dbo].[udtKeys] TO [User]
GO
See also here and here.

For me, the best way I was able to come up with was turning the list into a comma separated list in C# then using string_split in SQL to insert the data into a temp table. There are probably upper limits to this, but in my case I was only dealing with 6,000 records and it worked really fast.
public IList<int> LoadAnimalTypeIdsFromAnimalIds(IList<int> animalIds)
{
using (var db = new SqlConnection(this.connectionString))
{
return db.Query<int>(
#" --Created a temp table to join to later. An index on this would probably be good too.
CREATE TABLE #tempAnimals (Id INT)
INSERT INTO #tempAnimals (ID)
SELECT value FROM string_split(#animalIdStrings)
SELECT at.animalTypeID
FROM dbo.animalTypes [at]
JOIN animals [a] ON a.animalTypeId = at.animalTypeId
JOIN #tempAnimals temp ON temp.ID = a.animalID -- <-- added this
JOIN edibleAnimals e ON e.animalID = a.animalID",
new { animalIdStrings = string.Join(",", animalIds) }).ToList();
}
}
It might be worth noting that string_split is only available in SQL Server 2016 or higher or if using Azure SQL then compatibility mode 130 or higher. https://learn.microsoft.com/en-us/sql/t-sql/functions/string-split-transact-sql?view=sql-server-ver15

Pass table value type to SQL Server stored procedure via Entity Framework

I created a user-defined table type in SQL Server:
CREATE TYPE dbo.TestType AS TABLE
(
ColumnA int,
ColumnB nvarchar(500)
)
And I'm using a stored procedure to insert records into the database:
create procedure [dbo].[sp_Test_CustomType]
#testing TestType READONLY
as
insert into [dbo].[myTable]
select ColumnA, ColumnB
from #testing
And I would like to use EF to execute this stored procedure, but here's the problem: how can I pass a user defined table to the stored procedure?
I tried adding the stored procedure to the model, but I'm unable to find the desired stored procedure in the updated context.
What I'm trying to do is to execute a bulk insert to a table, here's the method that I'm currently using:
List<items> itemToInsertToDB = //fetchItems;
foreach(items i in itemToInsertToDB)
{
context.sp_InsertToTable(i.ColumnA, i.ColumnB)
}
Currently, I use a foreach loop to loop through the list to insert item to DB, but if the list have a lot of items, then there will be a performance issue, so, I'm thinking of passing a list to the stored procedure and do the insert inside.
So how to solve this problem? or are there any better ways to do this?

Lets say you want to send a table with a single column of GUIDs.
First we need to create a structure using SqlMetaData which represents the schema of the table (columns).
The below code demonstrates one column named "Id" of the GUID is the SQL stored procedure parameter table type
var tableSchema = new List<SqlMetaData>(1)
{
new SqlMetaData("Id", SqlDbType.UniqueIdentifier)
}.ToArray();
Next you create a list of records that match the schema using SqlDataRecord.
The below code demonstrates how to add the items inside a list using the above created schema. Create a new SqlDataRecord for each of the items in the list. Replace SetGuid with the corresponding type and Replace Guid.NewGuid() as the corresponding value.
Repeat new SqlDataRecord for each item and add them to a List
var tableRow = new SqlDataRecord(tableSchema);
tableRow.SetGuid(0, Guid.NewGuid());
var table = new List<SqlDataRecord>(1)
{
tableRow
};
Then create the SqlParameter:
var parameter = new SqlParameter();
parameter.SqlDbType = SqlDbType.Structured;
parameter.ParameterName = "#UserIds"; //#UserIds is the stored procedure parameter name
parameter.TypeName = "{Your stored procedure type name}"
parameter.Value = table;
var parameters = new SqlParameter[1]
{
parameter
};
Then simply call the stored procedure by using the Database.SqlQuery.
IEnumerable<ReturnType> result;
using (var myContext = new DbContext())
{
result = myContext.Database.SqlQuery<User>("GetUsers #UserIds", parameters)
.ToList(); // calls the stored procedure
// ToListAsync(); // Async
{
In SQL Server, create your User-Defined Table Type (I suffix them with TTV, Table Typed Value):
CREATE TYPE [dbo].[UniqueidentifiersTTV] AS TABLE(
[Id] [uniqueidentifier] NOT NULL
)
GO
Then specify the type as a parameter (don't forget, Table Type Values have to be readonly!):
CREATE PROCEDURE [dbo].[GetUsers] (
#UserIds [UniqueidentifiersTTV] READONLY
) AS
BEGIN
SET NOCOUNT ON
SELECT u.* -- Just an example :P
FROM [dbo].[Users] u
INNER JOIN #UserIds ids On u.Id = ids.Id
END

I suggest you not using Stored Procedure to insert bulk data, but just rely to Entity Framework insert mechanism.
List<items> itemToInsertToDB = //fetchItems;
foreach(items i in itemToInsertToDB)
{
TestType t = new TestType() { ColumnA = i.ColumnA, ColumnB = i.ColumnB };
context.TestTypes.Add(t);
}
context.SaveChanges();
Entity framework will smartly perform those insertion in single transaction and (usually) in single query execution, which will almost equal to executing stored procedure. This is better rather than relying on stored procedure just to insert bulk of data.

get list from table using stored procedure entity framework

I am using same stored procedure for inserting and fetching data on condition base.
Stored Procedure-
ALTER PROCEDURE dbo.sp_SaveUserRole
#company_name nvarchar(50)=null,
#address nvarchar(250)=null,
#mobileno int =0,
#phoneno int=0,
#faxno int=0,
#email nvarchar(250)=null,
#regno int=0,
#establish_date datetime=null,
#cond int=0
AS
if(#cond=1)
begin
insert into Company (company_name,address,mobileno,phoneno,faxno,regno,email,establish_date) values (#company_name,#address,#mobileno,#phoneno,#faxno,#regno,#email,#establish_date)
end
else if(#cond=2)
begin
select * from company where isactive = 'True';
end
RETURN
As on inserting data using entity framework I am doing this-
public ActionResult SaveRole(CompanyModel cmp)
{
var company_name = cmp.Company_Name;
var regno = cmp.regNo;
var mobileno = cmp.mobileNo;
var phoneno = cmp.phoneNo;
var establish_date = cmp.establish_Date;
var email = cmp.emaiL;
var address = cmp.Address;
var faxno = cmp.faxNo;
db.sp_SaveUserRole(company_name, address, mobileno, phoneno, faxno, email, regno, establish_date,1);
return Json(new { success = true });
Note-
Here condition is 1 so it goes to insert data procedure.
Trying to get a list-
var list = db.sp_SaveUserRole("", "", 0, 0, 0, "", 0, null, 2).ToList();
I tried this way of getting data from table, where I had to pass necessary arguments to this procedure call. Though I wanted to go this procedure only to 2nd condition I've mentioned there.
So only for this 2nd condition How can I modify this procedure without passing arguments?

Instead of using a stored procedure I would add your company table as an entity in your edmx and access it in code.
That way instead of passing #Cont=2 to a stored proc you can instead use LINQ to access the data you require as the SQL seems very basic.
You can then remove that piece of SQL from your stored proc as mixing Inserts with selects doesnt feel right.
e.g.
// For insert
if (cont == 1)
{
// Call your stored proc here
// or if you add the company table to EF you can use the Insert use the Add
// e.g. Rough Example
_yourEntities.Companies.Add(cmp);
// Update the DB
_yourEntities.SaveChanges();
}
else
{
var companies = _yourEntities.Companies.Where(c => c.isactive == 'True');
}
If this solution is not an option i would still look to split the stored procs into two to make life easily.
Ideally though as you using Entity Framework and are looking to insert and select from a single table you get this functionality for free when you add the Company table as a entity in EF.