In my .NET Core project (.NET 5) I'm using EF Core 6 and for SQL auth, it's using Azure AD Identity authentication.
Here is my startup where DB context is initialized:
services.AddDbContext<EPMO_DevContext>((provider, options) => //managed
{
SqlConnection azureConnection = new SqlConnection(Configuration["DefaultConnection"]);
var dbAuthentication = provider.GetService<Func<string, AzAuthentication>>()("DbScope");
azureConnection.AccessToken = dbAuthentication?.AccessToken;
SqlConnection localDb = new SqlConnection(Configuration.GetConnectionString("EpmoDb"));
var tokenResult = app.GetAuthenticationResultAsync( Configuration["DbScope"] , "mydomain.com").Result;
azureConnection.AccessToken = tokenResult.AccessToken;*/
string env = Configuration["Environment"];
options.UseSqlServer(env == "local" ? localDb:azureConnection, sqlServerOptionsAction: sqlOptions =>
{
sqlOptions.EnableRetryOnFailure(
maxRetryCount: 10,
maxRetryDelay: TimeSpan.FromSeconds(30),
errorNumbersToAdd: null);
});
});
And using DI in my constructor
public AdoExtract(IConfiguration configuration = null, EPMO_DevContext context = null)
{
_configuration = configuration;
_context = context;
}
But since EF taking too much time for bulk insert I am advised to use SQL bulk copy
private int WorkitemsToDb(List<AdoWorkitem> list )
{
try
{
using (var connection = _context.Database.GetDbConnection())
{
connection.Open();
SqlTransaction transaction = connection.BeginTransaction();
using (var bulkCopy = new SqlBulkCopy(connection, SqlBulkCopyOptions.Default, transaction))
{
bulkCopy.BatchSize = 100;
bulkCopy.DestinationTableName = "dbo.Person";
try
{
bulkCopy.WriteToServer(list.AsDataTable());
}
catch (Exception)
{
transaction.Rollback();
connection.Close();
}
}
transaction.Commit();
}
}
catch (Exception ex)
{
throw;
}
}
But at
SqlTransaction transaction = connection.BeginTransaction();
I get this error:
Cannot implicitly convert type 'System.Data.Common.DbTransaction' to 'Microsoft.Data.SqlClient.SqlTransaction'. An explicit conversion exists (are you missing a cast?)
So how can I get the SQL connection in class for bulk copy when using EF and Azure AD authentication for SQL Server?
Related
lib.ParamsInsert is called via another dll.
The transaction is rolled back when there is a problem with the throw statement. However, the operations in paramsInsert can not be undone.
In lib.ParamsInsert, there is a transaction in its own error.
spcarikart.Repository lib = new spcarikart.Repository();
using (var transaction = db.Database.BeginTransaction())
{
try
{
var result = db.Acenta.Add(obj).Entity;
var a = lib.ParamsInsert(db, new Params
{
Baslik = "Bahdir",
Deger = "1"
});
// ....Maybe Error other process
db.SaveChanges();
return result;
}
catch (Exception ex)
{
transaction.Rollback();
}
}
ParamsInsert
using (var transaction = db.Database.BeginTransaction())
{
try
{
var resul = db.Params.Add(obj).Entity;
db.SaveChanges();
transaction.Commit();
return resul;
}
catch (Exception ex)
{
transaction.Rollback();
throw new Exception();
}
}
They use different transactions. Don't start new transaction, as I see, you work with the same db context, so needn't start transaction in paramsInsert, remove using (var transaction = db.Database.BeginTransaction()) from it.
Here's my code:
TransactionScope trans = new TransactionScope();
dbDataContext db = new dbDataContext()
// did some insert
var user = new User();
user.ID = 1;
user.Name = "Nick";
db.Users.Add(user);
db.SubmitChanges();
trans.Complete();
Now how do I rollback transaction if some error occurs while saving changes?
There are two ways in which you can use transactions in entity framework. One using TransactionScope and another using database transaction.
For using database transaction, use the following code example:
using (dbDataContext db = new dbDataContext())
using (var dbContextTransaction = db.Database.BeginTransaction())
{
try
{
var user = new User(){ID = 1, Name = "Nick"};
db.Users.Add(user);
db.SaveChanges();
dbContextTransaction.Commit();
}
catch (Exception)
{
dbContextTransaction.Rollback();
}
}
And here's an example of EF's TransactionScope:
using (TransactionScope tranScope = new TransactionScope())
using (dbDataContext db = new dbDataContext())
{
try
{
var user = new User(){ID = 1, Name = "Nick"};
db.Users.Add(user);
db.SaveChanges();
tranScope.Complete();
}
catch(Exception ex){}
}
As you can see, you don't need to call any Roll-Back method in the case of TransactionScope. If you don't call the Complete() method, the transaction does not get committed and gets rolled back automatically before being disposed.
I am developing a WPF application that interacts with localdb using the Entity Framework 6.0
So the past couple of weeks I've been attempting to get synchronization setup between the following database types:
Server: SQL Server 2008 Database
Client: localdb (SQL Express)
While I can get the structure of the database from the server transferred onto the client it does not bring over the relationships.
This somehow changes the way the ADO.NET Entity Data Model is generated.
When the Entity Data Model is generated for the server structure it generates the many-to-one-to-many relationships as a collection (many-to-many), but on the localdb that is generated (with no relationships generated) it keeps the table structure. This causes issues with my application. I can't fully migrate to an offline application if the synchronization doesn't work properly.
Server to Client (Initialization of client):
using System;
using System.IO;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Data;
using System.Data.SqlClient;
using Microsoft.Synchronization;
using Microsoft.Synchronization.Data;
using Microsoft.Synchronization.Data.SqlServer;
namespace DatabaseSetup
{
class Program
{
static void Main()
{
try
{
CreateLocalDatabase();
ProvisionServer();
ProvisionClient();
Sync();
}
catch (Exception ex)
{
Console.WriteLine($"Exception thrown: {ex.Source}");
Console.WriteLine($"Exception thrown: {ex.Data}");
Console.WriteLine($"Exception thrown: {ex.Message}");
Console.ReadLine();
}
}
public static void CreateLocalDatabase()
{
var conn = new SqlConnection(LocalDb);
var command = new SqlCommand(#"CREATE DATABASE ********", conn);
try
{
conn.Open();
Console.WriteLine(command.ExecuteNonQuery() <= 0
? #"Creating '********' Database on '(localdb)\v11.0'"
: #"Database '********' already exists. Attempting to synchronize.");
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
if(conn.State == ConnectionState.Open)
conn.Close();
}
}
public static void ProvisionServer()
{
Console.WriteLine(#"Attempting to provision server for synchronization...");
// connect to server database
var serverConnection = new SqlConnection(Server);
// define a new scope named ProductsScope
var scopeDescription = DatabaseScope(serverConnection);
// create a server scope provisioning object based on the ProductScope
var serverProvision = new SqlSyncScopeProvisioning(serverConnection, scopeDescription);
if(!serverProvision.ScopeExists("DatabaseScope"))
{ // skipping the creation of table since table already exists on server
serverProvision.SetCreateTableDefault(DbSyncCreationOption.Skip);
// start the provisioning process
serverProvision.Apply();
Console.WriteLine(#"Provisioning complete.");
}
else
{
Console.WriteLine(#"Server already provisioned.");
}
}
public static void ProvisionClient()
{
Console.WriteLine(#"Attempting to provision client for synchronization...");
// create a connection to the SyncExpressDB database
var clientConnection = new SqlConnection(Client);
// create a connection to the SyncDB server database
var serverConnection = new SqlConnection(Server);
// get the description of ProductsScope from the SyncDB server database
var scopeDesc = DatabaseScope(serverConnection);
// create server provisioning object based on the ProductsScope
var clientProvision = new SqlSyncScopeProvisioning(clientConnection, scopeDesc);
if (!clientProvision.ScopeExists("DatabaseScope"))
{
// starts the provisioning process
clientProvision.Apply();
Console.WriteLine(#"Provisioning complete.");
}
else
{
Console.WriteLine(#"Client already provisioned.");
}
}
public static void Sync()
{
//Define conections
Console.WriteLine(#"Attempting to synchronize.");
var serverConnection = new SqlConnection(Server);
var clientConnection = new SqlConnection(Client);
//Create Sync Orchestrator
var syncOrchestrator = new SyncOrchestrator
{
Direction = SyncDirectionOrder.DownloadAndUpload,
LocalProvider = new SqlSyncProvider("DatabaseScope", clientConnection),
RemoteProvider = new SqlSyncProvider("DatabaseScope", serverConnection)
};
((SqlSyncProvider)syncOrchestrator.LocalProvider).ApplyChangeFailed += Program_ApplyChangeFailed;
var syncStats = syncOrchestrator.Synchronize();
Console.WriteLine("\n\nSynchronization complete:");
Console.WriteLine($"Start Time: {syncStats.SyncStartTime}");
Console.WriteLine($"Uploaded: {syncStats.UploadChangesTotal}");
Console.WriteLine($"Downloaded: {syncStats.DownloadChangesTotal}");
Console.WriteLine($"Time Elapsed: {syncStats.SyncEndTime}");
Console.Read();
}
private static DbSyncScopeDescription DatabaseScope(SqlConnection connection)
{
//Define scope
var scopeTables = new Collection<DbSyncTableDescription>();
foreach (var table in TableList)
{
scopeTables.Add(SqlSyncDescriptionBuilder.GetDescriptionForTable(table, connection));
}
var databaseScope = new DbSyncScopeDescription("DatabaseScope");
foreach(var table in scopeTables)
{
databaseScope.Tables.Add(table);
}
return databaseScope;
}
static void Program_ApplyChangeFailed(object sender, DbApplyChangeFailedEventArgs e)
{
// display conflict type
Console.WriteLine(e.Conflict.Type);
// display error message
Console.WriteLine(e.Error);
}
}
}
and this is the action I setup to happen when the user clicks a button on the application whenever they want to sync:
Synchronization between client and server:
using System;
using System.Collections.ObjectModel;
using System.Data.SqlClient;
using System.Windows;
using Microsoft.Synchronization;
using Microsoft.Synchronization.Data;
using Microsoft.Synchronization.Data.SqlServer;
using Outreach.Resources;
namespace WpfApplication
{
class DatabaseSynchronization
{
public static void Sync()
{
//Define conections
Console.WriteLine(#"Attempting to synchronize.");
var clientConnection = new SqlConnection(Constants.Client);
var serverConnection = new SqlConnection(Constants.Server);
//Create Sync Orchestrator
var syncOrchestrator = new SyncOrchestrator();
syncOrchestrator.LocalProvider = new SqlSyncProvider("DatabaseScope", clientConnection);
syncOrchestrator.RemoteProvider = new SqlSyncProvider("DatabaseScope", serverConnection);
syncOrchestrator.Direction = SyncDirectionOrder.UploadAndDownload;
((SqlSyncProvider)syncOrchestrator.LocalProvider).ApplyChangeFailed += Program_ApplyChangeFailed;
var syncStats = syncOrchestrator.Synchronize();
Console.WriteLine("\n\nSynchronization complete:");
Console.WriteLine($"Start Time: {syncStats.SyncStartTime}");
Console.WriteLine($"Uploaded: {syncStats.UploadChangesTotal}");
Console.WriteLine($"Downloaded: {syncStats.DownloadChangesTotal}");
Console.WriteLine($"Time Elapsed: {syncStats.SyncEndTime}");
Console.Read();
}
static void Program_ApplyChangeFailed(object sender, DbApplyChangeFailedEventArgs e)
{
// display conflict type
Console.WriteLine(e.Conflict.Type);
// display error message
Console.WriteLine(e.Error);
}
private static DbSyncScopeDescription DatabaseScope(SqlConnection connection)
{
//Define scope
var scopeTables = new Collection<DbSyncTableDescription>();
foreach(var table in Constants.MsoTableList)
{
scopeTables.Add(SqlSyncDescriptionBuilder.GetDescriptionForTable(table, connection));
}
var outreachScope = new DbSyncScopeDescription("DatabaseScope");
foreach(var table in scopeTables)
{
outreachScope.Tables.Add(table);
}
return outreachScope;
}
public static void ProvisionServer()
{
Console.WriteLine(#"Attempting to provision server for synchronization...");
// connect to server database
var serverConnection = new SqlConnection(Constants.Server);
// define a new scope named ProductsScope
var scopeDescription = DatabaseScope(serverConnection);
// create a server scope provisioning object based on the ProductScope
var serverProvision = new SqlSyncScopeProvisioning(serverConnection, scopeDescription);
if(!serverProvision.ScopeExists("DatabaseScope"))
{ // skipping the creation of table since table already exists on server
serverProvision.SetCreateTableDefault(DbSyncCreationOption.Skip);
// start the provisioning process
serverProvision.Apply();
Console.WriteLine(#"Provisioning complete.");
}
else
{
Console.WriteLine(#"Server already provisioned.");
}
}
public static void ProvisionClient()
{
Console.WriteLine(#"Attempting to provision client for synchronization...");
// create a connection to the SyncExpressDB database
var clientConnection = new SqlConnection(Constants.Client);
// create a connection to the SyncDB server database
var serverConnection = new SqlConnection(Constants.Server);
// get the description of ProductsScope from the SyncDB server database
var scopeDesc = DatabaseScope(serverConnection);
// create server provisioning object based on the ProductsScope
var clientProvision = new SqlSyncScopeProvisioning(clientConnection, scopeDesc);
if(!clientProvision.ScopeExists("DatabaseScope"))
{
// starts the provisioning process
clientProvision.Apply();
Console.WriteLine(#"Provisioning complete.");
}
else
{
Console.WriteLine(#"Client already provisioned.");
}
}
}
}
Is there something I'm doing wrong?
Are there better options than Sync Framework?
Sync Framework don't do schema syncs. it provisions the tables just so you sync data. If you want to include the full server schema, you'll have to script it yourself and execute on the client. if you're only after synching the FKs as well, you can include that as part of the db sync table description when you provision.
When calling sp_setapprole on SQL Server 2014 from a threaded app we get the error:"Impersonate Session Security Context" cannot be called in this batch because a simultaneous batch has called it. The connections are not pooled as far as I know and a new connection is created on each call so I'm not sure where the collision is happening. Here is the test app that creates the error:
using Microsoft.ApplicationBlocks.Data;
using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Threading.Tasks;
namespace PleaseBreak
{
class Program
{
static string ConnectionString = "Data Source=<SQL DB SERVER>; Initial Catalog=<SQL DB NAME>; user id=<USER NAME>; pwd=<PASSWORD>; Enlist=false; Persist Security Info=True; Pooling=false; MultipleActiveResultSets=True; APP=<APPNAME>;";
static int ThreadCount = 3;
static int LoopCount = 100000;
static void Main(string[] args)
{
var tasks = new List<Task>();
for (var i = 0; i < ThreadCount; i++)
{
tasks.Add(Task.Factory.StartNew(Work));
}
Task.WaitAll(tasks.ToArray());
}
static void Work()
{
for (var i = 0; i < LoopCount; i++)
{
using (var connection = new SqlConnection(ConnectionString))
{
connection.Open();
try
{
SqlHelper.ExecuteNonQuery(connection, CommandType.StoredProcedure, "sp_setapprole", new SqlParameter[]
{
new SqlParameter { ParameterName = "#RETURN_VALUE", Direction = ParameterDirection.ReturnValue, DbType = DbType.Int32 },
new SqlParameter { ParameterName = "#rolename", Direction = ParameterDirection.Input, DbType = DbType.AnsiString, Value = <APPROLE NAME> },
new SqlParameter { ParameterName = "#password", Direction = ParameterDirection.Input, DbType = DbType.AnsiString, Value = <APP ROLE PASSWORD> },
});
}
catch (Exception ex)
{
}
}
}
}
}
}
It appears that MultipleActiveResultSets was the issue. Once we added MultipleActiveResultSets=False to the connection the problem went away.
I am using TransactionScope in my repository unit tests to rollback any changes made by tests.
Setup and teardown procedures for tests look like this:
[TestFixture]
public class DeviceRepositoryTests {
private static readonly string ConnectionString =
ConfigurationManager.ConnectionStrings["TestDB"].ConnectionString;
private TransactionScope transaction;
private DeviceRepository repository;
[SetUp]
public void SetUp() {
transaction = new TransactionScope(TransactionScopeOption.Required);
repository = new DeviceRepository(ConnectionString);
}
[TearDown]
public void TearDown() {
transaction.Dispose();
}
}
Problematic test consists of code which inserts records to database and CUT that retrieves those records.
[Test]
public async void GetAll_DeviceHasSensors_ReturnsDevicesWithSensors() {
int device1Id = AddDevice();
AddSensor();
var devices = await repository.GetAllAsync();
// Asserts
}
AddDevice and AddSensor methods open sql connection and insert a row into a database:
private int AddDevice() {
var sqlString = "<SQL>";
using (var connection = CreateConnection())
using (var command = new SqlCommand(sqlString, connection)) {
var insertedId = command.ExecuteScalar();
Assert.AreNotEqual(0, insertedId);
return (int) insertedId;
}
}
private void AddSensor() {
const string sqlString = "<SQL>";
using (var connection = CreateConnection())
using (var command = new SqlCommand(sqlString, connection)) {
var rowsAffected = command.ExecuteNonQuery();
Assert.AreEqual(1, rowsAffected);
}
}
private SqlConnection CreateConnection() {
var result = new SqlConnection(ConnectionString);
result.Open();
return result;
}
GetAllAsync method opens a connection, executes query, and for each fetched row opens new connection to fetch child objects.
public class DeviceRepository {
private readonly string connectionString;
public DeviceRepository(string connectionString) {
this.connectionString = connectionString;
}
public async Task<List<Device>> GetAllAsync() {
var result = new List<Device>();
const string sql = "<SQL>";
using (var connection = await CreateConnection())
using (var command = GetCommand(sql, connection, null))
using (var reader = await command.ExecuteReaderAsync()) {
while (await reader.ReadAsync()) {
var device = new Device {
Id = reader.GetInt32(reader.GetOrdinal("id"))
};
device.Sensors = await GetSensors(device.Id);
result.Add(device);
}
}
return result;
}
private async Task<List<Sensor>> GetSensors(int deviceId) {
var result = new List<Sensor>();
const string sql = "<SQL>";
using (var connection = await CreateConnection())
using (var command = GetCommand(sql, connection, null))
using (var reader = await command.ExecuteReaderAsync()) {
while (await reader.ReadAsync()) {
// Fetch row and add object to result
}
}
return result;
}
private async Task<SqlConnection> CreateConnection() {
var connection = new SqlConnection(connectionString);
await connection.OpenAsync();
return connection;
}
}
The problem is that when GetSensors method calls SqlConnection.Open I get following exception:
System.Transactions.TransactionAbortedException : The transaction has aborted.
----> System.Transactions.TransactionPromotionException : Failure while attempting to promote transaction.
----> System.Data.SqlClient.SqlException : There is already an open DataReader associated with this Command which must be closed first.
----> System.ComponentModel.Win32Exception : The wait operation timed out
I could move code that fetches child object out of the first connection scope (this would work), but let's say I don't want to.
Does this exception mean that it is impossible to open simultaneous connections to DB inside single TransactionScope?
Edit
GetCommand just calls SqlCommand contructor and do some logging.
private static SqlCommand GetCommand(string sql, SqlConnection connection, SqlParameter[] parameters) {
LogSql(sql);
var command = new SqlCommand(sql, connection);
if (parameters != null)
command.Parameters.AddRange(parameters);
return command;
}
The issue is that two DataReader objects can't be open at the same time against the database (unless MARS is enabled). This restriction is by design. As I see it you have a few options:
Enable MARS on your connection string; add this MultipleActiveResultSets=True
Don't use the DataReader if it's really not necessary. But the way you've got your code written, it's pretty necessary.
Populate the Sensor property after loading the devices.
Use Dapper, it can do all of this (including populate the Sensor) and likely faster.
Using Dapper you could do something like this (and you wouldn't need GetSensors):
public async Task<List<Device>> GetAllAsync() {
var result = new List<Device>();
const string sql = "<SQL>";
using (var connection = await CreateConnection())
using (var multi = connection.QueryMultiple(sql, parms)) {
result = multi.Read<Device>().ToList();
var sensors = multi.Read<Sensors>().ToList();
result.ForEach(device => device.Sensors =
sensors.Where(s => s.DeviceId = device.Id).ToList());
}
return result;
}
Here your sql would look like this:
SELECT * FROM Devices
SELECT * FROM Sensors
See the Multi Mapping documentation for Dapper.