We Keep Coding

How to send SSIS failure notification when executing from SQL Agent Job

I have a SQL Agent Job with steps that resemble the following:
Exec SSIS package A
Exec SSIS package B
Exec SSIS package C
Exec stored proc
I have it set up so that if any of the SSIS packages fail, the job moves on to next step. The package failure notification comes from SSIS. I have 2 script tasks. One in event handler to capture/append error codes/messages to a variable:
The second script task is in control flow and grabs the appended error message variables and other email forming variables and sends them to an outlook email account:
This works when running the package with my credentials + just the individual package. When I try to run the package through the SQL Agent Job with default credentials (or even my own) I get the following errors:
The desired result is SQL Agent Job runs completely regardless of any of steps 1-3 failing. When any step 1-3 fail, within SSIS the failure email message is sent.
EDIT: SQL Agent Job
We are using the legacy Package Deployment Model. The Package lives on the file system.

Azure Pipelines - Test Automation - Test data setup SQL Server

We are currently using Azure pipelines to run automated UI and API tests on different test environments. Our tests are written in C# using visual studio, specflow and MStest
It's working well, but now i am looking to improve the test data set up before our tests are ran.
We use SQL server databases. Currently we use a 'Execute single or multiple SQL scripts' Azure Pipeline task to delete data in tables so we can reuse test data in our scripts. We currently only use DELETE FROM SQL commands. For example:
DELETE FROM [dbo].[table1]
DELETE FROM [dbo].[table2]
etc.
Obviously this is not a good solution as it requires maintenance each time there are db changes or new tables and columns are added. We run into foreign key constraints etc.
The application we are running tests on consists or 3 databases:
DB 1 = client db
DB 2 = Broker db
DB 3 = internal db
I was thinking a back up and restore of the test database before we run tests would be a better solution. However, I am unsure what the recommended best practice solution is?
I see Azure Pipelines has these tasks, but i was think maybe a sql/stored procedure script would be better?
Also, can anyone point me in the direction in what a SQL script would look like to do this?

I was thinking a back up and restore of the test database before we
run tests would be a better solution
You can follow below script to backup Azure SQL database in powershell task .
$exportRequest = New-AzureRmSqlDatabaseExport -ResourceGroupName $ResourceGroupName -ServerName $ServerName `
-DatabaseName $DatabaseName -StorageKeytype $StorageKeytype -StorageKey $StorageKey -StorageUri $BacpacUri `
-AdministratorLogin $creds.UserName -AdministratorLoginPassword $creds.Password
$importStatus = Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $importRequest.OperationStatusLink
[Console]::Write("Exporting")
while ($importStatus.Status -eq "InProgress")
{
$importStatus = Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $importRequest.OperationStatusLink
[Console]::Write(".")
Start-Sleep -s 10
}
[Console]::WriteLine("")
$importStatus
For details, please refer to this ticket.

C# Executing a query and getting data out of an Access Database (accdb) without using ACE.OLEDB

I'm writing a WPF application.
Trying to use the normal method of getting a connection returns an error similar to: "The 'Microsoft.ACE.OLEDB.12.0' provider is not registered on the local machine."
ACE.OLEDB has never been installed on this machine so this error makes sense.
I'm trying to create this application in a way so that our users won't need to contact IT to have the application installed. Getting IT involved is a no go situation and the project will be abandoned.
Another team has an Access database (accdb) that I want my application to extract information (only read, no insert or update). I talked to the team and they won't convert this database back to an earlier version (mdb).
After my research I assume that installing ACE.OLEDB without using Admin privileges is impossible. Because of this and my application requirement of not requiring admin privileges I need to start looking for "Mutant"/Dirty solutions that don't involve ACE.OLEDB.
I tried using power-shell but I'm getting the same problems as I had with C# (requires IT to install ACE.OLEDB).
I have two potential solutions. One write a VBA script that opens up the database and dumps a query result into a file. My C# application would call this VB script and then parse the created file.
The second option is to create a new Access process using Process.Start(fullFilePath) and somehow pass the command to execute a query and somehow pass the results back to the executing application (either via a method return or first to a file).
How would you get the data out?
Is there a way for C# to duplicate the DB file and convert it from (accdb -> mdb)?
This is the second question I ask that is very similar.
C# Connecting to Access DB with no install
The difference between the two (to prevent this is a duplicate question) is that in the previous question I was looking for ways to install ACE.OLEDB without admin privileges while here I'm just looking for any other work around.

Found a workaround. It uses Microsoft.Office.Interop.Access found in NuGet.
var accApp = new Microsoft.Office.Interop.Access.Application();
accApp.OpenCurrentDatabase(#tests.DatabasePath);
Microsoft.Office.Interop.Access.Dao.Database cdb = accApp.CurrentDb();
Microsoft.Office.Interop.Access.Dao.Recordset rst =
cdb.OpenRecordset(
"SELECT * FROM Users",
Microsoft.Office.Interop.Access.Dao.RecordsetTypeEnum.dbOpenSnapshot);
while (!rst.EOF)
{
Console.WriteLine(rst.Fields["username"].Value);
rst.MoveNext();
}
rst.Close();
accApp.CloseCurrentDatabase();
accApp.Quit();

Failed to execute SSIS package in SQL Job

I have a simple SSIS package that only has 2 steps. Step 1 is a script task that will return the creation date of the last created file in the specific folder. And the second step, it will save the step 1 data into the database. Here is the complete code for step 1 in C#.
string vDir = Dts.Variables["UnitDirectory"].Value.ToString();
string vDBName = Dts.Variables["DatabaseName"].Value.ToString();
var directory = new DirectoryInfo(vDir);
var myFile = directory.GetFiles(vDBName + "*").OrderByDescending(f => f.LastWriteTime).First();
DateTime creation = File.GetCreationTime(vDir + myFile);
Dts.Variables["CreatedDate"].Value = creation;
Dts.TaskResult = (int)ScriptResults.Success;
In this case, i pass UnitDirectory variable with network sharing directory.
\\192.168.0.2\Database Backup\
The problem is, I can execute this package both from the Integration Service Catalog in SQL Server Management Studio and from Visual Studio. But, when I executed it as SQL Job, it just errors in step 1 and giving an unclear message.
Script Task:Error: Exception has been thrown by the target of an invocation.
I run SQL Job with my domain administrator credentials which have login access into network sharing directory. And this problem starts when I run in new installed SQL Server 2014. Previously this package has run normally in my old SQL Server.
Can you give me solution for this? I need your solution based on your experience. Thanks in advance.

Synchronising databases SQL Server via C#

How do I synchronise a database on one server to the other? I need to do it via C#, not scripts. I was planning to use ADO.NET to retrieve the schema and dataset from one database, but how do I sync it in code?
Thanks

There are various options available to you:
SSIS to Export/Import data between System1 & System2
Mirroring to copy data between System1 & System2
Replication to keep System2 in sync with System1
Scripts for Database Backup/Restore between servers using C# to be the IO glue that scripts the backup on System1, copies the file to System2 and calls the restore script on System2
The easiest option to implement is #4 specially if changes occur to System1 that need to be replicated to System2 (tables, indexes, views, procedures, functions, triggers..etc..etc...)
See Database Mirroring in SQL Server 2005 # SQL Server Performance or Database Mirroring in SQL Server 2005 # Microsoft for mirroring information.
If you need some more enlightment on #4 just reply. Oh, it would help to specify what version of SQL server you are using. This information assumes >=2005 (Yukon, Katmai and possibly Killimanjaro)
Update: I would stay clear of trying to implement your own runtime on this as there are so many variations that just copying between 2 servers requires the ability to do diffs against the objects. Even using the SMO .NET objects this would be an ardous task that would require a lengthy development schedule.
Update 1: The poster is interested in the SSIS version so we will use those instructions.
Start SQL Server Management Studio
Navigate to the chosen database
Right click and Tasks->Export Data
Click Next
Select required source
Click Next
Select destination settings
Click Next
Select either use either tables or Write a query (we will assume use tables)
Click Next
Select the required tables
Click Edit Mappings
Ensure that enable identity insert is selected if required
Ensure Delete/Appends rows is selected as required
Ensure Drop and re-create destination table selected as required
Click Ok
Click Next
Now save this to an SSIS package either into SQL Server or on the filesystem
You can now use DTSExec to execute this package via a scheduler or use the .NET wrapper and call from your own C# runtime
C# Code example from Microsoft Website
using System;
using Microsoft.SqlServer.Dts.Runtime;
namespace RunFromClientAppCS
{
class Program
{
static void Main(string[] args)
{
string pkgLocation;
Package pkg;
Application app;
DTSExecResult pkgResults;
pkgLocation =
#"C:\Program Files\Microsoft SQL Server\100\Samples\Integration Services" +
#"\Package Samples\CalculatedColumns Sample\CalculatedColumns\CalculatedColumns.dtsx";
app = new Application();
pkg = app.LoadPackage(pkgLocation, null);
pkgResults = pkg.Execute();
Console.WriteLine(pkgResults.ToString());
Console.ReadKey();
}
}
}
Resources
dtexec Books Online (BOL)
SQL Server Import Export Wizard BOL
Visual Walkthru # http://www.accelebrate.com/sql_training/ssis_2008_tutorial.htm
Youtube Videos
Loading & Running SSIS Packages Programmatically

Why would you build this when there are db tools that do it for you?
Look into SSIS, or as it was previously known, DTS.