We have an existing MQ custom connection manager that's currently being used by several existing SSIS packages.
I want to add a new property and modify the code a little bit, but if I do that, it looks like I am breaking everything else (have to redo all of them).
Is there a way where I can get around this without disrupting the existing packages that use it?
Assuming that your you need to edit the SSIS package by adding the property and RUN it. After that you don't need the SSIS package. Following code method should be added to your application and call this method. So all the changes will be applied only to the new package without modifying another package.
Get the SSIS package
Create the copy of it by appending the GUID. so your package name is like PackageName_GUID.dtsx
Add your property.
RUN your SSIS package
OnSuccess full execution. Delete the SSIS package.
Code
public static DtsErrors RunSSISPackage(string packagePath, string MQProperty)
{
* Append the auto generated GUID with the package name for running the SSIS package
*/
string uniqueId = Guid.NewGuid().ToString();
string uniquePackage = Path.GetDirectoryName(packagePath) + #"\" + Path.GetFileNameWithoutExtension(packagePath) + "_" + uniqueId + ".dtsx";
File.Copy(packagePath, uniquePackage);
Package pkg;
Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
pkg = app.LoadPackage(uniquePackage, null);
//MessageBox.Show(srcFileName);
//MessageBox.Show(TPODBConnection);
pkg.Connections["MQConnection"].<<YourPropertyName>> = MQProperty;
//Uncomment this to overwrite the existing file
//Do nothing until you are using a version control system
//app.SaveToXml(packagePath, package, null);
DTSExecResult result = pkg.Execute();
if (result == DTSExecResult.Failure)
{
return pkg.Errors;
}
File.Delete(uniquePackage);
return null;
}
Related
I am trying to automate the creation of install packages for the company i work for and am using the Installshield Automation Interface to create an MSI project. One of the things we have done up to now (manually if you can believe it) is go through all of the files we want to release after importing them into installshield and setting them to "Always overwrite" on a folder by folder basis since it seems you cant do it recursively on a parent folder. When creating a Basic MSI on the installshield GUI it lets you do this, however when creating an MSI via the COM object it appears this option is only available to InstallScript which i cant make an MSI with.
Anywho my code kinda looks like this
static void AddFiles(string[] aFiles, ISWiAuto24.ISWiProject oISProj, string sProjName, string ePackName)
{
oISProj.OpenProject(sProjName, false);
string installdirectory = "[ProgramFilesFolder]" + ePackName;
oISProj.INSTALLDIR = installdirectory;
Console.WriteLine("Adding ePack files");
for (int i = 0; i < aFiles.Length;i++ )
{
Console.WriteLine(aFiles[i]);
ISWiComponent NewComponent = oISProj.AddComponent("Component_"+i);
string string_PathToFile = aFiles[i].Substring(0,aFiles[i].LastIndexOf("\\"));
string string_RelativeToInstallDir = string_PathToFile.Substring(aFiles[i].LastIndexOf(ePackName) + ePackName.Length);
NewComponent.Destination = installdirectory+string_RelativeToInstallDir ;
NewComponent.AddFile(aFiles[i]);
/*----------------------------Fails Here--------------------------------------*/
NewComponent.OverwriteMainOptions=0;
/*----------------------------------------------------------------------------*/
}
oISProj.SaveProject();
oISProj.CloseProject();
Console.WriteLine("Done");
}
static voidMain(string[] args){
ISWiAuto24.ISWiProject oISProj = new ISWiAuto24.ISWiProject();
string ePackName = "ThisMonthsBundle"
string[] aFiles = new[] {#"c:/Foo/Roo/Goo/"+ePackName+"/File0",#"c:/Foo/Roo/Goo/"+ePackName+"/File1",#"c:/Foo/Roo/Goo/"+ePackName+"/File2",#"c:/Foo/Roo/Goo/File3"}
string sProjName = "C:/Foo/Bar.ism"
oISProj.CreateProject(sProjName, ISWiProjectType.eptMsi);
AddFiles(aFiles,oISProj,sProjName);
}
does anyone know a way around this?
the error is: COM Exception was unhandled - This property is not supported for Basic MSI Project. You need to remove the line that calls the property from your automation code.
I found an old forum post back in 2010 on the flexera community forum where a flexera developer responded to a user saying that this can be done like so:
ISWiComponent NewComponent = oISProj.AddComponent("Component_1");
NewComponent.Destination = "[ProgramFilesFolder]" + "ProgramName";
NewComponent.AddFile("c:\File1");
ISWiFiles Files = NewComponent.ISWiFiles;
foreach (ISWiFile File in Files)
{
File.OverrideSystemVersion = true;
File.Version = "65535.0.0.0";
}
the developer in question recognised the need for the automation interface to support the ISWiFile.AlwaysOverwrite property and raised a work order for it. i guess they just havent gotten around to it in the 8 years since
https://community.flexerasoftware.com/showthread.php?194448-installshield-2009-automation-File-property-quot-Always-overwrite-quot
Anyway, The above appears to work
There is a requirement to extract source,destination and column names of source and destination. Why am I trying to do this is because I have thousands of packages and opening each package has on an average 60 to 75 of columns and listing all required info will take huge amount of time and its not a single time requirement and this task is done manually every two months in my organization currently.
I'm looking for some ways to reverse engineer keeping all packages in a single folder and then go through each package and get the info and put it in some spreadsheet.
I thought of opening package in xml and get the info of interested node and put in spreadsheet which is little cumbersome. Please suggest what are the available libraries to start with it.
SQL server provide assemblies to manipulate packages programmatically.
To do a reverse engineering (deserialize a dtsx package), You have to do this by looping over packages and read them programmatically, just follow this detailed link
Reading DTS and SSIS packages programmatically
There is another way (harder way and not recommended) to achieve this , by reading dtsx as text file and parse the xml content. check my answer at the following question to get an example:
Automate Version number Retrieval from .Dtsx files
Hint:
just open the package in visual studio. go to the package explorer Tab (near control flow and data flow tabs) you will find a treeview. it will leads you the way you have to search for the component you need
Update 1 - C# Script # 2019-07-08
If you are looking for a script that list all package objects you can use a similar script:
using System;
using DtsRuntime = Microsoft.SqlServer.Dts.Runtime;
using DtsWrapper = Microsoft.SqlServer.Dts.Pipeline.Wrapper;
public void Main()
{
string pkgLocation;
DtsRuntime.Package pkg;
DtsRuntime.Application app;
DtsRuntime. DTSExecResult pkgResults;
pkgLocation =
#"D:\Test\Package 1.dtsx";
app = new DtsRuntime.Application();
pkg = app.LoadPackage(pkgLocation, null);
//List Executables (Tasks)
foreach(DtsRuntime.Executable tsk in pkg.Executables)
{
DtsRuntime.TaskHost TH = (DtsRuntime.TaskHost)tsk;
MessageBox.Show(TH.Name + "\t" + TH.HostType.ToString());
//Data Flow Task components
if (TH.InnerObject.ToString() == "System.__ComObject")
{
try
{
DtsWrapper.MainPipe m = (DtsWrapper.MainPipe)TH.InnerObject;
DtsWrapper.IDTSComponentMetaDataCollection100 mdc = m.ComponentMetaDataCollection;
foreach (DtsWrapper.IDTSComponentMetaData100 md in mdc)
{
MessageBox.Show(TH.Name.ToString() + " - " + md.Name.ToString());
}
}
catch {
// If it is not a data flow task then continue foreach loop
}
}
}
//Event Handlers
foreach(DtsRuntime.DtsEventHandler eh in pkg.EventHandlers)
{
MessageBox.Show(eh.Name + " - " + CM.HostType);
}
//Connection Manager
foreach(DtsRuntime.ConnectionManager CM in pkg.Connections)
{
MessageBox.Show(CM.Name + " - " + CM.HostType);
}
//Parameters
foreach (DtsRuntime.Parameter Param in pkg.Parameters)
{
MessageBox.Show(Param.Name + " - " + Param.DataType.ToString());
}
//Variables
foreach (DtsRuntime.Variable Var in pkg.Variables)
{
MessageBox.Show(Var.Name + " - " + Var.DataType.ToString());
}
//Precedence Constraints
foreach (DtsRuntime.PrecedenceConstraint PC in pkg.PrecedenceConstraints)
{
MessageBox.Show(PC.Name);
}
}
References
Loading and Running a Local Package Programmatically
Update 2 - SSISPackageExplorer Project # 2019-07-10
I started a small project called SSISPackageExplorer on Git-Hub which allow the user to read the package objects in a TreeView, It is very basic right now but i will try to improve it in a while:
GitHub - SSISPackageExplorer
Some of the properties in dtsx Microsoft.SqlServer.Dts.Pipeline are not CLS-compliant.
ColumnInformation Constructors
ColumnInformation Class
Definition
Namespace:
Microsoft.SqlServer.Dts.Pipeline
Assembly:
Microsoft.SqlServer.PipelineHost.dll
Important
This API is not CLS-compliant.
C++
Copy
public ref class ColumnInformation
otherwise try this.
Just open your dtsx package in notepad++. Find table name then do the same search on the property name in all packages( find ion all files). I think that even if you search for the column in dtsx opened in a text editor it will give you everything. It's manual but can be updated with Regex and c#. I never did it with regex. I just did notepad++ and one package once.
We have a fully running database server, say serverA, whose data are refreshed daily.
We want to duplicate this database on a different server, says serverB, so that we have a test environment. The databases has been restored to serverB.
Like serverA, we want serverB's data to be refreshed daily also, so the tests we conduct on serverB can be said as fully accurate since they will have the same data as serverA. We deployed the SSIS packages used in serverA in serverB and copied the SQL Server Agent Jobs in serverB also.
I am trying to modify these jobs and packages so that they can run smoothly on serverB, I'm changing directory paths, server names, etc.
Now, there is this job that always fails because of a package, zip.dtsx.
zip.dtsx retrieves files from directoryA, compresses them and saves the compressed file to directoryB, then deletes the file in directoryA. However, I cannot figure out why it's having a runtime error.
zip.dtsx has a script task named Zip files.
The script language is Microsoft Visual C# 2010
The ReadOnlyVariables set are User::DestinationPath, User::NamePart, User::SourcePath,$Package::filename
The script is,
public void Main()
{
String sourcePath = Convert.ToString(Dts.Variables["SourcePath"].Value);
String namePart = Convert.ToString(Dts.Variables["NamePart"].Value);
String destinationPath = Convert.ToString(Dts.Variables["DestinationPath"].Value);
FileStream sourceFile = File.OpenRead(#sourcePath + namePart);
FileStream destFile = File.Create(#destinationPath + namePart);
GZipStream compStream = new GZipStream(destFile, CompressionMode.Compress);
try
{
int theByte = sourceFile.ReadByte();
while (theByte != -1)
{
compStream.WriteByte((byte)theByte);
theByte = sourceFile.ReadByte();
}
}
finally
{
compStream.Dispose();
sourceFile.Close();
destFile.Close();
File.Delete(#sourcePath + namePart);
}
Dts.TaskResult = (int)ScriptResults.Success;
}
The error I'm getting, when I execute the task in Microsoft Visual Studio -> Right click Script Task object -> Execute task
I am not familiar with Microsoft Visual C# and I have just also begun using SSIS packages, so I'm really at a loss.
UPDATE:
I tried commenting out different lines in the C# script. Finally, when I commented out File.Delete(#sourcePath + namePart);, the job calling zip.dtsx has succeeded. However, I am not sure why I'm having an error with this line. I'm not sure if it is because of permissions or any other else.
I am new to both VC# and SSIS. But here is my scenario, I have data from multiple projects within a single SQL database i.e. multiproject mode (One SQL database storing data from multiple projects). This data is separated based on proj_ID field. I am trying to create C# application which pulls this proj_id in one of it's combobox fields and runs the SSIS package on the project after I click on Export data button. Now I want to use this project id in SSIS package so that package dataflow should only execute on that project.
I have added this in C# code and not sure if this is correct:
public void button2_Click(object sender, EventArgs e)
{
//Start the SSIS Here
try
{
Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
Package package = null;
package = app.LoadPackage(#"C:\SSIS_Projects\XXX_Project\XXXX_Project\Package.dtsx", null);
Microsoft.SqlServer.Dts.Runtime.Variables myVars = package.Variables;
myVars["projectroot"].Value = projectroot;
myVars["path8"].Value =path8;
myVars["PROJ_NAME"].Value = comboBox1.ValueMember;
myVars["PROJ_ID"].Value = comboBox2.ValueMember;
//Excute Package
// working Microsoft.SqlServer.Dts.Runtime.DTSExecResult results = package.Execute();
Microsoft.SqlServer.Dts.Runtime.DTSExecResult results = package.Execute(null, myVars, null, null, null);
if (results == Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure)
{
foreach (Microsoft.SqlServer.Dts.Runtime.DtsError local_DtsError in package.Errors)
{
Console.WriteLine("Package Execution results: {0}", local_DtsError.Description.ToString());
Console.WriteLine();
}
}
}
catch (DtsException ex)
{
throw new FileNotFoundException("[Package.dtsx not found in directory]", ex);
}
}
P.S: PROJ_ID is main variable to call in package but I might need projectroot, name and path as well.
If this is correct, then How do I define and use these variable in SSIS package?
how to proceed further in SSIS package? I can guess that I have to write script task but again it will take me lot of time to learn and write. If you can guide me and provide some sample code then it will be very helpful.
Thanks in advance.
Vishal
I don't remember why, but I do remember that when I needed to execute an SSIS package with variables from C#, here is the approach that finally worked for me:
Create a job that calls the SSIS package.
Create a table that holds the values of the variables used by the package. The table must also include some kind of "Job Result" column.
Include a step at the beginning of the package that reads the values of its variables from the first row in the table that has NULL for JobResult.
Include a step at the end of the package that updates the JobResult column in the variables table, with "Success", "Failure" or whatever you want, as long as it's not NULL.
In C#, populate the table with the variable values you want, then start the job.
Hope this helps.
Is it possible to use the mssql2012 import wizard for databases and tables withing the c# code?
I wanted to write a program with which i can import databases and tables of an access db to a mssql db. Therefore, as a gui, the mssql2012 import wizard works fine, i just need to implement it in c# Code in my project. Are there any prebuilt classes in the .net framework?
Regards
I will suggest you use Bulk Insert. This has already been managed into a .NET class SqlBulkCopy .
If you just want to import data into the database, use SqlBulkCopy. The import wizard does more than just send a batch of data to the server.
Actually, the Import wizard creates and executes an SSIS package with data transformation that extracts data from the source and sends it to the target. It doesn't do anything by itself. The last step of the wizard allows you to save the generated package and use it again, as you would with any other SSIS package.
You can load a saved package and execute it using the code provider in "Loading and Running a Local Package Programmatically". The sample code is very simple:
string pkgLocation;
Package pkg;
Application app;
DTSExecResult pkgResults;
pkgLocation =
#"C:\Program Files\Microsoft SQL Server\100\Samples\Integration Services" +
#"\Package Samples\CalculatedColumns Sample\CalculatedColumns\CalculatedColumns.dtsx";
app = new Application();
pkg = app.LoadPackage(pkgLocation, null);
pkgResults = pkg.Execute();
Console.WriteLine(pkgResults.ToString());
If you want to create a new package each time, you can use the EzAPI library to create a package programmatically, save it and/or execute it. The MSDN blog contains a sample for creating a simple dataflow task:
public class EzOleDbToFilePackage : EzSrcDestPackage<EzOleDbSource, EzSqlOleDbCM, EzFlatFileDestination, EzFlatFileCM>
{
public EzOleDbToFilePackage(Package p) : base(p) { }
public static implicit operator EzOleDbToFilePackage(Package p) { return new EzOleDbToFilePackage(p); }
public EzOleDbToFilePackage(string srv, string db, string table, string file)
: base()
{
SrcConn.SetConnectionString(srv, db);
Source.Table = table;
DestConn.ConnectionString = file;
Dest.Overwrite = true;
// This method defines the columns in FlatFile connection manager which have the same
// datatypes as flat file destination
Dest.DefineColumnsInCM();
}
[STAThread]
static void Main(string[] args)
{
// DEMO 2
EzOleDbToFilePackage p2 = new EzOleDbToFilePackage("localhost", "AdventureWorks", "Address", "result.txt");
p2.DataFlow.Disable = true;
p2.Execute();
Console.Write(string.Format("Package2 executed with result {0}\n", p2.ExecutionResult));
}
}
You should note though that if the generated default mappings aren't suitable for your task, you'll have to define each one separately in code.