I'm a newbie to SSIS / C# (I'm generally a Java developer) so apologies if this is a really stupid question.
Essentially the problem is this: I have two Data Flow tasks which load data up and export them to a legacy flat file format. The formatting is done by a Script Task (C#).
What I'd like to do is share some common code between the two. e.g. I could create a common base class and then extend it for my two different script tasks.
However it seems that SSIS doesn't really make provision for this.
Does anyone know if there is a way of accomplishing what I want to do?
You're correct that there is not a straightforward way to do this directly from SSIS.
In a recent project, we took two different approaches, which both worked fairly well depending on what you need to do:
Create a utility class (as a simple class library) and reference it from your script tasks. This is done pretty much the same as any other sort of reference. If you use .NET 3.5, remember that you'll have to update the version manually in the script tasks since SSIS defaults to 2.0. We also found that if we wanted some manner of reusability in the utility assembly (not relying on hardcoded variable names, etc.) then the package still had to have a fairly large amount of "setup" boilerplate to use the utility scripts.
Create a custom data flow component. This is a much more involved process, but ultimately will do the best in terms of avoiding code duplication. Generally, coding the actual data flow is fairly simple and not that much different than a script component, but the various setup code you'll need can tend to make things complicated. There's also not a lot of support in SSIS for when something goes wrong. Led to a lot of detective work on our project.
If you plan on using something a whole lot, and are committed to getting rid of boilerplate code as much as possible, 2 is the preferred option. If it's being used a few places here and there, consider the simple approach of 1.
I am pretty sure it's possible to access .NET assemblies in SSIS scripts. So you could do it this way. See the article "Accessing .NET assemblies with SSIS" on SQL Server Central.
I believe you will have to create an assembly or webservice for this to work.
This does not completely solve your issue but it does help in not having to recreate all the classes every time you need them (I also do not want to deploy referenced assemblies for my current project ). Firstly you need a master copy of your classes, you can copy them from an existing Script Task using the same process below but in reverse.
Open the Editor for the Script Task and on the Property Explorer click on the Project File (the st_[Guid] ), in the Properties window you’ll see the Project Folder location. (This location gets recreated every time you edit the script task)
In explorer, copy your classes to this folder
On the Project Explorer, click on the “Show All Files” icon
Right click on your files and add to Project
Probably way too late to answer this, but you can click on the solution and add a class there. Then when you go into your scripts you can say add existing object and search for that class you created earlier. For me it was located by the solution for the project. Haven't gone through the deployment or anything for this, but at least you can access the class through the individual scripts.
Related
We have two projects underway and want to share some classes between them. The shared source must be in a single location so when a programmer on Project1 updates the code those changes are immediately seen in Project2.
The important part is that we do not want to maintain multiple copies of the same code. i think we would agree that its simply bad practise to do so.
The only option appears to be putting the code in a class library and load it as a plugin into each project. But I don't believe this works when referencing Unity types like GameObject or transform.
In C/C++ land we could put the source in a directory and include it into any solution we wanted - doesn't seem to be so easy in Unity-land. Is their a better way to share code between unity projects?
EDIT: All projects, including the shared code, are already under source control. This is not a Source Control question, its a framework issue. (And obviously I need to write better questions!)
Lets say the directory structure looks like this...
C:\
Project1\
Assets\
CodeForProject1\
StandardAssets\
Library\
C:\
Project2\
Assets\
CodeForProject2\
Resources\
Library\
D:\
UsefulCode\
PlayerClass\
WeaponsClass\
DataAccessClass\
Can Unity reference code from other projects? Can Unity reference code outside of its own Assets folder? If so, how?
I understand exactly what you're facing, we faced the same problem here some time ago.
The idea of our shared code is to be sort of a middleware, so it is natural that it will evolve through. Our solution was to create a git module for the shared code. I know you said that it's not a source control question, but at least here in my company, create a dll was not a good solution since we are changing the shared code constantly (we tried soft-links too, but it was very painful to maintain in all machines).
Here we created two projects: one for the middleware and another just for the tests. The last one have only the gitmodule to the middleware and a bunch of unit tests. We did it to ensure that there is no dependency to other projects. Both are in the source control. So when someone wants to implement a new feature inside the middleware, he/she make it inside the tests project, and when it's done commits to both projects.
Oh! And you can make a dll with Unity types and use them normally.
i'm working on a large c# project,i wonder why people use DLLs in their apps. I know that a dll file ( please correct if i'm wrong) contains some functions, but why don't we put those functions inside our main c# app?
Thanks
Most of it is summed up in the answer to this question, but the basic reasoning is "so you don't have to duplicate code".
Code reuse. Usually dll files contain functions that are useful in more than one app, and to have them in a single compiled file is a lot easier than copying over all that code.
Portability, Reusability, Modularity.
Splitting types and the like into separate assemblies allows you to reuse those types in different projects, maintain those types a modular fashion (e.g. update just one assembly instead of the whole app), and share parts of your code with others.
It also allows you to group common functionality into a single package.
Maintainability. When you need to fix a bug, you can release just the DLL containing the fix, instead of having to re-release the entire application.
This is an interesting question in a modern computing.
Back in the 16bit days DLLs cut down on the amount code in memory.
This was a big issue when 16 meg computers where considered fully loaded.
I find many of the answers interesting as though a DLL is the only way to have a reusable,maintainable and portable library.
Good reasons for dll's are that you want to share code with an external party.
Just as Visual Studio and other library vendors give you dll's this makes there code available to a external consumer. However, at one time they did distribute them in another way.
Patchable, This is true but how often does this really happen. Every company I've worked for has tested products as a unit. I suppose if you need to do incremental patching because of bandwidth or something this would be a reason.
As for all the other reasons including reusable, maintainable, modularity.
I guess most of you don't remember .LIB files which were statically linked libraries.
You can even distribute .LIB files but they have to be introduced at compile time and not runtime. They can help facilitate reusable, maintainable and modularity just like a DLL.
The big difference is that they are linked when the program is compiled not when it is executed.
I'm really beginning to wonder if we shouldn't return to .LIB files for many things and reducing the number of DLL files. Memory is plentiful and there is overhead in load time when you have to load and runtime link a bunch of DLL files.
Sadly, .LIB files are only an option if your a C++ guy. Maybe they will consider them with C# in the future. I"m just not sure the reasons for DLL's still exist in the broad context they are used for today.
In big softwares, you have many teams they work on several different modules of program, and thay can proceed their goals without needing to know what others is doing! So one of the best solutions, is that each team produces own codes in parallel. So,dll comes to scene....
Extensibility - a lot of plugin frameworks use DLLs/Assemblies for plugins.
dll : a dynamic link library :
it is a library.
It contain some functions and data.
Where we use these function?
we use these function and data which are inside the dll,in another application or program.
the most important thing is that dll will not get loaded into memory, when it require , called it is loaded into ram .
One of the best use is, one can integrate many third party functionalities into your application just by referencing the dlls, no need to use every third party tool/application into your system.
For example, you need to send a meeting invite via MS outlook through code, for this simply refer the dlls provided by MS outlook in your application and you can start coding your way to success!
I'm very annoyed by C# or Java refactoring of namespaces or packages. If you referenced in many classes a class in a common package used in many independent projects and if you decide to just move that package as a package child of the current parent package you have to modify all clients just because you cannot use generic imports like this
import mypackage.*
which would allow refactoring without impacting clients.
So how do you manage to do refactoring when impact can be so big for such a small change ?
What if it's client's code not under my control am I stuck ?
Use an IDE with support for refactoring. If you move a java file in Eclipse, all references are updated. Same for rename, package name changes, etc. Very handy.
It sounds like your asking about packages that are compiled and deployed to other projects as for instance, a jar file. This is one reason why getting your API as correct as possible is so important.
How to Design a Good API and Why it Matters
I think that you could deprecate the existing structure and modify each class to be a wrapper or facade to the new refactored class. This might give you flexibility to continue improving the new structure while slowing migrating projects that use the old code.
imagine someone doing an import like import com.* and if it was like what you wanted it to be, it will load anything and everything in a com package which means zillions of classes are going to be imported, and then you will complain about why it is so slow, why it requires too much memory......
In your case, if you use a IDE, that will take care of most of the work and will be very easy but you will still need to deploy new executables to your clients as well if your application architecture requires.
I have written quite a bit of code of the past few years. I've been using the Visual Studio Development Environment for my C# code, but I wouldn't call myself an advanced user of Visual Studio. I can create projects, create source code, and build/debug the project. I don't use many of the advanced features of the IDE, so perhaps there is a simple way to do what I'd like.
My code is often reused - especially thing like filter tools, custom controls (plots/etc) and some communications code (COM/USB/etc). Every time I create a new project, I end up importing a lot of code that I'll need. This code is copied to the new project directory. If I end up editing that code in some way, I then need to update all of the other versions of that file in my others projects. I'm always having to verify that the code that I am importing is the 'latest and greatest'.
I know it is possible to add code to your project by link, and then you'll update the source file, but I'm curious if there is a better way. My example of a 'better way' is the Allegro Lisp compiler. When you start up Allegro, all of your code is loaded into Allegro, and is instantly available. Then you can start hacking around on anything you'd like, and have access to all of your previous code. When you edit something, and compile it, it is instantly usable in the rest of your projects as well. (Usually even if the program is open!) Perhaps this is something fairly unique to Allegro Lisp?
Are there any ways to do something like this in C#? I'd like to still be able to keep separate projects, but I'd like to share source between them and not have to worry about versions getting out of sync. What does everyone else do when they would like to recycle code?
Thanks,
Giawa
Take some time, work through the code and create different projects, for the likes of filters, plots. Give meaningful namespaces to these assemblies, put the code under source control, use external references to these repos in the source control of your main project, or only import the generated assemblies.
Copying code will lead to errors due to not correcting an error in one place, but correcting it another. Use source control, it's gold.
Our internal HR application is developed by the vendor, but I've been given a requirement to change the behaviour of a certain piece of functionality without their assistance (which I don't want to do, but am investigating...). The functionality in question is an .ashx page which does a number of database lookups, and is called via javascript functions on a web page. We want to add one further database lookup.
We have control over the javascript code, so that's not a problem, but the code for the .ashx page is held in a compiled assembly. I've found where it is, and looked into it using .Net Reflector. Reimplementing the methods involved wouldn't be difficult, if it were technically feasible.
Can I create a new assembly, paste the source taken using Reflector into it, make my modifications, add the DLL to the application, and then set up a new .ashx to refer to it?
The bits I'm worried about not being possible are:
The code refers to some vendor classes held in other assemblies; can I just add references to those assemblies in Visual Studio to allow it to compile, and then when it's compiled and put on the server it'll all work?
Will I have trouble getting the web application to accept the new DLL, given that generally this application is not something we make changes to ourselves?
EDIT:
Some clarification: I'm not actually planning to replace the vendor's DLL with one of my own, that does sound like going a bit too far. Instead, I would make a new DLL with just the one function in it I need (based on stuff taken from the existing DLL using Reflector). I'd need that code to reference other utility vendor code so that it can get at classes needed to access the framework. I'd make a brand new .ashx page to serve up the code in the new DLL.
Luckily after all this I'm off the hook, because my customer agrees that things aren't desperate enough for us to attempt all this! But thanks for the assistance, which is definitely useful.
See my comment about the risk of the approach you're considering.
I really don't recommend the overall approach, but yes, it is possible to reference assemblies you don't have the source for; just add references to the project which will use them.
Replacing the code-behind for the .ashx may be possible depending on whether it's precompiled. If it's not, then I believe replacing the DLL in the BIN folder should do the trick. (Assuming no other assemblies are referencing it.) (DISCLAIMER: I've never tried this, and I don't recommend it.)
If you are just looking to get some additional data, I'd just implement my own lookup and call it from Javascript, rather than messing with reflector. Is there any reason that you can't do that?