We Keep Coding

Column names are not loading correctly c# scripting in SSIS

Im new to scripting in C# using SSIS so I'm not sure why this is happening. I'm using a script component to read some files. My column names that have _ "underscore" are not showing correctly
The error code is this.
Severity Code Description Project File Line Suppression State
Error CS1061 'Output0Buffer' does not contain a definition for 'funded_amnt' and no accessible extension method 'funded_amnt' accepting a first argument of type 'Output0Buffer' could be found (are you missing a using directive or an assembly reference?)

When you add columns to a Script task, I don't know the rules off the top of my head but, there is a translation that happens between Data Flow Column Name to the available name within a script task. Given that every column with an underscore in your data flow is reported to be in error, I'm guessing that is one of the valid characters in a column name that it doesn't allow in a Buffer's field name.
If I were to guess, underscores get eliminated as every column in a script has an _IsNull property added to it and perhaps multiple underscores would complicate logic somewhere.
Given a source query of
SELECT 1 AS col1, 2 AS col_underscore_2;
I get two columns in my data flow task named col1 and col_underscore_2. Adding both to a script task
This is the resulting auto-generated definition of my input buffer (BufferWrapper.cs)
public class Input0Buffer: ScriptBuffer
{
public Input0Buffer(PipelineBuffer Buffer, int[] BufferColumnIndexes, OutputNameMap OutputMap)
: base(Buffer, BufferColumnIndexes, OutputMap)
{
}
public Int32 col1
{
get
{
return Buffer.GetInt32(BufferColumnIndexes[0]);
}
}
public bool col1_IsNull
{
get
{
return IsNull(0);
}
}
public Int32 colunderscore2
{
get
{
return Buffer.GetInt32(BufferColumnIndexes[1]);
}
}
public bool colunderscore2_IsNull
{
get
{
return IsNull(1);
}
}
new public bool NextRow()
{
return base.NextRow();
}
new public bool EndOfRowset()
{
return base.EndOfRowset();
}
}
Of note, it mangles col_underscore_2 to an internal column name of colunderscore2. If you want to see the same for your, either double click on BufferWrapper.cs in Solution Explorer or put your cursor on Input0Buffer in the public override void Input0_ProcessInputRow(Input0Buffer Row) and hit F12 (Go to Definition)
If you want to see the autogenerated columns,
I think a similar logic goes on with buffer names.
I assume you're attempting to auto generate the script task and you'll want to take that into consideration. Also, if the LendingClassRow could have a null, you'll need to have logic like
if(LendingClassRow.verification_status != null)
{
Output0Buffer.verificationstatus = LendingClassRow.verification_status
}
// This entire block is not needed as it will be the default if no value is
// assigned but I call it out here in case future readers have a need to
// muck with it
else
{
Output0Buffer.verificationstatus_IsNull = true;
}

What is proper way to save data from file to object C#

what is proper way to save all lines from text file to objects. I have .txt file something like this
0001Marcus Aurelius 20021122160 21311
0002William Shakespeare 19940822332 11092
0003Albert Camus 20010715180 01232
From this file I know position of each data that is written in file, and all data are formatted.
Line number is from 0 to 3
Book author is from 4 to 30
Publish date is from 31 to 37
Page num. is from 38 to 43
Book code is from 44 to 49
I made class Data which holds information about start, end position, value, error.
Then I made class Line that holds list of type Data, and list that holds all error founded from some line. After load data from line to object Data I loop through lineError and add errors from all line to list, because I need to save errors from each line to database.
My question is this proper way to save data from file to object and after processing same data saving to database, advice for some better approach?
public class Data
{
public int startPosition = 0;
public int endPosition = 0;
public object value = null;
public string fieldName = "";
public Error error = null;
public Data(int start, int end, string name)
{
this.startPosition = start;
this.endPosition = end;
this.fieldName = name;
}
public void SetValueFromLine(string line)
{
string valueFromLine = line.Substring(this.startPosition, this.endPosition - this.startPosition);
// if else statment that checks validity of data (lenght, empty value)
this.value = valueFromLine;
}
}
public class Line
{
public List<Data> lineData = new List<Data>();
public List<Error> lineError = new List<Error>();
public Line()
{
AddObjectDataToList();
}
public void AddObjectDataToList()
{
lineData.Add(new Data(0, 3, "lineNumber"));
lineData.Add(new Data(4, 30, "bookAuthor"));
lineData.Add(new Data(31, 37, "publishData"));
lineData.Add(new Data(38, 43, "pageNumber"));
lineData.Add(new Data(44, 49, "bookCode"));
}
public void LoadLineDataToObjects(string line)
{
foreach(Data s in lineData)
{
s.SetValueFromLine(line);
}
}
public void GetAllErrorFromData()
{
foreach (Data s in lineData)
{
if(s.error != null)
{
lineError.Add(s.error);
}
}
}
}
public class File
{
public string fileName;
public List<Line> lines = new List<Line>();
}

I assume that the focus is on using OOP. I also assume that parsing is a secondary task and I will not consider options for its implementation.
First of all, it is necessary to determine the main acting object. Strange as it may seem, this is not a Book, but the string itself (e.g. DataLine). Initially, I wanted to create a Book from a string (through a separate constructor), but that would be a mistake.
What actions should be able to perform DataLine? - In fact, only one - process. I see two acceptable options for this method:
process returns Book or throws exceptions. (Book process())
process returns nothing, but interacts with another object. (void process(IResults result))
The first option has the following drawbacks:
It is difficult to test (although this applies to the second option). All validation is hidden inside DataLine.
It is impossible/difficult to return a few errors.
The program is aimed at working with incorrect data, so expected exceptions are often generated. This violates the ideology of exceptions. Also, there are small fears of slowing performance.
The second option is devoid of the last two drawbacks. IResults can contain methodserror(...), to return several errors, and success(Book book).
The testability of the process method can be significantly improved by adding IValidator. This object can be passed as a parameter to the DataLine constructor, but this is not entirely correct. First, this unnecessary expense of memory because it will not give us tangible benefits. Secondly, this does not correspond to the essence of the DataLine class. DataLine represents only a line that can be processed in one particular way. Thus, a good solution is the void process (IValidator validator, IResults result).
Summarize the above (may contain syntax errors):
interface IResults {
void error (string message);
void success (Book book);
}
interface IValidator {
// just example
bool checkBookCode (string bookCode);
}
class DataLine {
private readonly string _rawData;
// constructor
/////////////////
public void process (IValidator validator, IResults result) {
// parse _rawData
bool isValid = true; // just example! maybe better to add IResults.hasErrors ()
if (! validator.checkBookCode (bookCode)) {
result.error("Bad book code");
isValid = false;
}
if (isValid) {
result.success(new Book (...));
// or even result.success (...); to avoid cohesion (coupling?) with the Book
}
}
}
The next step is to create a model of the file with the lines. Here again there are many options and nuances, but I would like to pay attention to IEnumerable<DataLine>. Ideally, we need to create a DataLines class that will support IEnumerable<DataLine> and load from a file or from IEnumerable<string>. However, this approach is relatively complex and redundant, it makes sense only in large projects. A much simpler version:
interface DataLinesProvider {
IEnumerable <DataLine> Lines ();
}
class DataLinesFile implements DataLinesProvider {
private readonly string _fileName;
// constructor
////////////////////
IEnumerable <DataLine> Lines () {
// not sure that it's right
return File
. ReadAllLines (_fileName)
.Select (x => new DataLine (x));
}
}
You can infinitely improve the code, introduce new and new abstractions, but here you must start from common sense and a specific problem.
P. S. sorry for "strange" English. Google not always correctly translate such complex topics.

Mapping C# classes to Lua functions via dll

In my "LuaTest" namespace I have a class called "Planet". The C# code reads like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using LuaInterface;
namespace LuaTest
{
public class Planet
{
public Planet(string name)
{
this.Name = name;
}
public Planet() : this("NoName") { }
public string Name
{
get;
private set;
}
public void printName()
{
Console.WriteLine("This planet's name is {0}", Name);
}
}
}
Then I built LuaTest.dll and copied this file to the same folder where my Lua script is saved. In the Lua script I wrote:
--define Path for required dlls
package.cpath = package.cpath .. ";" .. "/?.dll"
package.path = package.path .. ";" .. "/?.dll/"
require 'luanet'
luanet.load_assembly("LuaTest")
local Planet = luanet.import_type("LuaTest.Planet")
local planet = Planet("Earth")
planet.printName()
However, this piece of code does not work. Lua interpreter throws this error:
lua: dllTest.lua:7: attempt to call local 'Planet' (a nil value)
I suspect that my LuaTest assembly is not loaded at all. Could anyone point out where I did wrong? I would very much appreciate it, since I've been stuck by this problem for days.
Also it might be helpful to add that my LuaInterface.dll is the rebuilt version in .NET4.0 environment.

So I spent a LOT of time similarly. What really drove me bonkers was trying to get Enums working. Eventually I ditched my project for a very simplified console application, very similar (ironically also named 'LuaTest').
Edit: I've noted that the initial "luanet.load_assembly("LuaTest")" appears superfluous. Works with it, or surprisingly without it.
Another Edit: As in my badly edited comment below, when I removed:
print(luanet.LuaTest.Pointless)
It all stopped working (LuaTest.Pointless became nil). But adding the luanet.load_assembly("LuaTest") then makes it work. It may be that there is some sort of odd implicit load in the print or in just expressing they type. Very Strange(tm).
In any case, it seems to work for me (note: after a lot of experimentation). I don't know why yours is failing, I don't note any real difference, but here's all my code in case someone else can spot the critical difference:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using LuaInterface;
namespace LuaTest
{
public class Program
{
static void Main(string[] args)
{
Lua lua = new Lua();
lua.DoFile("test.lua");
}
public int some_member = 3;
}
public class Pointless
{
public enum AnEnum
{
One,
Two,
Three
};
public static string aStaticInt = "This is static.";
public double i;
public string n = "Nice";
public AnEnum oneEnumVal = AnEnum.One;
private AnEnum twoEnumVal = AnEnum.Two;
private string very;
public Pointless(string HowPointLess)
{
i = 3.13;
very = HowPointLess;
}
public class MoreInnerClass
{
public string message = "More, please!";
}
public void Compare(AnEnum inputEnum)
{
if (inputEnum == AnEnum.Three)
Console.WriteLine("Match.");
else
Console.WriteLine("Fail match.");
}
}
}
and test.lua:
luanet.load_assembly("LuaTest")
--Pointless is a class in LuaTest assembly
local Pointless = luanet.import_type("LuaTest.Pointless")
print(Pointless)
--Gives 'ProxyType(LuaTest.Pointless): 46104728
print(Pointless.aStaticInt)
--'This is static.'
--Fails if not static, as we expect
--Instantiate a 'Pointless'.
local p = Pointless("Very")
print(p)
--Gives 'LuaTest.Pointless: 12289376'
--Now we can get at the items inside the Pointless
--class (well, this instance, anyway).
local e = p.AnEnum;
print(e)
--ProxyType(LuaTest.Pointless+AnEnum): 23452342
--I guess the + must designate that it is a type?
print(p.i)
--3.14
print(p.oneEnumVal)
--Gives 'One: 0'
print(p.twoEnumVal)
--Gives 'twoEnumVal'... private
--behaves very differently.
print(e.Two:ToString())
--Gives 'Two'
local more = p.MoreInnerClass()
print(more.message)
--'More, Please!'
--create an enum value here in the script,
--pass it back for a comparison to
--the enum.
local anotherEnumVal = p.AnEnum.Three
p:Compare(anotherEnumVal)
--outputs 'Match'

Having spent the last several days working on a project that required this exact functionality from LuaInterface, I stumbled across a piece of Lua code that turned out to be the perfect solution (see Reference 1). Whilst searching for this solution, I noticed this question and figured I'd drop my two cents in.
To apply this solution, I merely run the CLRPackage code while initializing my LuaInterface Lua object. However, the require statement works just as well.
The code provided in reference 1 allows the use of import statements, similar to C# using statements. Once an assembly is imported, its members are accessible in the global namespace. The import statement eliminates the need to use load_assembly or import_type (except in situations in which you need to use members of the same name from different assemblies. In this scenario, import_type would be used similar to C# using NewTypeName = Assembly.OldTypeName).
import "LuaTest"
planet = Planet("Earth")
planet:printName()
This package also works great with enums!
Further information regarding the use of this package may be found at Reference 2.
Hope this helps!
Reference 1: https://github.com/stevedonovan/MonoLuaInterface/blob/master/bin/lua/CLRPackage.lua
Reference 2: http://penlight.luaforge.net/project-pages/penlight/packages/LuaInterface/

I spent some time in binding C# dll to lua. Your posts were helpful but something was missing. The following solution should work:
(Make sure to change your compiler to .NET Framework 3.5 or lower!)
Planet.dll:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace Planets
{
public class Planet
{
private string name;
public string Name
{
get { return name; }
set { this.name = value; }
}
private float diameter;
public float Diameter
{
get { return diameter; }
set { this.diameter = value; }
}
private int cntContinents;
public int CntContinents
{
get { return cntContinents; }
set { this.cntContinents = value; }
}
public Planet()
{
Console.WriteLine("Constructor 1");
this.name = "nameless";
this.diameter = 0;
this.cntContinents = 0;
}
public Planet(string n, float d, int k)
{
Console.WriteLine("Constructor 2");
this.name = n;
this.diameter = d;
this.cntContinents = k;
}
public void testMethod()
{
Console.WriteLine("This is a Test!");
}
}
}
Use the code above, paste it into your class library project and compile it with .NET smaller or equal 3.5.
The location of the generated DLL needs to be known by the lua enviroment. Paste it e.g at "clibs"-folder or another well known lua system path. Then try to use the following lua example. It should work.
Test1.lua: (Option 1 with "import" from CLRPackage)
require "luanet"
require "CLRPackage"
import "Planet"
local PlanetClass = luanet.import_type("Planets.Planet")
print(PlanetClass)
local PlanetObject1 = PlanetClass()
print(PlanetObject1)
local PlanetObject2 = PlanetClass("Earth",6371.00*2,7)
print(PlanetObject1.Name)
PlanetObject1.Name = 'Mars'
print(PlanetObject1.Name)
print( "Planet " ..
PlanetObject2.Name ..
" is my home planet. Its diameter is round about " ..
PlanetObject2.Diameter .. "km." ..
" Our neighour is " ..
PlanetObject1.Name)
Test2.lua: (Option 2 with "load_assembly")
require "luanet"
require "CLRPackage"
luanet.load_assembly("Planet")
local PlanetClass = luanet.import_type("Planets.Planet")
print(PlanetClass)
local PlanetObject1 = PlanetClass()
print(PlanetObject1)
local PlanetObject2 = PlanetClass("Earth",6371.00*2,7)
print(PlanetObject1.Name)
PlanetObject1.Name = 'Mars'
print(PlanetObject1.Name)
print( "Planet " ..
PlanetObject2.Name ..
" is my home planet. Its diameter is round about " ..
PlanetObject2.Diameter .. "km." ..
" Our neighour is " ..
PlanetObject1.Name)
In both cases the console output will look like this:
ProxyType(Planets.Planet): 18643596
Constructor 1
Planets.Planet: 33574638
Constructor 2
nameless
Mars
Planet Earth is my home planet. Its diameter is round about 12742km. Our neighbour is Mars
I hope its helps some of you.
Edit 1:
by the way, a method call from lua looks like this:
PlanetObject1:testMethod()
PlanetObject2:testMethod()
Edit 2:
I found different dll's whitch needed to be handled differently. One needed the "import"-function and another needed the "load_assembly"-function. Keep that maybe in mind!

SSIS Script Component FireError DNE in Current Context

I am trying to add some more native error handling to an SSIS C# Script Component (in Data-Flow). Right now I force it to crash the component with 1/0 which works but is hackish.
My script does some jazz on input data, but I would like to validate it at several steps and fail the component if any validations fail. The source is a select, so i don't need to
roll back any transactions etc... but I would like the component to fail the dataflow so the dataflow component will fail and follow the error handling I prescribe in control flow.
Here is the simplist relavant snippet that is holding me up:
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
public override void PreExecute()
{
base.PreExecute();
/*
Add your code here for preprocessing or remove if not needed
*/
bool pbCancel = false;
////check Row Count>0
if (Variables.WeeklyLRrowCount == 0)
this.ComponentMetaData.FireError(-1, "", "Fails Validation due to Empty Table.", "", 0, out pbCancel);
}
public override void PostExecute()
{
base.PostExecute();
/*
Add your code here for postprocessing or remove if not needed
You can set read/write variables here, for example:
Variables.MyIntVar = 100
*/
}
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
/*
Add your code here
*/
}
}
I get the following from SSIS:
"The name 'FireError' does not exist in the current context."
Is there something I am missing here?
Thanks!

You should just move your code to the PostExecute method.

"Typically, during component design, the FireError, FireInformation, and FireWarning methods are called to provide user feedback when a component is incorrectly configured."
http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.dts.runtime.idtscomponentevents(v=sql.105).aspx
If I read this correctly this is not for runtime use in a script task, but during design time use of a custom component to indicate configuration errors.
This code from http://msdn.microsoft.com/en-us/library/ms135912(v=sql.105).aspx
may be what you want.
public override void RegisterEvents()
{
string [] parameterNames = new string[2]{"RowCount", "StartTime"};
ushort [] parameterTypes = new ushort[2]{ DtsConvert.VarTypeFromTypeCode(TypeCode.Int32), DtsConvert.VarTypeFromTypeCode(TypeCode.DateTime)};
string [] parameterDescriptions = new string[2]{"The number of rows to sort.", "The start time of the Sort operation."};
EventInfos.Add("StartingSort","Fires when the component begins sorting the rows.",false,ref parameterNames, ref paramterTypes, ref parameterDescriptions);
}
public override void ProcessInput(int inputID, PipelineBuffer buffer)
{
while (buffer.NextRow())
{
// Process buffer rows.
}
IDTSEventInfo100 eventInfo = EventInfos["StartingSort"];
object []arguments = new object[2]{buffer.RowCount, DateTime.Now };
ComponentMetaData.FireCustomEvent("StartingSort", "Beginning sort operation.", ref arguments, ComponentMetaData.Name, ref FireSortEventAgain);
}

LINQ to Entities - Best way to accomplish this?

OK, I am mostly a LAMP developer so I am new to the entity framework. However, I am familiar with the basics in LINQ and have generated a entity model from my DB. Now here is my requirement:
I have a datagrid on a WinForm that will be refreshed from a data source on a remote server every few seconds as changes to the dataset are made from other sources. Obviously, I'd like to construct a lambda expression to get the right anonymous type to satisfy the columns that needs to be shown in my datagrid. I have done this and here is the result (I'm using a custom datagrid control, btw):
And my code thus far:
Models.dataEntities objDB = new Models.dataEntities();
var vans = from v in objDB.vans
select v;
gcVans.DataSource = vans;
OK, so now I have my basic data set. One problem I had is that the "Status" column will show a calculated string based on several parameters in the data set. I added this to my entities via a partial class. As you can see in the screenshot, this is working correctly:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace WindowsFormsApplication1.Models {
public partial class van {
public string van_status {
get {
if (this.is_offline == 1) {
return "Offline";
} else if (this.is_prayer_room == 1) {
return "In Prayer Room";
} else {
return "TODO: Create statuses";
}
}
}
}
}
This added property works fine. However, the second I try to project the status in an anonymous type so I can also retrieve the school name, I get an error:
Models.dataEntities objDB = new Models.dataEntities();
var vans = from v in objDB.vans
select new {
van_name = v.van_name,
school_name = v.school.school_name,
capacity = v.capacity,
phone = v.phone,
van_status = v.van_status
};
gcVans.DataSource = vans;
So, I have two questions:
1) If I cannot use computed properties of my partial classes in LINQ projections, how am I supposed to show my computed string in my datagrid?
2) Am I even approaching this correctly? When I resolve #1, how would I refresh this data (ie during a timer fire event)? Would I simply call objDB.refresh() and then tell my datagrid to update from the datasource? Does calling that method actually run the lambda expression above or does it load everything from the DB?
Thanks for any assistance with this. Also, if you have any best practices to share that would be awesome! I hope I explained this as thoroughly as you need to provide assistance.

1) Instead of modifying your EF object with a partial class you could always create your own class that contains your read only property van_status. The code you've got would be nearly identical:
Models.dataEntities objDB = new Models.dataEntities();
gcVans.DataSource = from v in objDB.vans
select new DisplayVan {
van_name = v.van_name,
school_name = v.school.school_name,
capacity = v.capacity,
phone = v.phone,
};
The van_status property, since it's read-only, will not need to be specified in the query.
2) I'm more a web developer than a desktop developer so I'll give you my take on how to refresh the grid (it may not be the preferred methodology for fat clients)...
I'm reluctant to trust .Refresh() methods and hope all works to maximal efficiency and work properly. Instead, encapsulate the code from #1 in a method of your own and invoke it from your timer event firing (or however you choose to implement the periodic refresh).

Another good option would be to create an extension method.
Here is a simple example:
using System;
namespace WindowsFormsApplication1 {
static class Program {
[STAThread]
static void Main() {
Van van = new Van();
string status = van.GetStatus();
}
}
public static class VanExtension {
public static string GetStatus(this Van van) {
if(van.is_offline == 1) {
return "Offline";
}
else if(van.is_prayer_room == 1) {
return "In Prayer Room";
}
return "TODO: Create statuses";
}
}
public class Van {
public int is_offline { get; set; }
public int is_prayer_room { get; set; }
}
}
Be sure to put this extension class in the same namespace as the entity class.