We Keep Coding

From child class, call base class method while using child fields

I have created a small ETL program with a single method that only takes a connection string for the source database and a connection string for a target database.
Basically, it has 5 steps:
Step 1. Select data from the source.
Step 2. Create a temporary table on the target.
Step 3. Load data from the source into the temporary on the target.
Step 4. Merge the data from temporary table into the actual target table.
Step 5. Drop the temporary table
This works great for a single transformation that needs to take place, but I have about 20 different ETL "jobs" that need to take place.
So instead of copying and pasting the same method verbatim 19 different times, I would like to define a base class that defines this single method one time, and then call this single method from each child class, with its own select, create, merge and drop statements.
Is this possible?
Base Class:
public class ETLBase
{
private static string Select;
private static string CreateTemp;
private static string Merge;
private static string CleanUp;
private static string DestinationTable;
public static void ExecuteJob(string sourceConnectionString, string destinationConnectionString)
{
using (OracleConnection sourceConnection = new OracleConnection(sourceConnectionString))
{
sourceConnection.Open();
OracleCommand selectCommand = new OracleCommand(Select, sourceConnection);
OracleDataReader reader = selectCommand.ExecuteReader();
using (SqlConnection destinationConnection = new SqlConnection(destinationConnectionString))
{
destinationConnection.Open();
SqlCommand createTempCommand = new SqlCommand(CreateTemp, destinationConnection);
createTempCommand.ExecuteNonQuery();
SqlCommand mergeCommand = new SqlCommand(Merge, destinationConnection);
SqlCommand dropCommand = new SqlCommand(CleanUp, destinationConnection);
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(destinationConnection))
{
bulkCopy.DestinationTableName = DestinationTable;
try
{
bulkCopy.WriteToServer(reader);
mergeCommand.ExecuteNonQuery();
dropCommand.ExecuteNonQuery();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
reader.Close();
}
}
}
}
}
}
Child Class:
public class ChildETL: ETLBase
{
private static string Select = #"Select THIS DataStatement";
private static string CreateTemp = #"CREATE TABLE Statement";
private static string Merge = #"Merge Table statement";
private static string CleanUp = "DROP TABLE Statement";
private static string DestinationTable = "##TempTable";
}
And then execute it something like this, but where each child class uses its own defined fields, so it uses it's own SQL statements.
public class Program
{
public static void Main(string[] args)
{
IConfiguration config = new ConfigurationBuilder()
.AddJsonFile("appsettings.json", true, true)
.Build();
string schoolDBConnection = config["SchoolConnection"];
string courseDBConnection = config["CourseConnection"];
string teacherDBConnection = config["Connection"];
ChildETLA.ExecuteJob(schoolDBConnection, courseDBConnection);
ChildETLB.ExecuteJob(teacherDBConnection, courseDBConnection);
//...and so on for each child ETL class
}
}

First of all, you need to run "ExecuteJob()" polymorphically, which means behaviour of "ExeceuteJob" will be different based on source and destination connection string. You can't achieve polymorphism for static function or properties. First of all, you need to refactor it to instance method by taking out the "Static" keywords. Also, the behaviour of the base class is deciding by child classes so that nobody should be able to create an object of base class to make it an abstract class. Look at your code you have two behaviours one for school and another one for the teacher. So you have to create two different child classes which inherit the abstract base class. It is the responsibility of object creation to compose object with source and destination connection string so pass it to the constructor and set it while creating the object itself. Please find the refactored code,
public abstract class ETLBase
{
private readonly string sourceConnectionString;
private readonly string destinationConnectionString;
protected virtual string Select { get; set; } = #"Select THIS DataStatement";
protected virtual string CreateTemp { get; set; } = #"CREATE TABLE Statement";
protected virtual string Merge { get; set; } = #"Merge Table statement";
protected virtual string CleanUp { get; set; } = "DROP TABLE Statement";
protected virtual string DestinationTable { get; set; } = "##TempTable";
protected ETLBase(string sourceConnectionString, string destinationConnectionString)
{
this.sourceConnectionString = sourceConnectionString;
this.destinationConnectionString = destinationConnectionString;
}
public void ExecuteJob()
{
using (OracleConnection sourceConnection = new OracleConnection(sourceConnectionString))
{
sourceConnection.Open();
OracleCommand selectCommand = new OracleCommand(Select, sourceConnection);
OracleDataReader reader = selectCommand.ExecuteReader();
using (SqlConnection destinationConnection = new SqlConnection(destinationConnectionString))
{
destinationConnection.Open();
SqlCommand createTempCommand = new SqlCommand(CreateTemp, destinationConnection);
createTempCommand.ExecuteNonQuery();
SqlCommand mergeCommand = new SqlCommand(Merge, destinationConnection);
SqlCommand dropCommand = new SqlCommand(CleanUp, destinationConnection);
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(destinationConnection))
{
bulkCopy.DestinationTableName = DestinationTable;
try
{
bulkCopy.WriteToServer(reader);
mergeCommand.ExecuteNonQuery();
dropCommand.ExecuteNonQuery();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
reader.Close();
}
}
}
}
}
And the child classes,
public class ChildETLSchool : ETLBase
{
public ChildETLSchool(string sourceConnectionString, string destinationConnectionString)
: base(sourceConnectionString, destinationConnectionString)
{
//Change values of below lines only if you want to override the values
//Select = #"Select THIS DataStatement";
//CreateTemp = #"CREATE TABLE Statement";
//Merge = #"Merge Table statement";
//CleanUp = "DROP TABLE Statement";
//DestinationTable = "##TempTable";
}
}
public class ChildETLTeacher : ETLBase
{
public ChildETLTeacher(string sourceConnectionString, string destinationConnectionString)
: base(sourceConnectionString, destinationConnectionString)
{
//Change values of below lines only if you want to override the values
//Select = #"Select THIS DataStatement";
//CreateTemp = #"CREATE TABLE Statement";
//Merge = #"Merge Table statement";
//CleanUp = "DROP TABLE Statement";
//DestinationTable = "##TempTable";
}
}
And the object creation in main function,
static void Main(string[] args)
{
IConfiguration config = new ConfigurationBuilder()
.AddJsonFile("appsettings.json", true, true)
.Build();
string schoolDBConnection = config["SchoolConnection"];
string courseDBConnection = config["CourseConnection"];
string teacherDBConnection = config["Connection"];
var childETLSchool = new ChildETLSchool(schoolDBConnection, courseDBConnection);
var childETLTeacher = new ChildETLTeacher(teacherDBConnection, courseDBConnection);
childETLSchool.ExecuteJob();
childETLTeacher.ExecuteJob();
}

Loading XML document loads the same group twice

Some classes to start, I'm writing them all so you can reproduce my problem:
public class PermissionObject
{
public string permissionName;
public string permissionObject;
public bool permissionGranted;
public PermissionObject()
{
permissionName = "";
permissionObject = "";
permissionGranted = true;
}
public PermissionObject(string name, string obj, bool granted)
{
permissionName = name;
permissionObject = obj;
permissionGranted = granted;
}
}
public class Config
{
public string cmsDataPath = "";
public string cmsIP = "";
public List<UserClass> usersCMS = new List<UserClass>();
static public string pathToConfig = #"E:\testconpcms.xml";
public string cardServerAddress = "";
public void Save()
{
XmlSerializer serializer = new XmlSerializer(typeof(Config));
using (Stream fileStream = new FileStream(pathToConfig, FileMode.Create))
{
serializer.Serialize(fileStream, this);
}
}
public static Config Load()
{
if (File.Exists(pathToConfig))
{
XmlSerializer serializer = new XmlSerializer(typeof(Config));
try
{
using (Stream fileStream = new FileStream(pathToConfig, FileMode.Open))
{
return (Config)serializer.Deserialize(fileStream);
}
}
catch (Exception ex)
{
return new Config();
}
}
else
{
return null;
}
}
}
public class UserClass
{
public string Name;
public string Login;
public string Password;
public PCMS2 PermissionsList; // OR new PCMS1, as I will explain in a bit
public UserClass()
{
this.Name = "Admin";
this.Login = "61-64-6D-69-6E";
this.Password = "61-64-6D-69-6E";
this.PermissionsList = new PCMS2(); // OR new PCMS1, as I will explain in a bit
}
}
The problematic bit: consider two implementations of PCMS class, PCMS1 and PCMS2:
public class PCMS1
{
public PermissionObject p1, p2;
public PCMS1()
{
p1 = new PermissionObject("ImportConfigCMS", "tsmiImportCMSConfigFile", true);
p2 = new PermissionObject("ExportConfigCMS", "tsmiExportCMSConfigFile", true);
}
}
public class PCMS2
{
public List<PermissionObject> listOfPermissions = new List<PermissionObject>();
public PCMS2()
{
listOfPermissions.Add(new PermissionObject("ImportConfigCMS", "tsmiImportCMSConfigFile", true));
listOfPermissions.Add(new PermissionObject("ExportConfigCMS", "tsmiExportCMSConfigFile", true));
}
}
And finally main class:
public partial class Form1 : Form
{
private Config Con;
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
Con = Config.Load();
if (Con == null)
{
Con = new Config();
Con.cmsDataPath = #"E:\testconpcms.xml";
Con.Save();
}
if (Con.usersCMS.Count == 0)
{
UserClass adminDefault = new UserClass();
Con.usersCMS.Add(adminDefault);
Con.Save();
}
}
}
Now, using either PCMS1 or PCMS2, the config file generates properly - one user with 2 permissions.
However, when config file is present, calling Con = Config.Load() in the main class gives different results.
Using PCMS1, the Con object is as expected - 1 user with 2 permissions.
However, using PCMS2, the Con object is 1 user with 4 (four) permissions. It doubles that field (it's basically p1, p2, p1, p2). Put a BP to see Con after Load().
I guess the list (PCMS2) implementation is doing something wonky during load which I'm not aware of, but I can't seem to find the issue.

You creates your permission objects in constructor of PMCS2 you do it in the constructor of PMCS1 too, but there you do have two properties that will be overwritten by serializer.
In case of of PMCS2 your constructor adds two items to List and than serializer adds the items it has deserilized to the same list.
I don't know exactly your usecase but i would suggest to move init of the permissions to separated method:
public class PCMS1
{
public PermissionObject p1, p2;
public void Init()
{
p1 = new PermissionObject("ImportConfigCMS", "tsmiImportCMSConfigFile", true);
p2 = new PermissionObject("ExportConfigCMS", "tsmiExportCMSConfigFile", true);
}
}
public class PCMS2
{
public List<PermissionObject> listOfPermissions = new List<PermissionObject>();
public void Init()
{
listOfPermissions.Add(new PermissionObject("ImportConfigCMS", "tsmiImportCMSConfigFile", true));
listOfPermissions.Add(new PermissionObject("ExportConfigCMS", "tsmiExportCMSConfigFile", true));
}
}
after that you could call it, if you want to get initial settings:
if (Con.usersCMS.Count == 0)
{
UserClass adminDefault = new UserClass();
adminDefault.PermissionsList.Init();
Con.usersCMS.Add(adminDefault);
Con.Save();
}

Nullreference, learning OOP [duplicate]

This question already has answers here:
What is a NullReferenceException, and how do I fix it?
(27 answers)
Closed 7 years ago.
I'm making an assignment for school and it involves creating an application with certain restrictions on how to create it. Such as having a set amount of methods. I figured i'd say that before posting code that looks unoptimized, and can infact be done with less methods.
Anyway, currently i have a Northwind class, that handles everything that comes from the Database and in this case, creates a list filled with the Shipper object, then i use another Northwind Method to convert that to a queue. (this is what i mean with unoptimized, i could just use a queue from the get-go but i'm not allowed to.) However, when i use it it comes up with a very common error. But i can't figure out why...
class Northwind
{
public Queue<Shipper> Queue { get; set; }
public List<Shipper> GetList()
{
var con = new SqlConnection("Data Source=DESKTOP-G5VBFCN;Initial Catalog=Northwind;Integrated Security=True");
var cmd = new SqlCommand("SELECT CompanyName, Phone FROM Shippers",con);
con.Open();
var reader = cmd.ExecuteReader();
var ShipList = new List<Shipper>();
while (reader.Read())
{
var s = new Shipper
{
CompanyName = reader["CompanyName"].ToString(),
Phone = reader["Phone"].ToString()
};
ShipList.Add(s);
}
con.Close();
return ShipList;
}
public Queue<Shipper> GetQueue(List<Shipper> List)
{
Queue<Shipper> ShipperQueue = new Queue<Shipper>(List);
return ShipperQueue;
}
}
}
And i use the class in my Form1.cs
public partial class Form1 : Form
{
Northwind db;
Shipper Shipper;
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
db.Queue = db.GetQueue(db.GetList());
}
Do please notice that the Northwind db; is actually underlined in green. I just started learning OOP. Thanks for taking the time to look at my code.

Northwind db = new Northwind();
Or make the class static and you don't need to initialize on your own.
static class Northwind
{
public static Queue<Shipper> Queue { get; set; }
public static List<Shipper> GetList()
{
var con = new SqlConnection("Data Source=DESKTOP-G5VBFCN;Initial Catalog=Northwind;Integrated Security=True");
var cmd = new SqlCommand("SELECT CompanyName, Phone FROM Shippers",con);
con.Open();
var reader = cmd.ExecuteReader();
var ShipList = new List<Shipper>();
while (reader.Read())
{
var s = new Shipper
{
CompanyName = reader["CompanyName"].ToString(),
Phone = reader["Phone"].ToString()
};
ShipList.Add(s);
}
con.Close();
return ShipList;
}
public static Queue<Shipper> GetQueue(List<Shipper> List)
{
Queue<Shipper> ShipperQueue = new Queue<Shipper>(List);
return ShipperQueue;
}
}
and invoke like
private void Form1_Load(object sender, EventArgs e)
{
Northwind.Queue = Northwind.GetQueue(Northwind.GetList());
}

Importing array values in constructor

In my main form I have the following code
[ImportingConstructor]
public MainForm([ImportMany] IEnumerable<AudioPlugin> content)
{
InitializeComponent();
listBox.DisplayMember = "Name";
foreach (var listing in content)
{
listBox.Items.Add(listing);
}
}
In my AudioPlugin class I have the following code
[Export(typeof(INAudioPlugin))]
public class RecordingPanelPlugin : AudioPlugin
{
private string _customer { get; set; }
public void ConnectionString()
{
using (var conn = new SqlCeConnection("Data Source=MyDatabase.sdf;Password=pass;Persist Security Info=True"))
{
conn.Open();
var comm = new SqlCeCommand("SELECT * FROM main", conn);
SqlCeDataReader reader = comm.ExecuteReader();
while (reader.Read())
{
_customer = (string)(reader["CustomerName"]);
Console.WriteLine(_customer);
}
}
}
public string Name
{
get
{
ConnectionString();
return _customer;
}
}
public Control CreatePanel()
{
return new RecordingPanel();
}
}
With the code as it is, I'm only getting the last value returned from the SQL query. What am I missing?

You're assigning the value of the last read element to the variable _customer, you should use a datastructure (like a List) to keep all the elements you're getting and then pass that to the constructor.
Your code should be fixed this way:
private List<string> _customers = new List<string>();
public void ConnectionString()
{
using (var conn = new SqlCeConnection("Data Source=MyDatabase.sdf;Password=pass;Persist Security Info=True"))
{
conn.Open();
var comm = new SqlCeCommand("SELECT * FROM main", conn);
SqlCeDataReader reader = comm.ExecuteReader();
string customer;
while (reader.Read())
{
customer = (string)(reader["CustomerName"]);
Console.WriteLine(customer);
_customers.Add(customer);
}
}
}

I guess I was over-thinking this problem. I removed the Import/Export from both classes and instead decided to call the query directly in my main form so I could populate the listbox as needed. Then I'm assigning a variable to the listbox.SelectedItem and am passing that over to the AudioPlugin class. Everything is working as expected now, thanks for the suggestions to try and resolve the issue though.

Any way to avoid nesting usings when opening a SqlDataReader?

Not a big deal but for neatness sake is there any way to "create and open" a SqlConnection?
I naively wrote this code:
using (var strConnection = new SqlConnection(sourceConnection))
using (var strCommand = new SqlCommand(query, strConnection))
using (var reader = strCommand.ExecuteReader())
{
...
}
Which of course fails on line 3 because the connection isn't open.
Is there a neat way to avoid that nesting that opening the connection introduces?
using (var strConnection = new SqlConnection(sourceConnection))
{
strConnection.Open();
using (var strCommand = new SqlCommand(query, strConnection))
using (var reader = strCommand.ExecuteReader())
{
...
}
}

Good question, my idea is an Extension-Method for SqlConnection.
Check this:
public static class SqlExtensions {
public static SqlConnection OpenAndReturn(this SqlConnection con) {
try {
con.Open();
return con;
} catch {
if(con != null)
con.Dispose();
throw;
}
}
}
Usage:
using(var strConnection = new SqlConnection("CONNECTION").OpenAndReturn())
using(var strCommand = new SqlCommand("QUERY", strConnection))
using(var reader = strCommand.ExecuteReader()) {
//...
}

What about something like that:
class SqlHelper : IDisposable
{
public SqlHelper(string connectionString, string query) { ... }
public SqlConnection Connection { get; set; }
public SqlCommand Command { get; set; }
// SQL querying logic here
public void Execute() { ... }
/** IDisposable implementation **/
}
and in your code
using (SqlHelper sql = new SqlHelper(sourceConnection, query))
{
var reader = sql.Execute();
...
}