We Keep Coding

Connection to snowflake database using C#

I'm new to snowflake database. I need to read the data from multiple tables and do data mapping like sql tables.
I'm trying to get the data from table using the below code:
`
using (IDbConnection snowflakedb = new SnowflakeDbConnection())
{
snowflakedb.ConnectionString = ConfigurationManager.ConnectionStrings["SnowFlakeParserConnection"].ConnectionString;
snowflakedb.Open();
var cmd = snowflakedb.CreateCommand();
cmd.CommandText = "select * from EMAILPARSER_TABLE;";
var reader = cmd.ExecuteReader();
dynamic accountList;
while (reader.Read())
{
Console.WriteLine(reader.GetString(0));
employeeList = reader.GetString(0);
}
snowflakedb.Close();
}
But its giving only single value- value of first row first column .How to read the entire row of the table? Later I also need to merge the data of two tables.
Also, When do we use snowflake ODBC driver connection:
var defaultConnection = _connectionFactory.GetOdbcConnection;
var snowflakeConnection = new OdbcConnection(defaultConnection.ConnectionString);

As #stuartd says, you are getting one column because that is all your code is requesting. You may want to try something like this code from the Github repo
// All types except TIME fail conversion when calling GetTimeSpan
for (int i = 0; i < 12; i++)
{
try
{
((SnowflakeDbDataReader)reader).GetTimeSpan(i);
Assert.Fail("Data should not be converted to TIME");
}
catch (SnowflakeDbException e)
{
Assert.AreEqual(270003, e.ErrorCode);
}
}
Also see this Stack Overflow answer for another approach using a data set.
As for when you should use the ODBC connection, consider using it as you would make that decision for any other database. One thing that has been noted before by other Snowflake developers is that the ODBC driver seems to be more strongly supported than the .Net driver.

How to Insert to multi MYSQL tables C# fast?

I am trying to insert some data into two MYSQL tables.
the second table stores the first table row Id as a foreign key.
I have this code that works fine but it is super slow. what is the best/fastest way to make it faster?
string ConnectionString = "server=localhost; password = 1234; database = DB ; user = Jack";
MySqlConnection mConnection = new MySqlConnection(ConnectionString);
mConnection.Open();
int index = 1;
for (int i = 0; i < 100000; i++)
{
string insertPerson = "INSERT INTO myentities(Name) VALUES (#first_name);"
+ "INSERT INTO secondtable(Id, Name,myentities) VALUES (#ID, #city, LAST_INSERT_ID());";
MySqlCommand command = new MySqlCommand(insertPerson, mConnection);
command.Parameters.AddWithValue("#first_name", "Jack");
command.Parameters.AddWithValue("#ID", i+1);
command.Parameters.AddWithValue("#city", "Frank");
command.ExecuteNonQuery();
command.Parameters.Clear();
}
I have found the following code on one of the StackoverFlow questions but it was inserting data to a single table only, not to multiple tables which are connected through a foreign key.
This code is pretty fast, but I was not sure how I can make it work with multiple tables.
public static void BulkToMySQL()
{
string ConnectionString = "server=192.168.1xxx";
StringBuilder sCommand = new StringBuilder("INSERT INTO User (FirstName, LastName) VALUES ");
using (MySqlConnection mConnection = new MySqlConnection(ConnectionString))
{
List<string> Rows = new List<string>();
for (int i = 0; i < 100000; i++)
{
Rows.Add(string.Format("('{0}','{1}')", MySqlHelper.EscapeString("test"), MySqlHelper.EscapeString("test")));
}
sCommand.Append(string.Join(",", Rows));
sCommand.Append(";");
mConnection.Open();
using (MySqlCommand myCmd = new MySqlCommand(sCommand.ToString(), mConnection))
{
myCmd.CommandType = CommandType.Text;
myCmd.ExecuteNonQuery();
}
}
}

The fastest way possible is to craft a strategy for not calling mysql in a loop via the .NET MySQL Connector. Especially for i=0 to 99999 . The way you achieve this is either thru CASE A: direct db table manipulation or CASE B: thru CSV to db imports with LOAD DATA INFILE.
For CASE B: it is often wise to bring that data into a staging table or tables. Checks can be made for data readiness depending on the particular circumstances. What that means is that you may be getting external data that needs scrubbed (ETL). Other benefits include not committing unholy data to your production tables not fit for consumption. So it leaves an abort option open to you.
Now onto performance anecdotes. With MySQL and the .NET Connector version 6.9.9.0 in late 2016, I can achieve up to 40x performance gains by going this route. It may seem unnatural not to call an INSERT query but I don't in loops. Ok, sure, in small loops, but not in data ingest with bulk. Not even for 500 rows. You will experience noticable UX improvement if you re-craft some routines.
So the above is for data that truly came from external sources. For CASE A: the normal data that is already in your db the above does not apply. In those situations you strive to craft your SQL to massage your data as much as possible (read: 100%) on the server-side. As such it does so without bringing the data back to the client thus requiring some client-side with Connector looping call to get it back into the server. This does not mandate Stored Procedures necessarily or at all. Client-side calls that operate on the data in place without toward client transfers then back up are what you shoot for.

You can gain some improvement by moving unnecessary operations out of the loop, since anything you do there is repeated 100,000 times:
string insertPerson =
"INSERT INTO myentities(Name) VALUES (#first_name);"
+ "INSERT INTO secondtable(Id, Name,myentities) VALUES (#ID, #city, LAST_INSERT_ID());";
string ConnectionString = "server=localhost; password = 1234; database = DB ; user = Jack";
using (var Connection = new MySqlConnection(ConnectionString))
using (var command = new MySqlCommand(insertPerson, mConnection))
{
//guessing at column types and lengths here
command.Parameters.Add("#first_name", MySqlDbType.VarChar, 50).Value = "Jack";
var id = command.Parameters.Add("#ID", MySqlDbType.Int32);
command.Parameters.Add("#city", MySqlDbType.VarChar, 50).Value = "Frank";
mConnection.Open();
for (int i = 1; i <= 100000; i++)
{
id.Value = i;
command.ExecuteNonQuery();
}
}
But mostly, you try to avoid this scenario. Instead, you'd do something like use a numbers table to project the results for both tables in advance. There are some things you can do with foreign key constraints to set locking (you need to lock the whole table to avoid bad keys if someone else inserts or tries to read partially inserted records), transaction logging (you can set it only log the batch, rather than each change) and foreign keys enforcment (you can turn it off while you handle the insert).

Backup SQL Server Schema With Data

I've been tasked with creating a backup of the data in our "default schema" database dbo to the same database using a new schema called dbobackup.
I honestly do not understand what this means as far as a database goes. Apparently, it is like having a database backup inside the existing database. I guess there is some advantage to doing that.
Anyway, I can't seem to find anywhere online that will allow me to do this.
I have found a few posts on here about copying the schema without data, but I need the data too.
Backup SQL Schema Only?
How do I check to see if a schema exists, delete it if it does, and then create a schema that accepts data in the current database?
Once I have the new schema created, can I dump data in there with a simple command like this?
SELECT * INTO [dbobackup].Table1 FROM [dbo].Table1;
That line only backs up one table, though. If I need to do this to 245 tables for this particular customer, I'd need a script.
We have several customers, too, and their databases are not structured identically.
Could I do something along these lines?
I was thinking about creating a small console program to walk through the tables.
How would I modify something like the code below to do what I want?
public static void Backup(string sqlConnection)
{
using (var conn = new SqlConnection(sqlConnection))
{
conn.Open();
var tables = new List<String>();
var sqlSelectTables = "SELECT TableName FROM [dbo];";
using (var cmd = new SqlCommand(sqlSelectTables, conn))
{
using (var r = cmd.ExecuteReader())
{
while (r.Read())
{
var item = String.Format("{0}", r["TableName"]).Trim();
tables.Add(item);
}
}
}
var fmtSelectInto = "SELECT * INTO [dbobackup].{0} FROM [dbo].{0}; ";
using (var cmd = new SqlCommand(null, conn))
{
foreach (var item in tables)
{
cmd.CommandText = String.Format(fmtSelectInto, item);
cmd.ExecuteNonQuery();
}
}
}
}

SQL Server already has this built in. If you open SQL Server Management Studio and right click on the database you want to back up, then select all tasks then backup, you will get an option to back up your database into an existing database.
This is the important part and why you should use the built in functionality: You must copy the data from one DB to the other DB in the correct order or you'll get foreign key errors all over the place. If you have a lot of data tables with a lot of relationships, this will really be hard to nail down on your own. You could write code to make a complete graph of all of the dependencies and then figure out what order to copy the table data (which is essentially what SQL Server already does).
Additionally, there are third-party programs available to do this type of backup as well (see: Google).

This is sort of a "work in progress" approach I got started with that looks promising:
public static void CopyTable(
string databaseName, // i.e. Northwind
string tableName, // i.e. Employees
string schema1, // i.e. dbo
string schema2, // i.e. dboarchive
SqlConnection sqlConn)
{
var conn = new Microsoft.SqlServer.Management.Common.ServerConnection(sqlConn);
var server = new Microsoft.SqlServer.Management.Smo.Server(conn);
var db = new Microsoft.SqlServer.Management.Smo.Database(server, databaseName);
db.Tables.Refresh();
for (var itemId = 0; itemId < db.Tables.Count; itemId++)
{
var table = db.Tables.ItemById(itemId);
if (table.Name == tableName)
{
table.Schema = String.Format("{0}", DatabaseSchema.dboarchive);
table.Create();
}
}
}
The only issue I am currently running into is that my db variable always comes back with Tables.Count == 0.
If I get a chance to fix this, I will update.
For now, I've been told to remove this piece of code and check my code in.

.NET MySQL Data Reader Select Function?

It has been a long time since I have used .NET, but thankfully have almost finished writing a tool to compare an sqlite and mysql database. I am running into an issue though when trying to write a function for my wrapper that will handle SELECT calls as I cannot entirely figure out the Data Reader.
My understanding is that each iteration of a loop on the reader is the next row, and GetString(x) returns the column value for "x" as the index. All the examples I found though went into it knowing the row/column names they needed. How can I do this with a "SELECT * FROM" call and save the column names/values for later access? Microsoft seems to have a "FieldCount" function but I am coming up empty on the MySQL Connector.
Thanks!
public void Query(string query, string tableName)
{
//Open connection
if (this.OpenConnection() == true)
{
//Create Command
MySqlCommand cmd = new MySqlCommand(query, connection);
MySqlDataReader dataReader = cmd.ExecuteReader();
//Read the data and store them in the list
while (dataReader.Read())
{
int count = Count(tableName);
for (int x = 0; x < count; x++)
{
Console.WriteLine(dataReader.GetString(count));
}
}
//close Data Reader
dataReader.Close();
//close Connection
this.CloseConnection();
}
}

You can use DbDataReader.GetName to get the name of a column given its ordinal x.

use the "mysql connector" to access data in MySql it is more simple then to write the queries by your self:
http://dev.mysql.com/downloads/connector/net/
Then use the EntityFramework to access the data through this connector. Also you can automaticly generate *.edmx model from your existing DB in mysql, this will let you to fastly access and work with the Data in your database. Here is information about adding *.edmx model from existing DB:
http://msdn.microsoft.com/en-us/library/vstudio/cc716703(v=vs.100).aspx
The query which will select all data from the table, for example - "Products" will look like that:
List products = dbContext.Products.Where(e=>e.ProductId!=-1).ToList();
this will return the whole list of products in your data base in table Products.
then you can work with products as you want. for example taking the "Name" column for the first product in 'products' will look like that:
String firstProductName = products[0].name;
I hope it helps.

Working with MySQL in C#

Here's my code to print the data to the terminal:
public static void WriteData()
{
string connString = "SERVER=localhost;" +
"DATABASE=db;" +
"UID=user;" +
"PASSWORD=pass;";
MySqlConnection connection = new MySqlConnection(connString);
MySqlCommand command = connection.CreateCommand();
MySqlDataReader reader;
command.CommandText = "SELECT * FROM table1";
connection.Open();
reader = command.ExecuteReader();
while (reader.Read())
{
for (int i = 0; i < reader.FieldCount; i++)
Console.Write(reader.GetValue(i).ToString() + " ");
Console.WriteLine();
}
connection.Close();
}
Now I'd like to view the results in a DataGridView. All the tutorials I've found involve adding external data sources to the grid, which I have no idea how to do in MySQL. (Also please note that I have no experience in developing Windows Forms, but I guess that most GUI development is drag-and-drop anyway).

As Daniel Said, a DataTable would be sufficient for this.
If you use a DataAdapter you can fill a DataTable and then bind this to your grid, e.g.:
DataGridView.DataSource = DataTable
If you set the DataGridView to auto generate columns then you will see each column in the data table, else, you need to specify each column.
Here is the code to populate a data table from a SQL command:
using (SqlDataAdapter oSqlDataAdapter = new SqlDataAdapter(oSqlCommand))
{
DataTable oDataTable = new DataTable();
oSqlDataAdapter.Fill(oDataTable);
return oDataTable;
}
Obviously you would use MySQL classes instead of SQL classes.

the best way to learn about this is to learn about data tables and datasets. This is pretty much the same across the board. You can do it drag and drop in visual studio but it is best to have more control over it.
this is an excellent tutorial in 4 parts
http://www.codeproject.com/KB/grid/practicalguidedatagrids1.aspx

If I'm not wrong, the mysql connector for .net has MySqlAdapter class that you can use to get a DataSet and then put the information into a Datatable or a Grid in the way WraithNath 've said.