Method with MySql query for LOAD DATA IN FILE never finishes - c#

I am populating a MySQL table in C# using the LOAD DATA INFILE method.
Therefore I wrote a method for creating the query string...
public static string LoadDataInFile(string TempCsv, string MatchId)
{
StringBuilder MySqlString = new StringBuilder();
MySqlString.AppendFormat(#"LOAD DATA INFILE '{0}' INTO TABLE {1}_trackingdata
FIELDS TERMINATED BY ';'
IGNORE 1 LINES", TempCsv, MatchId);
return MySqlString.ToString();
}
and one that executes raw queries:
public static void ExecuteRawQuery(string Query, string ConnectionString)
{
using (MySqlConnection conn = new MySqlConnection(ConnectionString.ToString()))
{
try
{
conn.Open();
MySqlCommand cmd = new MySqlCommand(Query, conn);
cmd.ExecuteNonQuery();
conn.Close();
Console.WriteLine("Import probably done...");
return;
}
catch (MySqlException SqlException)
{
Console.ForegroundColor = ConsoleColor.DarkRed;
Console.WriteLine("SqlException: {0}", SqlException.Message);
Console.WriteLine("Exit program.");
Console.ResetColor();
Environment.Exit(0);
}
}
}
The table gets populated as expected (checked with Sequel Pro), but the application never continues.
public static void Main(string[] args)
{
// ...
ExecuteRawQuery(LoadDataInFile("myFile.csv", "123456"));
Console.WriteLine("I will never be displayed..");
// ...
}
Is there any reason why ExecuteRawQuery() never finishes? When I execute other queries (SELECT, UPDATE) everything works as expected.

I don't know why, but using the MySqlClient.MySqlBulkLoader-class solves the problem. Leaving this here for anybody experiencing the same issue:
public static void ImportCSV(string CSV, string TableName, string ConnectionString)
{
using (MySqlConnection conn = new MySqlConnection(ConnectionString.ToString()))
{
MySqlBulkLoader Bulkloader = new MySqlBulkLoader(conn);
Bulkloader.TableName = TableName;
Bulkloader.FileName = CSV;
Bulkloader.Timeout = 0;
Bulkloader.NumberOfLinesToSkip = 1;
Bulkloader.FieldTerminator = ";";
var result = Bulkloader.Load();
Console.WriteLine("Imported {0} rows into the database", result);
}
}

Related

is there a way of returning a bool value from db.Execute(sql) in C# mysql?

I am fiddling around with some mysql commands and would like to know how I can return a bool if my command was successful, here is some example code...
public static bool RunSQL(string SQL)
{
try
{
// Info: db is defined in my dependency service!
bool execute = false;
Console.WriteLine(SQL);
execute = Convert.ToBoolean(db.Execute(SQL));
db.Close();
return execute;
}
catch (Exception ex)
{
GlobalErrorList.ErrorControler(ex.Message, "DatabaseManager", SQL, "RunSQL");
return false;
}
}
My issue here is that db.Execute(sql) always returns 0 even when the command is successful, this I have checked in my database, for example the sql string is a string that is defined to create a table, it works fine, but the db.Execute(sql) returns 0;
My question is... Can anyone help my find a reliable way of returning the value of the executed string?
I.E true if was successful false if not!
According to your description, you want to return true or false when execute the sql sentence.
I used SQLiteCommand.ExecuteScalar() method to check it.
You can try the following code to solve this problem.
class Program
{
SQLiteConnection m_dbConnection;
static void Main(string[] args)
{
Program p = new Program();
string sql1 = "create table scores(name varchar(20),score int)";
string sql2 = "insert into scores(name,score) values ('tom',90),('aya',97)";
string sql3 = "delete from scores where name='tom'";
string sql4 = "select * from scores where name='tom'";
bool a = p.operate(sql1);
bool b = p.operate(sql2);
bool c = p.operate(sql3);
bool d = p.operate(sql4);
Console.WriteLine(a.ToString());
Console.WriteLine(b.ToString());
Console.WriteLine(c.ToString());
Console.WriteLine(d.ToString());
Console.ReadKey();
}
//Add, delete, modify, query and return results
bool operate(string SQL)
{
m_dbConnection = new SQLiteConnection("Data Source=MyDatabase.sqlite;Version=3;");
m_dbConnection.Open();
SQLiteCommand command = new SQLiteCommand(SQL, m_dbConnection);
try
{
if (null == command.ExecuteScalar())
{
return true;
}
}
catch (Exception e)
{
return false;
}
return false;
}

Sqlite DataAdapter isn't updating when called

I have to write identical tables to two different Sqlite database. The tables rewrite existing data and ids, so I currently just delete the entire table and rewrite. Depending on the order that I write to the db, the C# SQLiteCommandBuilder does not update the second db called.
If I call Db_A before Db_B, then A gets written and B gets deleted and vice versa. Can anyone tell me why the second table enters the code to get deleted, but the Sqlite adapter never updates the second table? It doesn't throw an error either.
public static bool WriteDt_Name(DataTable dt)
{
using (connA = GetDbAConn())
{
SaveDataTable(connA, dt);
}
using (connB = GetDbBConn())
{
SaveDataTable(connB, dt);
}
return true;
}
public static void SaveDataTable(SQLiteConnection conn, DataTable dt)
{
string table = dt.TableName;
var cmd = conn.CreateCommand();
cmd.CommandText = string.Format("DELETE FROM {0}", table);
int val = cmd.ExecuteNonQuery();
cmd.CommandText = string.Format("SELECT * FROM {0}", table);
using (var adapter = new SQLiteDataAdapter(cmd))
{
using (SQLiteCommandBuilder builder = new SQLiteCommandBuilder(adapter))
{
adapter.Update(dt);
conn.Close();
}
}
}
public static SQLiteConnection GetDbAConn()
{
string base_dir = System.AppDomain.CurrentDomain.BaseDirectory;
string path = Directory.GetParent(Directory.GetParent(Directory.GetParent(Directory.GetParent(Directory.GetParent(base_dir).ToString()).ToString()).ToString()).ToString()).ToString();
path = path + "\\db\\DbA.sqlite;";
SQLiteConnection conn = new SQLiteConnection("Data Source=" + path + "Version=3;");
return conn;
}
I have tried splitting SaveDataTable into a SaveDt_A and SaveDt_B and calling it that way. I still get the same result.

C# Mysql Connection must be valid and open

First of all: I got my code running without using oop. I declared all my variables inside the same class and opened/closed the connection right before and after passing the query to the db. That worked! Now with some new experiences I tried to split my code into different classes. Now it wont work anymore.
It tells me "Connection must be valid and open". Enough text, here's my current code:
Services.cs
public static MySqlConnection conn // Returns the connection itself
{
get
{
MySqlConnection conn = new MySqlConnection(Services.ServerConnection);
return conn;
}
}
public static string ServerConnection // Returns the connectin-string
{
get
{
return String.Format("Server={0};Port=XXXX;Database=xxx;Uid=xxx;password=xxXxxXxXxxXxxXX;", key);
}
}
public static void DB_Select(string s, params List<string>[] lists)
{
try
{
MySqlCommand cmd = conn.CreateCommand();
cmd.CommandType = CommandType.Text;
string command = s;
cmd.CommandText = command;
MySqlDataReader sqlreader = cmd.ExecuteReader();
while (sqlreader.Read())
{
if (sqlreader[0].ToString().Length > 0)
{
for (int i = 0; i < lists.Count(); i++)
{
lists[i].Add(sqlreader[i].ToString());
}
}
else
{
foreach (List<string> save in lists)
{
save.Add("/");
}
}
}
sqlreader.Close();
}
catch (Exception ex)
{
MessageBox.Show("Error while selecting data from database!\nDetails: " + ex);
}
}
LoginForm.cs
private void checkUser(string username, string password)
{
using (Services.conn)
{
Services.conn.Open();
Services.DB_Select("..a short select statement..");
Services.conn.Close();
}
I guess this is all we need. I have shortened my code to get a focus on the problem.
I created Services.cs to get a global way to access the db from all forms without copy&pasting the connection info. Now when I reach my LoginForm.cs it throws an error "Connection must be valid and open". I've already debugged my code. It's all time closed. Even when passing conn.Open() it stays closed. Why?
Another try: I've also tried placing conn.Open() and conn.Close() inside Services.DB_Select(..) at the beginning and end. Same error here.
I have to say: The code worked before and I've used the same connection-string. So the string itself is surely valid.
I appreciate any help given here!
The problem is that you don't store the connection that was returned from your factory property. But don't use a property like a method. Instead use it in this way:
using (var con = Services.conn)
{
Services.conn.Open();
Services.DB_Select("..a short select statement..", con ));
//Services.conn.Close(); unnecessary with using
}
So use the same connection in the using that was returned from the property(or better created in the using) and pass it to the method which uses it. By the way, using a property as factory method is not best practise.
But in my opinion it's much better to create the connection where you use it, best place is in the using statement. And throw the con property to the garbage can, it is pointless and a source for nasty errors.
public static void DB_Select(string s, params List<string>[] lists)
{
try
{
using(var conn = new MySqlConnection(Services.ServerConnection))
{
conn.Open();
MySqlCommand cmd = conn.CreateCommand();
cmd.CommandText = s;
using( var sqlreader = cmd.ExecuteReader())
while (sqlreader.Read())
{
if (sqlreader[0].ToString().Length > 0)
{
for (int i = 0; i < lists.Count(); i++)
{
lists[i].Add(sqlreader[i].ToString());
}
}
else
{
foreach (List<string> save in lists)
{
save.Add("/");
}
}
} // unnecessary to close the connection
} // or the reader with the using-stetement
}
catch (Exception ex)
{
MessageBox.Show("Error while selecting data from database!\nDetails: " + ex);
}
}
Try to restructure your Services class as follows
public static MySqlConnection conn // Returns the connection itself
{
get
{
MySqlConnection conn = new MySqlConnection(Services.ServerConnection);
return conn;
}
}
private static string ServerConnection // Returns the connectin-string - PRIVATE [Improved security]
{
get
{
return String.Format("Server={0};Port=XXXX;Database=xxx;Uid=xxx;password=xxXxxXxXxxXxxXX;", key);
}
}
// Rather than executing result here, return the result to LoginForm - Future improvement
public static void DB_Select(MySqlConnection conn ,string s, params List<string>[] lists)
{
try
{
MySqlCommand cmd = conn.CreateCommand();
cmd.CommandType = CommandType.Text;
string command = s;
cmd.CommandText = command;
MySqlDataReader sqlreader = cmd.ExecuteReader();
while (sqlreader.Read())
{
if (sqlreader[0].ToString().Length > 0)
{
for (int i = 0; i < lists.Count(); i++)
{
lists[i].Add(sqlreader[i].ToString());
}
}
else
{
foreach (List<string> save in lists)
{
save.Add("/");
}
}
}
sqlreader.Close();
}
catch (Exception ex)
{
MessageBox.Show("Error while selecting data from database!\nDetails: " + ex);
}
}
In LoginForm.cs use returning connection and store it there. When you need to execute query, use
MySqlConnection conn=Services.conn(); // Get a new connection
Services.DB_Select(conn,"..a short select statement.."); // Executing requirement
Services.conn.Close();
Additional - I suggest you need to return MySqlDataReader to LoginForm and handle results there
private MySqlConnection _conn;
public MySqlConnection conn // Returns the connection itself
{
get
{
if(_conn == null)
_conn = new MySqlConnection(Services.ServerConnection);
return _conn;
}
}

Delete from database only works when debugging

I am reading rows from a database which contains BLOB's. I take each of these blobs and saves them as a file to my disk. Afterwards I want to delete the rows I have just read. Now the problem: When I debug, all works like a charm. When I run without having a debug breakpoint, it doesn't delete anything! And it doesn't come up with an error.
I use C# and MS-SQL server.
Here is my code:
class Program
{
static void Main(string[] args)
{
if (args[0] == "/?" || args.Length != 1)
{
Console.WriteLine("BlobReader");
Console.WriteLine("Will read blob from database and write is as a file to destination specified in database.");
Console.WriteLine();
Console.WriteLine("BlobReader <AppName>");
Console.WriteLine();
Console.WriteLine("<AppName>: Application name which identifies which files to extract and save.");
EndProgram();
}
string now = DateTime.Now.ToString("yyyy-MM-dd hh:mm:ss.fff");
Console.WriteLine("writing files to disk");
bool success = SqlBlob2File(args[0], now);
if (success)
{
Console.WriteLine("Deleting db");
DeleteRows(args[0], now);
Console.WriteLine("Db deleted");
}
EndProgram();
}
static void EndProgram()
{
Console.WriteLine();
Console.WriteLine("Press any key to end.");
Console.ReadLine();
}
static private void DeleteRows(string app, string now)
{
try
{
string connectionString = ConfigurationManager.ConnectionStrings["dbConn"].ToString();
SqlConnection connection = new SqlConnection(connectionString);
string sql = string.Format("DELETE FROM Blobs WHERE Application = '{0}' AND CreatedDate < '{1}'", app,
now);
SqlCommand cmd = new SqlCommand(sql, connection);
connection.Open();
cmd.BeginExecuteNonQuery();
connection.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
static private bool SqlBlob2File(string app, string now)
{
bool success;
string connectionString = ConfigurationManager.ConnectionStrings["dbConn"].ToString();
SqlConnection connection = new SqlConnection(connectionString);
try
{
int blobCol = 0; // the column # of the BLOB field
string sql =
string.Format(
"SELECT Blob, Drive, Folder, FileName FROM Blobs WHERE Application='{0}' AND CreatedDate < '{1}'",
app, now);
SqlCommand cmd = new SqlCommand(sql, connection);
connection.Open();
SqlDataReader dr = cmd.ExecuteReader();
while (dr.Read())
{
string destFilePath = string.Format("{0}{1}{2}", dr["Drive"], dr["Folder"], dr["FileName"]);
Byte[] b = new Byte[(dr.GetBytes(blobCol, 0, null, 0, int.MaxValue))];
dr.GetBytes(blobCol, 0, b, 0, b.Length);
System.IO.FileStream fs = new System.IO.FileStream(destFilePath, System.IO.FileMode.Create,
System.IO.FileAccess.Write);
fs.Write(b, 0, b.Length);
fs.Close();
Console.WriteLine("Blob written to file successfully");
}
dr.Close();
success = true;
}
catch (SqlException ex)
{
success = false;
Console.WriteLine(ex.Message);
}
finally
{
connection.Close();
}
return success;
}
}
If I set up a breakpoint in method DeleteRows it does delete from the database. If I don't, nothing is deleted.
If you use this cmd.BeginExecuteNonQuery(),you have use EndExecuteNonquery().
Otherwise,it wont be deleted and it may create memory leak problem.
Best way is to use cmd.ExecuteNonQuery();
Does this also happen when you remove the Connection.Close? It might be that since you are removing async that the connection is closed before the operation can complete unlike when you have a breakpoint and the code execution waits.
Also instead of using SqlConnection the way you do, you might want to use:
using (sqlConnection con = new SqlConnection())
{
// your code
}
This way the connection is automatically closed when it is out of scope or when something goes wrong.
I agree with MahaSwetha that cmd.ExecuteNonQuery() would make your life easier. Also cmd.ExecuteNonQuery() returns an int telling you how much rows have changed. Can come in handy sometimes.

reader only pulling one row in while loop C#

I have two problems. One is that it is only pulling one row and sending it to ms fax when there are a few hundred to be sent. The other is that it doesn't pull any more after that first and it throws an error. I thought I was closing out my connections. I don't understand what the problem is. I have included the code and error.
Service1.cs
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Linq;
using System.ServiceProcess;
using System.Text;
using System.Timers;
using MySql.Data.MySqlClient;
using FAXCOMLib;
using FAXCOMEXLib;
namespace ProcessFaxes
{
public partial class Service1 : ServiceBase
{
public Service1()
{
InitializeComponent();
}
public static Timer timer = new Timer();
protected override void OnStart(string[] args)
{
timer.Elapsed += new ElapsedEventHandler(Tick);
timer.Interval = 600000; // every 10 minutes
timer.Enabled = true;
// Console.ReadLine();
}
protected override void OnStop()
{
}
public static void Tick(object source, ElapsedEventArgs e)
{
string connString = "Server=localhost;Port=3306;Database=communications;Uid=root;password=pass;";
MySqlConnection conn = new MySqlConnection(connString);
MySqlCommand command = conn.CreateCommand();
MySqlConnection connupdate = new MySqlConnection(connString);
MySqlCommand commandupdate = connupdate.CreateCommand();
command.CommandText = "SELECT * FROM outbox WHERE `faxstat` = 'Y' AND `fax` <> '' AND `faxpro` = 'PENDING'";
//command.CommandText = "UPDATE blah blah";
//conn.Open();
//conn.ExecuteNonQuery();
//conn.Close();
try
{
conn.Open();
connupdate.Open();
}
catch (Exception ex)
{
// Console.WriteLine(Ex.Message);
LogException(ex.ToString());
throw; // or whatever you want to do with it
}
MySqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
while (reader.Read())
{
//Console.WriteLine(reader["filepath"].ToString());
SendFax(reader["id"].ToString(), reader["filepath"].ToString(), #"C:\FAXDOC\" + reader["filepath"].ToString(), reader["account"].ToString(), reader["fax"].ToString(), reader["fax_orig"].ToString());
string id = reader["id"].ToString();
commandupdate.CommandText = "UPDATE outbox SET `faxpro` = 'DONE' WHERE `id` = '" + id + "'";
commandupdate.ExecuteNonQuery();
}
}
conn.Close();
connupdate.Close();
}
public static void SendFax(string DocumentId, string DocumentName, string FileName, string RecipientName, string FaxNumber, string RecipientHomePhone2)
{
if (FaxNumber != "")
{
try
{
FAXCOMLib.FaxServer faxServer = new FAXCOMLib.FaxServerClass();
faxServer.Connect(Environment.MachineName);
FAXCOMLib.FaxDoc faxDoc = (FAXCOMLib.FaxDoc)faxServer.CreateDocument(FileName);
faxDoc.RecipientName = RecipientName;
faxDoc.FaxNumber = FaxNumber;
faxDoc.BillingCode = DocumentId;
faxDoc.DisplayName = DocumentName;
faxDoc.RecipientHomePhone = RecipientHomePhone2;
int Response = faxDoc.Send();
faxServer.Disconnect();
}
catch (Exception Ex) {
// Console.WriteLine(Ex.Message);
LogException(Ex.ToString());
throw; // or whatever you want to do with it
}
}
}
public static void LogException(string ErrorDescription)
{
// The name of our log in the event logs
string Log = "Process Faxes";
// Check to see fi the log for AspNetError exists on the machine
// If not, create it
if ((!(EventLog.SourceExists(Log))))
{
EventLog.CreateEventSource(Log, Log);
}
// Now insert your exception information into the AspNetError event log
EventLog logEntry = new EventLog();
logEntry.Source = Log;
logEntry.WriteEntry(ErrorDescription, EventLogEntryType.Error);
}
}
}
error
Event Type: Error
Event Source: Process Faxes
Event Category: None
Event ID: 0
Date: 3/6/2012
Time: 2:01:06 PM
User: N/A
Computer: FAXSERVER
Description:
MySql.Data.MySqlClient.MySqlException (0x80004005): Too many connections
at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
at MySql.Data.MySqlClient.NativeDriver.Open()
at MySql.Data.MySqlClient.Driver.Open()
at MySql.Data.MySqlClient.Driver.Create(MySqlConnectionStringBuilder settings)
at MySql.Data.MySqlClient.MySqlPool.GetPooledConnection()
at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver()
at MySql.Data.MySqlClient.MySqlPool.GetConnection()
at MySql.Data.MySqlClient.MySqlConnection.Open()
at ProcessFaxes.Service1.Tick(Object source, ElapsedEventArgs e) in C:\Documents and Settings\bruser\My Documents\Visual Studio 2010\Projects\ProcessFaxes\ProcessFaxes\Service1.cs:line 56
I think you should refactor a bit. I explained a little in a comment above, but here's how I would change it (I added some comments for you too):
public static void Tick(object source, ElapsedEventArgs e)
{
// Prevent another Tick from happening if this takes longer than 10 minutes
(source as Timer).Enabled = false;
// It would be better practice to put this in a settings or config file
// so you can change it without having to recompile your application
string connString = "Server=localhost;Port=3306;Database=communications;Uid=root;password=pass;";
// I won't change them here, but since these classes implement IDisposable,
// you should be using a using statement around them:
// using (var conn = new MySqlConnection(connString))
// {
// // use conn
// }
MySqlConnection conn = new MySqlConnection(connString);
MySqlCommand command = conn.CreateCommand();
MySqlCommand updateCommand = conn.CreateCommand();
command.CommandText = "SELECT * FROM outbox WHERE `faxstat` = 'Y' AND `fax` <> '' AND `faxpro` = 'PENDING'";
try
{
conn.Open();
MySqlDataReader reader = command.ExecuteReader();
if (reader.HasRows)
{
while (reader.Read())
{
SendFax(reader["id"].ToString(), reader["filepath"].ToString(), #"C:\FAXDOC\" + reader["filepath"].ToString(), reader["account"].ToString(), reader["fax"].ToString(), reader["fax_orig"].ToString());
string id = reader["id"].ToString();
// I would use a prepared statement with either this query
// or a stored procedure with parameters instead of manually
// building this string (more good practice than worrying about
// SQL injection as it's an internal app
updateCommand.CommandText = "UPDATE outbox SET `faxpro` = 'DONE' WHERE `id` = '" + id + "'";
updateCommand.ExecuteNonQuery();
}
}
}
catch (Exception ex)
{
LogException(ex.ToString());
throw;
}
finally
{
// If you're not going to use using-statements, you might want to explicitly
// call dispose on your disposable objects:
// command.Dispose();
// updateCommand.Dispose();
conn.Close();
// conn.Dispose();
}
// Enable the timer again
(source as Timer).Enabled = true;
}
As to why you're only receiving one row when you're expecting many, I suspect your SQL is at fault.
You should not be using a timer.
A timer fires at regular intervals, and does not care if the previous event has completed or not.
Look at using a Background Worker to send your faxes, having it loop over the queue, and then pause when the queue is empty.
The issue lies in your connection object. You have defined a couple of connection objects. You only need one.
Here are the two in question:
MySqlConnection connupdate = new MySqlConnection(connString);
MySqlConnection conn = new MySqlConnection(connString);
Eliminate the connection to one of them.
Here is one way to fix your code:
string connString = "Server=localhost;Port=3306;Database=communications;Uid=root;password=pass;";
using(MySqlConnection conn = new MySQLConnection(connString))
{
using(MySQlCommand command = conn.CreateCommand())
{
command.CommandText = "SELECT ...";
conn.Open();
using(MySqlDataReader reader = command.ExecuteReader())
{
//process rows...
}
}
}

Categories