We Keep Coding

MySql command "load data infile" execution error in C#

I am trying to import a text file into MySql database using C# code but getting errors.
My table structure is:
and the C# code that I'm executing is:
fileQuery =
"load data infile '{0}' into table dgl.deliveries fields terminated by '\t' lines terminated by \r\n' (#ImagePath, Delivery_Note, Shipment_Number, #Delivery_Date, Deliver_To_Code, Deliver_To_Name, Sold_To_Code, Sold_To_Name, Material_Number, Doctype) set Delivery_Date = tr_to_date(#Delivery_Date, '%d/%m/%Y'), ImagePath = Concat('USERFILES/', #ImagePath)";
string q = string.Format(fileQuery,fileName);
MySqlConnection conn = new MySqlConnection(dglConnection.ConnectionString);
MySqlCommand command = conn.CreateCommand();
command.CommandText = q;
conn.Open();
command.ExecuteNonQuery();
and the error is:
An exception of type 'MySql.Data.MySqlClient.MySqlException' occurred in MySql.Data.DLL but was not handled in user code
Additional information: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%d/%m/%Y'), ImagePath = Concat('USERFILES/', #ImagePath)' at line 2
The following is a line from source input file:
123.pdf 802661341 1061611 18/02/2015 00:00:00 22280 ABC LIMITED 22280 XYZ LIMITED 30679795 30744488 DELIVERY NOTE 1

Your problem is that Your Date that you are passing from your C# code is 18/02/2015. Mysql only excepts a date in the format YYYY-MM-DD. You need to adjust that data so that it formats to the way Mysql will except a date if you want to store it as a date.
I actually wrote a stored procedure that you maybe able to use (or at least get an idea of what needs to be done): Here is the link.
Also when in doubt just refer to dev.mysql its a great resource also.

SMO fails to execute script with user-defined table type

I am trying to execute a sql script read from a file; however, when I go to execute the script I am met with the exception:
Column, parameter, or variable #1: Cannot find data type dbo.ExistingTableType.
Must declare the table variable "#existingTableType".
Parameter or variable '#existingTableType' has an invalid data type.
I have tried testing the script by copying the text and running it in SQL server management studio and it does run successfully. Additionally, other scripts are able to run but for some reason those with a user-defined table type as a parameter are not. Below is a simple example:
C#
string script = scriptFile.OpenText().ReadToEnd();
SqlConnection connection = new SqlConnection(connectionString);
Server server = new Server(new ServerConnection(connection));
server.ConnectionContext.StatementTimeout = 100;
server.ConnectionContext.ExecuteNonQuery(script);
SQL
CREATE PROCEDURE [dbo].[failedStoredProcedure]
(
#existingTableType dbo.ExistingTableType READONLY
)
AS
-- Do something here
GO
Other tests done:
Tried another user in connection string.
Edit: Additionally, when scripts run it returns 'Invalid Object Name {x}' when the object does exist.

I tried your code on my end (SQL 2012 Express) and it worked just fine. Does the sql account has grants to the type?

It seems that at some point something happens to the SMO object where it no longer applies to the specified database, despite the database name being the intended object.
To correct this I created a new database object and applied the scripts from there:
Server server = new Server();
server.ConnectionContext.ConnectionString = connectionString;
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder(connectionString);
Database db = server.Databases[builder.InitialCatalog];
Then later instead run
db.ExecuteNonQuery(script);

Paramertized SQL, Oracle 11g, and C# Failing to create index

I am trying to use a parametrized query to create an index on our Oracle server. I can create the index fine if I use string concatenation, so I believe its not an account or permissions issue. I get the error:
ORA-01036: illegal variable name/number
I can not find any error in the code, but I am sure I am missing something. I am using Oracle.DataAccess.dll version 4.112.3.0. The version the server reports is "Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production"
I have tried removing all the params, except one and get the same error. Any combination of 1 or more params causes the error. The error happens every time without fail. I have used watches to verify that the params are correctly getting set, and I have tried clearing the params before setting. No matter what I do so far, I still get the error. Here is the code:
using (OracleCommand mycom = new OracleCommand())
{
using (OracleConnection Connection = new OracleConnection(connectionString))
{
mycom.BindByName = true;
mycom.CommandText = "CREATE INDEX :indexname on :tablename (:colname)";
mycom.Parameters.Add("indexname", indexName);
mycom.Parameters.Add("tablename", tableName);
mycom.Parameters.Add("colname", colName);
mycom.Connection = Connection;
mycom.CommandType = CommandType.Text;
mycom.Connection.Open();
mycom.ExecuteNonQuery();
}
}

You cannot use bind variables in a DDL statement (like CREATE INDEX). Bind variables are only used in DML statements. You would need to build up the DDL statement in your C# code.
Is this a part of the installation process for your application? Otherwise, it would be exceedingly unusual to want to create an index from an application. Your application shouldn't be creating any database objects outside of the initial installation process.

How do I set encoding in an NpgsqlConnection

I have a PostgreSQL database, which uses character encoding WIN1252.
When querying the database, some records will produce an error when trying to read the data, because it is trying to convert it to UTF8. This happens on some foreign names containing certain non-Latin characters.
The error is:
ERROR: 22P05: character with byte sequence 0x81 in encoding "WIN1252" has no equivalent in encoding "UTF8"
It happens when I call Read() on the NpgsqlDataReader.
My connection is defined as:
new NpgsqlConnection("Server=127.0.0.1;Port=5432;Database=xyz;User Id=****;Password=****;");
What can I do to read this data using C#?

I've managed to solve the problem. There is no way of setting the property in the connection string or any of the properties of the NpgsqlConnection or NpgsqlCommand.
However, I was able to set the value of client_encoding in a query. So directly after opening the connection I first executed the (non)query:
set client_encoding = 'WIN1252'
After that, any subsequent command on the same connection used the proper encoding and returned the results without complaints.

I tried to change the connection string but i had no luck with that.
The problem got solved by chaning the database settings file and reload it.
So i started pgadmin and executed
SHOW config_file;
which gave me
C:/Program Files/PostgreSQL/14/data/postgresql.conf
in this file i changed the lc_messages from lang_language.1252 to UTF8.
After that i reloaded this config in pg admin by right click on the server name and press "Reload Configuration".
All settings are now set to UTF8 and it just worked fine.
lc_messages = 'UTF8' # locale for system error message
# strings
lc_monetary = 'UTF8' # locale for monetary formatting
lc_numeric = 'UTF8' # locale for number formatting
lc_time = 'UTF8' # locale for time formatting
...

Issue with MySqlBulkLoader from C# app

I am writing an app to export some data from MSSQL to a MySQL server on a nightly basis. I use a simple query to grab all the data for the previous day and have then tried a few different approaches for getting it to MySQL. The fastest approach is using MySqlBulkLoader but for some reason it isn't moving all the data. After I do the insert, I am comparing the records in the text file that was generated to the number of records that are in MySQL and the counts are off from 1 in some cases all the way up to 10.
If I do the same approach for getting the data to the text file but loop through each row of the text file rather than bulk upload and do insert statements, all the records get imported.
Here is the bulk upload code I am currently using. I recently added the FieldQuotationCharacter to see if that would help and it didn't (when I added that, I made the text generation script enclose fields in quotes).
uploader.TableName = "testtable";
uploader.FieldTerminator = "\t";
uploader.LineTerminator = "\r\n";
uploader.NumberOfLinesToSkip = 0;
uploader.FileName = updateFile; //this is a variable pointing to the current file
uploader.Timeout = 120;
uploader.FieldQuotationCharacter = '"';
int totalExported = uploader.Load();
Any ideas?

Seems a little strange but on a whim I decided to write my output file with a blank line at the top and then set NumberOfLinesToSkip = 1. After doing this, everything worked and no records went missing. Kinda strange. Seems that setting it to 0 didn't work and may not be supported.

I was having a similar problem where it was loading about 2/3 of the rows in my file and the problem turned out to be the behavior of the local keyword in the load data sql that is generated by MySqlBulkLoader.
By default, it seems that MySqlBulkLoader.Local == true, this allows it to use a local file, but also affects the error handling for invalid rows. Instead of throwing an error when a row is invalid, it will give you "warnings". You can verify this and view the warning by running an actual sql command for load data (which is what MySqlBulkLoader uses):
load data local infile '/Temp/bb7dd81c-c79f-49c7-9ae4-fdc8e48df6d5.csv'
ignore into table my_staging_tbl
fields terminated by ',' enclosed by '"' escaped by '\\'
lines terminated by '\n'
This outputs the count of affected rows and a list of all warnings
In my case, I had the LineTerminator as "\n" instead of "\r\n" and after the warnings were suppressed it still ended up importing most of the rows.
You can override this behavior by setting MySqlBulkLoader.ConflictOption to MySqlBulkLoaderConflictOption.Replace instead of Ignore which is the default. Note that this also affects the way it handles duplicate keys.
More on the load data stuff for mysql: http://dev.mysql.com/doc/refman/5.7/en/load-data.html

If you stumble upon this post looking for a solution to trap those warnings, here it is. You actually need to add an event handler on the underlying connection to grab them. Like this:
connection.InfoMessage += new MySqlInfoMessageEventHandler(OnInfoMessage);
connection.Open();
MySqlBulkLoader bulkLoader = new MySqlBulkLoader(connection);
void OnInfoMessage(object sender, MySqlInfoMessageEventArgs e) {
MySqlCommand myCommand = new MySqlCommand("SHOW WARNINGS", (MySqlConnection)sender);
MySqlDataReader reader = myCommand.ExecuteReader();
while (reader.Read()) {
Console.WriteLine(reader[0].ToString() + " " + reader[1].ToString() + " " + reader[2].ToString());
}
}