Extremely slow mysql connection establishment only when called from code - c#

I am currently trying to do something that should be simple and straight-forward - connect to a database server, run a query, see if I get anything back and if so send it back to the user. This is the code I'm using to do it:
MySqlDataReader reader = MySqlHeaper.ExecuteReader(connectionString, $"SELECT * FROM table WHERE insertDateTime > '{DateTime.Now.AddSeconds(-1800).ToString("yyyy-MM-ddTHH:mm:ss")}'";
I have also tried this with a MySqlCommand and MySqlConnection object pair, and either way the result is the same - it takes approximately 7100ms to connect to the MySql server. I know that sounds like a problem that should be on ServerFault, but my testing tells me otherwise. When I use the command line MySql client to connect to my database server using exactly the same credentials and run exactly the same query I get my connection established and my data back in nothing flat. I don't know at this stage if it's a server setting or not, but here's what I've tried so far:
Rebooting the server
Restarting the MySQL server
Setting the skip_name_resolve setting to 1 in order to prevent reverse name lookups on connect
Using alternative means of querying the server (mysql command line client and MySQL Workbench)
Opening all AWS IAM permissions on the RDS instance to allow everything from the server
Nothing seems to be making any difference, so I'm at a loss to explain this terrible performance. It's also only happening when I open the connection. Running queries, inserts, what have you is lightning fast. Any suggestions anyone might have would be most helpful.

I would not expect IAM permissions to have any impact on performance. I would expect them to be either successful or not successful.
I would execute some diagnostic protocols to get more information.
1) Try a subsequent query, to see if it is an issue with the stack being initialized. Are subsequent queries faster?
2) Try a query that is just an identity query. Something that doesn't require any sort of IO.
3) Try a query from a different platform (maybe a scripting language like ruby or php)
Once you answer those it should help you narrow it down.

This is most likely caused by Connector/NET executing a slow WMI query to query connection attributes when opening the connection; this is logged as MySQL bug 80030.
As far as I know, this isn't fixed in newer versions of the driver, but you can work around it by switching to MySqlConnector, an OSS MySQL ADO.NET library.

Related

Is there a better way to see MySql statements with parameters in C# using Visual Studios debugger?

I have recently been changing some C# programs to add proper parameterizing to some MySQL statements that had originally been written with concatenated strings. Invariably, I've run into some problems with my statements and I can't find a way to directly see the complete MySQL statement with parameters applied other than this workaround that I have where I pass the MySQL command to this:
private string getMySqlStatement(MySqlCommand cmd)
{
string result = cmd.CommandText.ToString();
foreach (MySqlParameter p in cmd.Parameters)
{
string addQuote = (p.Value is string) ? "'" : "";
result = result.Replace(p.ParameterName.ToString(), addQuote + p.Value.ToString() + addQuote);
}
return result;
}
This works, but I was wondering if there was a more proper way to see the full statement with parameters applied. Reading up on this, it looks like the parameters aren't actually applied until it reaches the server - is this correct? In that case, I suppose I can stick to my function above, but I just wanted to know if there was a better way to do it.
Note: I am just using this function for debugging purposes so I can see the MySQL statement.
MySQL supports two protocols for client/server communication: text and binary. In the text protocol, there is no support for command parameters in the protocol itself; they are simulated by the client library. With Connector/NET, the text protocol is always used, unless you set IgnorePrepare=true in the connection string and call MySqlCommand.Prepare() for each command. So it's most likely the case that you are using the text protocol. This is good, because it will be easier to log the actual statements with parameters applied.
There are three ways to view the statements being executed:
Use Connector/NET Logging
Add Logging=true to your connection string and create a TraceListener that listens for the QueryOpened event. This should contain the full SQL statement with parameters interpolated. Instructions on setting this up are here.
Use MySQL Server Logging
Enable the general query log on your server to see all queries that are being executed. This is done with the --general_log=1 --general_log_file=/var/path/to/file server options.
Packet Sniffing
If you're not using SslMode=Required (to encrypt the connection between client and server), then you can use WireShark to capture network traffic between your client and the server. WireShark has MySQL Protocol analysers that will inspect MySQL traffic and identify command packets (that contain SQL queries). This option is ideal if you aren't able to modify your client program nor change server logging settings.

.NET SQL Server connection issue - Maybe connection pool related

I am having a very strange problem and am hoping someone out there has had a similar experience.
My companies application for one client is getting "banned" from the SQL Server at the beginning of our application. The behavior is strange. I'll write it out in point form.
SQL Connections are created, data is retrieved, the connections are closed, talk to another datasource and then denied access to SQL Server.
Here's the long winded version:
.NET application connects to database multiple times. Gets some data, does some work. It then goes to get some more data and then gets an error that the "SQL Server cannot be found or access is denied". If the process is started over again without re-starting the app then no more connections are able to be made to SQL Server. All new connections result in "SQL Server cannot be found or access is denied". If the application is restarted then it will repeat the above process.
This is the first in 5 years of my experience with the software to have this problem. The application does have code written in Delphi 7. The dephi 7 / VBA code has not issues. My .NET code that performs the actual query looks like:
protected abstract DbConnection GetConnection();
protected abstract DbDataAdapter GetDataAdapter(DbCommand cmd);
protected abstract DbCommand GetCommand(DbConnection conn, String sql);
protected abstract DbCommandBuilder GetCommandBuilder(DbDataAdapter adapter);
public virtual DataTable Query(string sql)
{
var dt = new DataTable();
using (var conn = GetConnection())
{
try
{
using (var cmd = GetCommand(conn, sql))
{
using (var adapter = GetDataAdapter(cmd))
{
adapter.Fill(dt);
}
}
}
catch (Exception ex)
{
throw new SqlStatementException(sql, ex);
}
}
return dt;
}
It is my own quite and dirty DAL. When it is used it is using an OleDbConnection.
Note: Due to legacy code the connection string is configured for OleDbConnection. After taking a moment to review my code I do have the ability to change the connection type to SqlConnection. I haven't tried that yet.
On the client's machine I have not been able to reproduce the issue outside of the main application. I tried creating a little app that would make 100 calls back to back using the format above with an OleDbConnection but it executed successfully.
The failure in the main app happens in the same spot. That should give me a clue except I cannot make sense of it since it is making duplicate query, getting the same data. But I will say that the application talks to two data sources and transfers data from one to the other. Before it does the transfer it does some validation on the sources. So it talks to another database (proprietary file based) via ODBC and comes back successfully and then fails when trying to talk to SQL Server through OleDbConnection.
My suspicion is something is happening in the connection pool. That is causing a failure which in turns causes a denial of access.
Other interesting points. All worked fine for about a year, client got a new machine a couple of months ago, all work fine and then suddenly stopped. I put the application on another machine at the client's site and all worked well for a week and then the same issue appeared. We turned everything off on the client's machine but the issue persisted. I thought firewall but no luck there.
Any assistance is greatly appreciated.
Was gonna put this in a comment, but it got too big :-)
I see your connection-creating methods are abstract. This of course means that derivatives can do all sorts of bad things when they create the connection. I'd look there first.
One thing I found in a similar situation...if you're doing something in the code that creates the connection that makes the connection string unique, you won't be reusing those pooled connections. So...doing something like adding an "App=MyApp" + an incrementing number, date/time, or guid, it will destroy your ability to use pooled connections. When this happened to me, it took me forever to figure it out.
If your application was "slow enough" in the past, such that "old" pooled connections fall out of the pool, you might never see a problem...but then, say a customer gets hot new hardware...and blam...weird errors from nowhere! This might not be what's happening to you, but maybe it will give you some ideas about where to look. Good luck!

ODBC ISAM_EOF without any reason

I have a strange behavior with an ODBC Driver and the underlying COBOL database. The database driver is acuODBC of AcuCorp(now Microfocus), the database itself is a COBOL database.
The DSN is a system DSN and works just fine. Via Access/Excel u can read/write data with the DSN successfully.
In C# the ODBCConnection.ConnectionState is open. Retreiving the table headers works just fine. But when reading data, a strange error occurs. Here is a summerize of the trace log:
[Retreiving Column Headers]
ISAMRestrict - NO_ISAM_ERR
ISAMRewind - NO_ISAM_ERR
ISAMNextRecord – ISAM_EOF
This occurs when I add a WHERE clausel to the SELECT statement. When I do not add a WHERE clausel and just retreive the whole table, it takes incredible long (about 12 minutes for 40000 records), but at least I retreive data.
So my question would now be, has anyone else occured such a strange behavior with an ODBC driver? End of File where data should be?
Just as a side note, I have contacted Microfocus too, if they have a solution I will post it here.
It seems to be Windows UAC reliant. As our application run in compatibility mode, UAC visualization is active and causing may some problems. The reason for this is, that the COBOL databse is a file based database, and the client where are coding for uses these files in ODBC DSN config directly instead of running an ODBC server to handle the requests.
So the UAC leads to some strange behavior.

SQL Server: Could not find prepared statement with handle x

Recently our QA team reported a very interesting bug in one of our applications. Our application is a C# .Net 3.5 SP1 based application interacting with a SQL Server 2005 Express Edition database.
By design the application is developed to detect database offline scenarios and if so to wait until the database is online (by retrying to connect in a timely manner) and once online, reconnect and resume functionality.
What our QA team did was, while the application is retrieving a bulk of data from the database, stop the database server, wait for a while and restart the database. Once the database restarts the application reconnects to the database without any issues but it started to continuously report the exception "Could not find prepared statement with handle x" (x is some number).
Our application is using prepared statements and it is already designed to call the Prepare() method again on all the SqlCommand objects when the application reconnects to the database. For example,
At application startup,
SqlCommand _commandA = connection.CreateCommand();
_commandA.CommandText = #"SELECT COMPANYNAME FROM TBCOMPANY WHERE ID = #ID";
_commandA.CommandType = CommandType.Text;
SqlParameter _paramA = _commandA.CreateParameter();
_paramA.ParameterName = "#ID";
_paramA.SqlDbType = SqlDbType.Int;
_paramA.Direction = ParameterDirection.Input;
_paramA.Size = 0;
_commandA.Parameters.Add(_paramA);
_commandA.Prepare();
After that we use ExceuteReader() on this _commandA with different #ID parameter values in each cycle of the application.
Once the application detects the database going offline and coming back online, upon reconnect to the database the application only executes,
_commandA.Prepare();
Two more strange things we noticed.
1. The above situation on happens with CommandType.Text type commands in the code. Our application also uses the same exact logic to invoke stored procedures but we never get this issue with stored procedures.
2. Up to now we were unable to reproduce this issue no matter how many different ways we try it in the Debug mode in Visual Studio.
Thanks in advance..
I think with almost 3 days of asking the question and close to 20 views of the question and 1 answer, I have to conclude that this is not a scenario that we can handle in the way we have tried with SQL server.
The best way to mitigate this issue in your application is to re-create the SqlCommand object instance again once the application detects that the database is online.
We did the change in our application and our QA team is happy about this modification since it provided the best (or maybe the only) fix for the issue they reported.
A final thanks to everyone who viewed and answered the question.
The server caches the query plan when you call 'command.Prepare'. The error indicates that it cannot find this cached query plan when you invoke 'Prepare' again. Try creating a new 'SqlCommand' instance and invoking the query on it. I've experienced this exception before and it fixes itself when the server refreshes the cache. I doubt there is anything that can be done programmatically on the client side, to fix this.
This is not necessarily related exactly to your problem but I'm posting this as I have spent a couple of days trying to fix the same error message in my application. We have a Java application using a C3P0 connection pool, JTDS driver, connecting to a SQL Server database.
We had disabled statement caching in our the C3P0 connection pool, but had not done this on the driver level. Adding maxStatements=0 to our connection URL stopped the driver caching statements, and fixed the error.

Turning on multiple result sets in an ODBC connection to SQL Server

I have an application that originally needed to connect to Sybase (via ODBC), but I've needed to add the ability to connect to SQL Server as well. As ODBC should be able to handle both, I thought I was in a good position.
Unfort, SQL Server will not let me, by default, nest ODBC commands and ODBCDataReaders - it complains the connection is busy (Connection is busy with results for another command).
I know that I had to specify that multiple active result sets (MARS) were allowed in similar circumstances when connecting to SQL Server via a native driver, so I thought it wouldn't be an issue.
The DSN wizard has no entr
y when creating a SystemDSN.
Some people have provided registry hacks to get around this, but this did not work (add a MARS_Connection with a value of Yes to HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\system-dsn-name).
Another suggestion was to create a file-dsn, and add "MARS_Connection=YES" to that. Didn't work.
Finally, a DSN-less connection string. I've tried this one (using MultipleActiveResultSets - same variable as a Sql Server connection would use),
"Driver={SQL Native Client};Server=xxx.xxx.xxx.xxx;Database=someDB;Uid=u;Pwd=p;MultipleActiveResultSets=True;"
and this one:
"Driver={SQL Native Client};Server=192.168.75.33\\ARIA;Database=Aria;Uid=sa;Pwd=service;MARS_Connection=YES;"
I have checked the various connection-string sites - they all suggest what I've already tried.
I should state that I've tried both the SQL Server driver, and the SQL Server native driver...
According to the SNI documentation on Using Multiple Active Result Sets (MARS):
The SQL Server Native Client ODBC
driver supports MARS through additions
to the SQLSetConnectAttr and
SQLGetConnectAttr functions.
SQL_COPT_SS_MARS_ENABLED has been
added to accept either
SQL_MARS_ENABLED_YES or
SQL_MARS_ENABLED_NO, with
SQL_MARS_ENABLED_NO being the default.
In addition, a new connection string
keyword, Mars_Connection, as been
added. It accepts "yes" or "no"
values; "no" is the default.
Make sure your client loads the right drivers, use Mars_Connection=yes, and validate in the app by checking SQL_COPT_SS_MARS_ENABLED on SQLGetConnectAttr.

Categories