I have a block of code intended to pull text descriptions from a database table and save them to a text file. It looks like this (C# .NET):
OdbcCommand getItemsCommand = new OdbcCommand("SELECT ID FROM ITEMS", databaseConnection);
OdbcDataReader getItemsReader = getItemsCommand.ExecuteReader();
OdbcCommand getDescriptionCommand = new OdbcCommand("SELECT ITEMDESCRIPTION FROM ITEMS WHERE ID = ?", databaseConnection);
getDescriptionCommand.Prepare();
while (getItemsReader.Read())
{
long id = getItemsReader.GetInt64(0);
String outputPath = "c:\\text\\" + id + ".txt";
if (!File.Exists(outputPath))
{
getDescriptionCommand.Parameters.Clear();
getDescriptionCommand.Parameters.AddWithValue("id", id);
String description = (String)getDescriptionCommand.ExecuteScalar();
StreamWriter outputWriter = new StreamWriter(outputPath);
outputWriter.Write(description);
outputWriter.Close();
}
}
getItemsReader.Close();
This code has successfully saved a portion of the data to .txt files, but for many rows, an AccessViolationException is thrown on the following line:
String description = (String)getDescriptionCommand.ExecuteScalar();
The Exception text is "Attempted to read or write protected memory. This is often an indication that other memory is corrupt".
The program will usually throw the exception on the same rows of the table, but it doesn't appear to be 100% consistent. Sometimes data that had thrown the exception in the past will suddenly work.
Some people are undoubtedly wondering why I didn't just SELECT ID, ITEMDESCRIPTION FROM ITEMS in the getItemsCommand and skip the second query. Actually, I did it that way initially, and I was encountering the same error with getItemsCommand.GetString(). I was afraid that perhaps the dataset was taking up too much memory and maybe that was causing the error. So I decided to try this method to see if it would help. It didn't. Does anyone know why this might be happening?
By the way, ID is an INT and ITEMDESCRIPTION is a VARCHAR(32000) column. If it makes any difference, the database is Borland Interbase 6.0 (Ick!)
EDIT: I gave the wrong line when describing where the exception was being thrown!! ARGH!! Fixed now. Also, I've tried the things suggested so far, but they didn't help. However, I found that only very old records in the database were causing this error, which is strange. If I change the query to only pull records inserted in the last 5 years, there are no problems. Someone suggested to me this might be an encoding conversion problem or something like that?
Update: Solved it. The problem turned out to be a bug in the ODBC driver for our not-very-reliable database software. A workaround with other drivers fixed the problem.
It could be a bug in the ODBC driver you are using. What driver is it? What is your connection string?
A shot in the dark here...
Try executing your reader, saving your result (maybe in an array or list), and making sure the reader is closed before executing or preparing the next command. You may even want to go extreme and put your getItemsCommand construction inside a using block so you know that it has no resources open before you execute your next command...
Related
Issue
I am refactoring a project written by a developer in C# .NET 4.5 project which contains a dataset.xsd. I was asked to increase the efficiency of the project. Basically, the problem is because the tableadapter fills my dataset with data from the entire table in the database. This is several million rows of data.
Problem
I had a line that is basically doing this.
this.customersTableAdapter.Fill(this.derpDataset.Customers);
So I decided to do something like this (not wanting to change the .xsd):
//This references a class written in order to get the database manually instead of using the .xsd
SqlConnection sqlConnection = DB_Connection.OpenDatabase();
SqlCommand sqlCommand = new SqlCommand("SELECT * FROM COMDB WHERE ID = " + ID.ToString() + ";", sqlConnection);
this.customersTableAdapter.Adapter.SelectCommand = sqlCommand;
this.customersTableAdapter.Adapter.Fill(this.derpDataset.Customers);
Code
Basically, the .xsd has a bunch of auto generated stuff, but I just needed a way step around it and fill with a much more optimized query.
[global::System.Diagnostics.DebuggerNonUserCodeAttribute()]
[global::System.CodeDom.Compiler.GeneratedCodeAttribute("System.Data.Design.TypedDataSetGenerator", "4.0.0.0")]
private void InitCommandCollection() {
this._commandCollection = new global::System.Data.SqlClient.SqlCommand[1];
this._commandCollection[0] = new global::System.Data.SqlClient.SqlCommand();
this._commandCollection[0].Connection = this.Connection;
this._commandCollection[0].CommandText = "SELECT customerID, name, SSN FROM dbo.tblActualCustomerValueFloat";
this._commandCollection[0].CommandType = global::System.Data.CommandType.Text;
}
Here is the autogenerated Fill function.
public virtual int Fill(customersTableAdapter dataTable) {
this.Adapter.SelectCommand = this.CommandCollection[0];
if ((this.ClearBeforeFill == true)) {
dataTable.Clear();
}
int returnValue = this.Adapter.Fill(dataTable);
return returnValue;
}
Error
This is frustrating because I get a new error which makes no sense to me. It says:
A first chance exception of type 'System.Data.ConstraintException' occurred in System.Data.dll
The exception snapshot states the following:
{"Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints."}
I will not use the following fix because it breaks the next line in my code which is.
CustomerDataSet.Runs_CustomerRow runRow = derpDataset.Runs_Customer.First(t => t.ID == ID);
Attempts
Tried changing .xsd and caused the whole thing to crash.
This convinced me not to mess with the .xsd at all. Decided to search here and found here.
Read the documentation MSDN provided here and here.
Googled, but found no similar issue. Maybe this, but not what his fix was?
Code provided above
I know what I want to do and the idea is simple, but I have found the application extremely hard given the reliance of the project on .xsd.
I would really appreciate any assistance and comments on this issue.
I figured out the answer. First, I created a way to get a more effective error message. I wrote the function that MSDN suggested. You can write a function to get errors from a dataset. I then put it in try/catch and read the output on console.
It turns out I was querying the wrong database and getting nulls. This was easy to do because the schema names were so similar, but the error message wasn't useful until I could see what the values were in my columns.
You can find the link to the error function I used here.
I am using SQLite-net for accessing an SQLite database file in a WinRT app. I don't have any problems reading from the database using the ExecuteQuery (I actually use the modified version from https://github.com/praeclarum/sqlite-net/issues/82 as I don't use table mappings and want the results as dictionary which calls ExecuteDeferredQuery underneath).
When I try to insert records into my database using ExecuteNonQuery, I am getting an exception with the message "CannotOpen".
Well, just a few lines above, I can read from the database successfully. I double checked the file permissions to the sqlite database file and gave everyone on the computer full control to the file to avoid any file permission issues, but the result is the same, still getting "CannotOpen".
I just tried to do a select statement with ExecuteNonQuery (just to see if it works"), I still get an exception, this time saying "Row" as the exception message.
I tried to execute my insert with ExecuteQuery to see what happens, no exception is thrown, everything seems OK, but no rows are inserted into my database.
Well, that may be explainable as ExecuteQuery is designed for querying, not inserting, but what could be the reason for ExecuteNonQuery throwing exceptions?
Here is my code (removed actual table names and parameters with generic ones for privacy):
SQLiteCommand cmd = conn.CreateCommand("insert into mytable(myfields...) values (?,?,?,?,?,?,?)", my parameters...);
cmd.ExecuteNonQuery(); //throws SQLiteException
However this code doesn't throw exception:
SQLiteCommand cmd = conn.CreateCommand("select * from mytable where some condition...", some parameters...);
var result = cmd.ExecuteToDictionary(); //renamed ExecuteQuery method from https://github.com/praeclarum/sqlite-net/issues/82
UPDATE: I've further tracked the issue down to something even simpler (and weird). This code is the very first call to SQLite framework after initialization of the connection. This very code, when executed, throws an exception in the fourth line:
SQLiteCommand cmd = conn.CreateCommand("select * from mytable");
cmd.ExecuteNonQuery();
cmd = conn.CreateCommand("select * from mytable"); //yes, the same simple query as above
cmd.ExecuteNonQuery();//getting error
UPDATE 2: If I call ExecuteToDictionary instead of ExecuteNonQuery, it works.
UPDATE 3: If I try a direct query (from the conn object such as conn.Execute("query...")) before all these calls it fails. If it's an insert query, I get CannotOpen error, if it's a select query, I get a Row error.
Why am I getting an exception on the second call to ExecuteNonQuery?
Why am I getting a different error message "Row" when I try SELECT with ExecuteNonQuery? And lastly, why are these exceptions so user-unfriendly?
Found out the answer. The SQLite file was in a directory that didn't have write access (the file DID have all the access in file properties, but I think it's a WinRT security model issue as the file was outside the sandbox of WinRT's storage folders. I could read the file, but not write. Thanks to SQLite-net's extremely helpful exception messages such as "Row" and "CannotOpen", without giving any real details about the problem, it took me days to realize that it was a file access issue rather than an SQLite configuration/query issue.
If anyone has any similar problems in the future, first, check that the SQLite database is in the storage directory of the WinRT app.
try to close and open the connection object before executing any other operations
I'm having some trouble with an application that we're porting to Mono.
(For reference, the .NET runtime is 4.0, and the mono version is 2.6.7.)
EDIT: This problem persists on Mono 2.10.2
As part of the startup of the application, it reads data into memory. For this part I'm just using in-line SQL commands, but for some reason, I'm seeing inconsistent behaviour on Linux/Mono (when the whole thing works in Windows/.NET).
I have a query that works fine in some scenarios, but not in others.
This particular example doesn't work:
var cmd = new SqlCommand("SELECT ID, Name, VATTerritoryID, NativeCurrencyID FROM PricingZones", conn);
var reader = cmd.ExecuteReader();
var objectToLoad = new SomeObjectType();
while (reader.Read())
{
objectToLoad.Property1 = reader.GetInt32(row.GetOrdinal("ID"));
objectToLoad.Property2 = reader.GetString(row.GetOrdinal("Name"));
objectToLoad.Property3 = reader.GetInt32(row.GetOrdinal("VATTerritoryID"));
objectToLoad.Property3 = reader.GetInt32(row.GetOrdinal("NativeCurrencyID"));
}
EDIT: For comparison, here's one that does work:
var cmd = new SqlCommand("SELECT VATTerritoryID, ProductDescriptionID, VATBandID FROM VATTerritoryBandExceptions", conn);
var reader = cmd.ExecuteReader();
var someOtherObjectToLoad = new SomeOtherObjectType();
while (reader.Read())
{
someOtherObjectToLoad.Property1 = reader.GetInt32(row.GetOrdinal("VATTerritoryID"));
someOtherObjectToLoad.Property2 = reader.GetString(row.GetOrdinal("ProductDescriptionID"));
someOtherObjectToLoad.Property3 = reader.GetInt32(row.GetOrdinal("VATBandID"));
}
I had possible suspicions that there were differences down to:
Casing (Since I know this is different on windows/linux), but putting everything to lowercase hasn't solved the problem
Column names (perhaps Mono cares more about reserved words?), but it seems replacing Name with [Name] or 'Name' didn't make any different
The error I had in the first case was:
[IndexOutOfRangeException: Array index is out of range.]
at System.Data.SqlClient.SqlDataReader.GetInt32(Int32 i)
Suggesting that there is no "column1" in the returned result set.
(EDIT: updated this section a little for clarity)
Oddly, if I do this:
var cmd = new SqlCommand("SELECT ID, Name, VATTerritoryID, NativeCurrencyID FROM PricingZones", conn);
var reader = cmd.ExecuteReader();
var objectToLoad = new SomeObjectType();
while (reader.Read())
{
Console.WriteLine("First row, first column is " + row.GetValue(0));
Console.WriteLine("First row, second column is " + row.GetValue(1));
Console.WriteLine("First row, third column is " + row.GetValue(2));
Console.WriteLine("First row, fourth column is " + row.GetValue(3));
}
The output is:
First row, first column is 0
First row, second column is New
Array index is out of range.
I assume something strange is happening with the Mono framework in this case, but I can't find a relevant bug report, and I can't identify why this only happens in some cases but not others! Has anyone else had a similar experience?
EDIT: I've changed some of the statements to match those in the failing query exactly, in case there is an issue with reserved words or similar. Note that the query I'm issuing in the second case really does request four columns and seemingly only gets two very odd ones back (0 | New ).
Ok mate I managed to reproduce your problem. You have a threading problem there. That was my only idea what may be the cause of this problem and I managed to actually reproduce it.
In order to fix it you need to do the following:
Make sure every reader has a reader.Close() call after parsing the
data.
Use the following code to do thread-safe calls:
Object executeLock = new Object();
private IDataReader ExecuteThreadSafe(IDbCommand sqlCommand)
{
lock (executeLock)
{
return sqlCommand.ExecuteReader(CommandBehavior.CloseConnection);
}
}
Looks like mono has a different implementation of SQL objects than .NET. It's true that I wasn't able to reproduce it on Windows!
I would start out by trying to figure out exactly where the difference lies.
First off, you have the following code:
objectToLoad.Property1 = reader.GetInt32(row.GetOrdinal("column1"));
objectToLoad.Property2 = reader.GetString(row.GetOrdinal("column2"));
objectToLoad.Property3 = reader.GetString(row.GetOrdinal("column ")); //Is the space on the end intended?
I believe you said the first two lines work, and the third line blows up. We first need to figure out where this blows up. I'd try:
int test = row.GetOrdinal("column ");
Does test equal a valid column index? If not, that's your problem. Make sure that's the exact column name, try different casings, etc. If the index is valid, try:
object foo = reader[test];
Console.WriteLine(foo.GetType().Name);
to figure out what the data type of this column is. Perhaps there's a casting problem.
If that fails, I'd try loading the reader into a DataSet object instead so you can look at the exact schema more carefully.
If you do find a difference in behavior between Mono and the .NET Framework, the Mono team is usually very willing to fix these. I strongly recommend filing this as a bug.
Hope this helps!
Try accessing your reader in this manner:
reader["column1isastring"].ToString();
(Int32)reader["column2isInt32"];
Also, as a side note, make sure you're using the "using" directive for disposable objects. I'm not sure if Mono implements this, but it's worth a shot. What the using directive does is cleans up disposable objects as soon as you're done using them. It's very handy and makes for clean code. Quick example:
using (MySqlCommand command = new MySqlCommand("SELECT column1, column2, column FROM tablename", conn))
{
try
{
conn.Open();
using (MySqlDataReader reader = command.ExecuteReader())
{
reader.Read();
var someString = reader["column1isastring"].ToString();
var whatever = (Int32)reader["column2isInt32"];
} //reader closes and disposes here
}
catch (Exception ex)
{
//log this exception
}
finally
{
conn.Close();
}
} //conn gets closed and disposed of here
Also, if you're getting user input that goes straight to your commands, make sure you use the MySqlParameter class to keep malicious parameters from dropping tables, for example.
Mono is not .NET and there are a lot of differences especially with earlier versions. The root methodology to connect to SQL is using a TDS (tabular data stream) implementation in C# which for earlier versions of Mono (and TDS as a result) can cause a lot of problems. Almost all essential classes for SQL and data are having differences that potentially may cause exceptions. Maybe it's worthy to try with Mono 2.10+ because the Mono team improves the whole project continuously.
I am using Mono 4.0 and I have a similar problem. My research shows, that most probable reason of this problem is flawed implementation of connection pool in Mono. At least, if I switch off pooling, the problem disappears.
To switch off connection pooling you need to add the following string to your connection string: Pooling=false;
After that, creation of connection object should look like that:
var conn = new SqlConnection("Server=localhost; Database=somedb; user=user1; password=qwerty; Pooling=false;");
I was going through some old code that was written in years past by another developer at my organization. Whilst trying to improve this code, I discovered that the query it uses had a very bad problem.
OdbcDataAdapter financialAidDocsQuery =
new OdbcDataAdapter(
#"SELECT a.RRRAREQ_TREQ_CODE,
b.RTVTREQ_SHORT_DESC,
a.RRRAREQ_TRST_DESC,
RRRAREQ_STAT_DATE,
RRRAREQ_EST_DATE,
a.RRRAREQ_SAT_IND,
a.RRRAREQ_SBGI_CODE,
b.RTVTREQ_PERK_MPN_FLAG,
b.RTVTREQ_PCKG_IND,
a.RRRAREQ_MEMO_IND,
a.RRRAREQ_TRK_LTR_IND,
a.RRRAREQ_DISB_IND,
a.RRRAREQ_FUND_CODE,
a.RRRAREQ_SYS_IND
FROM FAISMGR.RRRAREQ a, FAISMGR.RTVTREQ b
WHERE a.RRRAREQ_TREQ_CODE = b.RTVTREQ_CODE
and a.RRRAREQ_PIDM = :PIDM
AND a.RRRAREQ_AIDY_CODE = :AidYear ",
this.bannerOracle);
financialAidDocsQuery.SelectCommand.Parameters.Add(":PIDM", OdbcType.Int, 32).Value = this.pidm;
financialAidDocsQuery.SelectCommand.Parameters.Add(":AidYear", OdbcType.Int, 32).Value = this.aidYear;
DataTable financialAidDocsResults = new DataTable();
financialAidDocsQuery.Fill(financialAidDocsResults);
FADocsGridView.DataSource = financialAidDocsResults;
FADocsGridView.DataBind();
The problem is that the column a.RRRAREQ_TRST_DESC does not exist. A fact you learn very quickly when running it in Oracle SQL Developer.
The strange thing?
This code works.
The gridview binds successfully. (It doesn't try to bind to that field.) And it's been in production for years.
So, my question is...why? I've never seen a bad query work. I've never seen Oracle allow it or a data provider hack around it.
Does anyone have any idea what's going on here?
Hmmm...A few things to check:
Does this code actually run? It may seem silly to suggest this, but there may be a newer file that replaced this one.
Is an exception being squelched by your code? (Anyone who would name columns like that is definitely capable of squelching those pesky exceptions)
Is the exception being squelched by 3rd party code? (Not as likely, but sometimes 3rd party code prefers to use annoying error codes instead of exceptions).
Past those suggestions, I'm not sure.
EDIT:
Revisiting the 2nd point, if you are working in ASP.NET, check that there is no global-level exception handler that is squelching exceptions. I ran into that problem on one site that I worked on and found dozens of exceptions in a single day.
Try running
select * from v$sql where sql_fulltext like '%a.RRRAREQ_TRST_DESC%'
shortly after you bind the grid. That will tell you if the statement was actually seen by Oracle. Note that you should only see the above query if it was not seen by Oracle.
Use ODBC trace log to see if this query is really send to database, and see what database returns. Then use any other ODBC based database tool and check if this query work from this tool. As an ultimate test you can write simple Python script. Easiest way it to use ActiveState Python 2.x with odbc module included. Test code can look like:
import odbc
connection = odbc.odbc('dnsname/user/password')
cursor = connection.cursor()
cursor.execute("select ...")
for row in cursor.fetchall():
print '\t'.join([str(r) for r in row])
If there was no error in your program and an error in other tools then compare theirs ODBC traces.
If I understand what the original author was trying to do, and with Banner that is never easy to figure out, then this query should be correct:
SELECT a.rrrareq_treq_code,
b.rtvtreq_short_desc,
c.rtvtrst_desc,
rrrareq_stat_date,
rrrareq_est_date,
a.rrrareq_sat_ind,
a.rrrareq_sbgi_code,
b.rtvtreq_perk_mpn_flag,
b.rtvtreq_pckg_ind,
a.rrrareq_memo_ind,
a.rrrareq_trk_ltr_ind,
a.rrrareq_disb_ind,
a.rrrareq_fund_code,
a.rrrareq_sys_ind
FROM faismgr.rrrareq a,
faismgr.rtvtreq b,
faismgr.rtvtrst c
WHERE a.rrrareq_treq_code = b.rtvtreq_code
AND a.rrrareq_trst_code = c.rtvtrst_code
AND a.rrrareq_pidm = :PIDM
AND a.rrrareq_aidy_code = :AidYear;
Well, let's file this in the false alarm category.
I decided to have our VAT send a copy of the DLL from test. I pulled it apart with reflector and found, much to my embarrassment, that the query is right. Which makes sense.
I still can't figure out why my working copy would have one incorrect field_name. To my knowledge, I had never touched this file before this week. But, SVN doesn't have any history showing this error in previous versions.
So strange...maybe I'm losing my mind.
Thanks for all of the quality feedback on this question. I certainly learned some new trouble shooting techniques and for that I'm very appreciative. :)
Happy coding,
Clif
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 7 years ago.
Improve this question
I'm getting the following error whenever my code creates a DataTableReader from a valid DataTable Object:
"DataTableReader is invalid for current DataTable 'TempTable'."
The thing is, if I reboot my machine, it works fine for an undetermined amount of time, then dies with the above. The code that throws this error could have been working fine for hours and then: bang. you get this error.
It's not limited to one line either; it's every single location that a DataTableReader is used. Also, this error does NOT occur on the production web server - ever.
This has been driving me nuts for the best part of a week, and I've failed to find anything on Google that could help (as I'm pretty positive this isn't a coding issue).
Some technical info:
DEV Box:
Vista 32bit (with all current windows updates)
Visual Studio 2008 v9.0.30729.1 SP
dotNet Framework 3.5 SP1
SQL Server:
Microsoft SQL Server 2005 Standard Edition- 9.00.4035.00 (X64)
Windows 2003 64bit (with all current windows updates)
Web Server:
Windows 2003 64bit (with all current windows updates)
Any help, ideas, or advice would be greatly appreciated!
UPDATE 1:
Ok - Have tried the following now with no success:
1: Rebooted
2: SFC / ScanNow
3: Changed SQL Servers
4: Tried a different method that uses DataTableReaders
5: Cleaned solution
The only thing I did find that worked was copy & pasting the code
from the main Visual studio instance, into another which had a simple console app.
This then worked as expected (queried database and got results into a dataTable, created a datatablereader on that table, then queried hasrows before calling .Read()...
All of which worked.
I am struggling to see what could cause this, as there are NO code faults - i'm 100% certain, as it runs perfectly when published to the webserver.
I think using the while(reader.read()) may solve your problem.
if (myReader.HasRows)
while (myReader.Read())
Console.WriteLine("\t{0}\t{1}", myReader.GetInt32(0), myReader.GetString(1));
else
Console.WriteLine("No rows returned.");
myReader.Close();
UPDATE:
Also from msdn:
The HasRows property returns information about the current result set. If the DataTableReader contains multiple result sets, you can examine the value of the HasRows property immediately after you call the NextResult method in order to determine whether the new result set contains rows.
Use the HasRows property to avoid the requirement to call the Read method of the DataTableReader if there are no rows within the current result set.
DataTableReader.HasRows Property
Had the same problem and got rid of it after clearing the variables in the watch window.
Clearing the watch window & doing rebuilds worked for me. However, because I had to remember to frequently rebuild, I eventually just renamed it also. (prior to renaming, adding additional watch variables on an object could cause previous watch variables on that object to become invalid -- even without progressing through the code, ie staying on the same line)
Wrap usage of DataTableReader (and all IDisposables) with using.
OK.. Further down in the code, I have the following code:
using (DataTableReader tr = dtCustomers.CreateDataReader())
{
....
}
If I change this to read:
using (DataTableReader tr2 = dtCustomers.CreateDataReader())
{
....
}
Then, and remember this bit of code is much later down in the same procedure, BOTH bits of code work without fault!
So, this doesn't work:
using (DataTableReader tr = dt.CreateDataReader())
{
...
}
....
using (DataTableReader tr = dt.CreateDataReader())
{
...
}
But this does:
using (DataTableReader tr = dt.CreateDataReader())
{
...
}
....
using (DataTableReader tr2 = dt.CreateDataReader())
{
...
}
I don't understand why this way works, but it does and as I've not had another answer, I'll be going with this.
If you know why this works, and why the original doesn't, please can you enlighten me? :)
Just thought I would post on here in case it's helps someone else. I tried a number of things and in the end i simply changed the name of the datareader and it worked, kind of similar to here. i dont know why for sure but i think it might be because the datareader (originally) wasnt being closed, so maybe after a few times debugging, there was lots of "stuff" in memory with a certain name attached and it said "no more!". still, i could be talking bullpies. my advice, change the name of your datareader variable and make sure you close it after you use it
Seems like a bug on getting the tableReader... i have code that has been working for ears and if i change another method sometimes i get that errro... some times it's solved just recompiling (rebuild), another times i reinstalled the .NET framework or use the option repair of it... i am starting to put try catch sections to use the reader if the system "wants" to givme the reader and the DataTable if not.
Greetings.