I am having an issue where an exception is not being caught by try/catch block. The issue occurs at command.ExecuteReader(), however it never gets caught. I am running in debug mode and have already tried a few suggested options in regards to the debugger settings with no avail.
I do want to mention that I am using SQLite as my provider, and I can see that it throws an SQLiteException, however the issue remains. Would there be any specific scenario where an exception is not caught? (with exception of StackOverflowException, ThreadAbortedException etc...)
public IEnumerable<dynamic> Query(string sql, params object[] parms)
{
try
{
return QueryCore(sql, parms);
}
catch (Exception ex)
{
throw new DbException(sql, parms, ex);
}
}
private IEnumerable<dynamic> QueryCore(string sql, params object[] parms)
{
using (var connection = CreateConnection())
{
using (var command = CreateCommand(sql, connection, parms))
{
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
yield return reader.ToExpando();
}
}
}
}
}
I also want to add that if I produce a correct query against the database, I get results back, however when I break the query, the exception is thrown, however not caught.
This happens because you are returning the data with the yield keyword.
This makes the data actual data method to run only when it's results are enumerated.
You probably don't want this to happen, especially because if the results are enumerated twice (e.g. two seperate foreach loops) the data will be read twice.
You can do this to make the enumeration to happen immediately and catch any exception:
public IEnumerable<dynamic> Query(string sql, params object[] parms)
{
try
{
return QueryCore(sql, parms).ToArray();
}
catch (Exception ex)
{
throw new DbException(sql, parms, ex);
}
}
Yielding is good for situations where getting an item takes some time, and you don't want to get ALL the items before you can loop through them. So another possible solution, that might be better for the readability of your code (that I assume doesn't need to yield) will be this:
public IEnumerable<dynamic> Query(string sql, params object[] parms)
{
try
{
return QueryCore(sql, parms);
}
catch (Exception ex)
{
throw new DbException(sql, parms, ex);
}
}
private IEnumerable<dynamic> QueryCore(string sql, params object[] parms)
{
using (var connection = CreateConnection())
{
using (var command = CreateCommand(sql, connection, parms))
{
using (var reader = command.ExecuteReader())
{
var results = new List<dynamic>();
while (reader.Read())
{
results.Add(reader.ToExpando());
}
return results;
}
}
}
}
Related
I've been searching everywhere, to try and get over this issue but I just can't figure this out.
I'm trying to make many changes to the DB with one single transaction using LINQ to SQL.
I've created a .dbml that represents the SQL Table, then I use basicaly this code:
foreach (var _doc in _r.Docs)
{
try
{
foreach (var _E in _Es)
{
Entity _newEnt = CreateNewEnt(_EListID, _doc, _fileName, _E);
_db.Etable.InsertOnSubmit(_newEnt);
_ECount++;
if (_ECount % 1000 == 0)
{
_db.SubmitChanges();
}
}
}
catch (Exception ex)
{
throw;
}
}
But when I do a SQL Profiler, the commands are all executed individually. It won't even start an SQL Transaction.
I've tried using TransactionScope (using statement and Complete()) and DbTransaction (BeginTransaction() and Commit()), none of them did anything at all, it just keeps on executing all commands individually, inserting everything like it was looping through all the inserts.
TransactionScope:
using(var _tans = new TransactionScope())
{
foreach (var _doc in _r.Docs)
{
try
{
foreach (var _E in _Es)
{
Entity _newEnt = CreateNewEnt(_EListID, _doc, _fileName, _E);
_db.Etable.InsertOnSubmit(_newEnt);
_ECount++;
if (_ECount % 1000 == 0)
{
_db.SubmitChanges();
}
}
}
catch (Exception ex)
{
throw;
}
}
_trans.Complete();
}
DbTransaction:
_db.Transaction = _db.Connection.BeginTransaction();
foreach (var _doc in _r.Docs)
{
try
{
foreach (var _E in _Es)
{
Entity _newEnt = CreateNewEnt(_EListID, _doc, _fileName, _E);
_db.Etable.InsertOnSubmit(_newEnt);
_ECount++;
if (_ECount % 1000 == 0)
{
_db.SubmitChanges();
}
}
}
catch (Exception ex)
{
throw;
}
}
_db.Transaction.Commit();
I also tried commiting transactions everytime I Submit the changes, but still nothing, just keeps on executing everything individually.
Right now I'm at a loss and wasting time :\
GSerg was right and pointed me to the right direction, Transactions do not mean multiple commands in one go, they just allow to "undo" all that was made inside given transaction if need be. Bulk statements do what I want to do.
You can download a Nuget Package directly from Visual Studio called "Z.LinqToSql.Plus" that helps with this. It extends DataContext from LINQ, and allows to do multiple insertions, updates or deletes in bulks, which means, in one single statement, like this:
foreach (var _doc in _r.Docs)
{
try
{
foreach (var _E in _Es)
{
Entity _newEnt = CreateNewEnt(_EListID, _doc, _fileName, _E);
_dictionary.add(_ECount, _newEnt); //or using a list as well
_ECount++;
if (_ECount % 20000 == 0)
{
_db.BulkInsert(_dictionary.Values); //inserts in bulk, there are also BulkUpdate and BulkDelete
_dictionary = new Dictionary<long, Entity>(); //restarts the dictionary to prepare for the next bulk
}
}
}
catch (Exception ex)
{
throw;
}
}
As in the code, I can even insert 20k entries in seconds. It's a very useful tool!
Thank you to everyone who tried helping! :)
I have a Web API 2 endpoint set up to test a 2+ second ADO.NET call. When attempting to "burst" this api, it fails horribly when using async methods. I'm getting connection timeouts and reader timeouts. This doesn't happen with the synchronous version of this method when "bursted". The real problem is as follows...
Async ADO.NET behaves strangely when exceptions are thrown. I can see the exceptions thrown in the Visual Studio output, but they are not caught by my code. As you'll see below, I've tried wrapping try/catches around just about everything and have had no results. I did this to be able to set break points. I understand that catching exceptions just to throw them is bad. Initially, I only wrapped the call in the API layer. The worst part is, this locks up the entire server.
Clearly, there's something I'm missing about async ADO.NET. Any ideas?
EDIT:
Just to clarify what I'm trying to do here. This is just some test code on my local computer that's talking to our developmental database. It was just to prove/disprove that we can handle more traffic with async methods against our longer running db calls. I think what's happening is that as the calls are stacking up. In doing so, the await'ed connections and readers are timing out because we're not getting back to them quickly enough. This is why it doesn't fail when it's ran synchronously. This is a completely different issue. My concern here is that the operations are not throwing exceptions in a way that can be caught. The below is not production code :)
Web API2 Controller:
[Route("api/async/books")]
[HttpGet]
public async Task<IHttpActionResult> GetBookAsync()
{
// database class instantiation removed
// search args instantiation removed
try
{
var books = await titleData.GetTitlesAsync(searchArgs);
return Ok(books);
}
catch (Exception ex)
{
return InternalServerError(ex);
}
}
Data Access:
public async Task<IEnumerable<Book>> GetTitlesAsync(SearchArgs args)
{
var procName = "myProc"
using(var connection = new SqlConnection(_myConnectionString))
using (var command = new SqlCommand(procName, connection) { CommandType = System.Data.CommandType.StoredProcedure, CommandTimeout = 90 })
{
// populating command parameters removed
var results = new List<Book>();
try
{
await connection.OpenAsync();
}
catch(Exception ex)
{
throw;
}
try
{
using (var reader = await command.ExecuteReaderAsync())
{
try
{
while (await reader.ReadAsync())
{
// FROM MSDN:
// http://blogs.msdn.com/b/adonet/archive/2012/07/15/using-sqldatareader-s-new-async-methods-in-net-4-5-beta-part-2-examples.aspx
// Since this is non-sequential mode,
// all columns should already be read in by ReadAsync
// Therefore we can access individual columns synchronously
var book = new Book
{
Id = (int)reader["ID"],
Title = reader.ValueOrDefault<string>("Title"),
Author = reader.ValueOrDefault<string>("Author"),
IsActive = (bool)reader["Item_Active"],
ImageUrl = GetBookImageUrl(reader.ValueOrDefault<string>("BookImage")),
ProductId = (int)reader["ProductID"],
IsExpired = (bool)reader["Expired_Item"]
};
results.Add(book);
}
}
catch(Exception ex)
{
throw;
}
}
}
catch(Exception ex)
{
throw;
}
return results;
}
}
Function that throws the ThirdPartyException (I don't know how does their code work) exception:
private void RequestDocuments(/* arguments... */) {
while(true) {
var revision = lastRevision;
var fetchedDocuments = 0;
try {
foreach(var document in connection.QueryDocuments(revision)) {
if(fetchedDocuments > fetchQuota) return;
container.Add(document);
++fetchedDocuments;
Logger.Log.InfoFormat("added document (revision: {0}) into inner container", document.Revision);
}
Logger.Log.Info("Done importing documents into the inner container");
return;
}
catch(Exception ex) {
if(ex is ThirdPartyException) {
// handle this in a certain way!
continue;
}
}
}
}
this function is called inside a worker thread like this:
private void ImportDocuments() {
while(!this.finishedEvent.WaitOne(0, false)) {
try {
var documents = new List<GohubDocument>();
RequestDocuments(remoteServerConnection, documents, lastRevision, 100);
}
catch(Exception ex) {
// here is where it really gets handled!!!?
}
}
}
the exception is handled only in the outermost (which is inside the ImportDocuments method) try/catch.
Why is that?
If that's a LINQ API which exposes IQueryable you don't get an error due to the deferred execution that LINQ to SQL implementations typically uses.
To prevent it you have to invoke .ToList(), FirstOrDefault() etc within your first method. That makes sure that the query really have been executed against your data source.
Solution:
var documents = connection.QueryDocuments(revision).ToList();
foreach(var document in documents) {
if(fetchedDocuments > fetchQuota) return;
// [...]
}
I want to read data to a list from database.
I tried the following code
public List<T> StoredProcedureForIList<T>(string spName, params IDataParameter[] commandParameters)
{
List<T> list = new List<T>();
T item;
Type listItemType = typeof(T);
item = (T)Activator.CreateInstance(listItemType);
list.Add(item);
using (IDatabaseConnection connection = new DatabaseConnection())
{
IDbCommand cmd = connection.CreateCommandForStoredProcedure(spName);
foreach (SqlParameter par in commandParameters)
{
cmd.Parameters.Add(par);
}
try
{
using (IDataReader reader = cmd.ExecuteReader())
{
while (reader != null && reader.Read())
{
for (int i = 0; i < reader.FieldCount; i++)
{
var prop = listItemType.GetProperty(reader.GetName(i));
prop.SetValue(item, reader[i], null);
}
list.Add(item);
}
}
}
catch(Exception ex)
{ }
return list;
}
}
But the problem is that when the for loop starts the reader loses data.
The data reader ResultView value is Enumeration yielded no results.
My guess is that some error occurs during execution of your loop. This technique...
try
{
...
}
catch(Exception ex)
{ }
...ensures that this error is ignored and all you get is an incomplete result. As you have noticed, this makes debugging quite hard. So don't do that.
Thus, the solution is:
remove the try-catch block (i.e., replace try { ... } catch(Exception ex) {} by ...),
run the code again,
note the error that occurs,
if you understand the error
fix it
else
ask again on StackOverflow, in a new question.
And, never, never write catch (Exception ex) {} again. ;-) Do proper error handling, or don't do error handling at all.
The reader won't be dropping rows; the reader is pretty well-tested. Swallowing exceptions won't help. If I had to guess, the problem here is that you are adding the same item over and over. In fact, you add it N+1 times (you add it once at the top even if no rows are returned).
However, can I suggest: just use something like dapper, which does everything above, except a: it gets it right, and b: it is highly optimized (it emits custom IL to avoid constant reflection, and caches that IL). It would be something akin to:
var list = connection.Query<T>(procName, namedArgs,
commandType: CommandType.StoredProcedure).ToList();
where namedArgs would be, to pass in #id and #name, for example:
new {id=123, name="abc"}
i.e.
int id = ...
string name = ...
var list = connection.Query<T>("MyProc", new {id, name},
commandType: CommandType.StoredProcedure).ToList();
I have the following function which reads from a firebird database. The Function works but does not handle exceptions (Required).
public IEnumerable<DbDataRecord> ExecuteQuery(string Query)
{
var FBC = new FbCommand(Query, DBConnection);
using (FbDataReader DBReader = FBC.ExecuteReader())
{
foreach (DbDataRecord record in DBReader)
yield return record;
}
}
Adding try/catch to this function gives an error regarding yield. I understand why I get the error but any workround I've tried has resulted in DBReader being disposed indirectly via using() too early or Dispose() not being called all. How do I get this code to use Exceptions & Cleanup without having to wrap the method or duplicate DBReader which might contain several thousand record?
Update:
Here is an example of an attempted fix. In this case DBReader is being disposed too early.
public IEnumerable<DbDataRecord> ExecuteQuery(string Query)
{
var FBC = new FbCommand(Query, DBConnection);
FbDataReader DBReader = null;
try
{
using (DBReader = FBC.ExecuteReader());
}
catch (Exception e)
{
Log.ErrorException("Database Execute Reader Exception", e);
throw;
}
foreach (DbDataRecord record in DBReader) <<- DBReader is closed at this stage
yield return record;
}
The code you've got looks fine to me (except I'd use braces round the yield return as well, and change the variable names to fit in with .NET naming conventions :)
The Dispose method will only be called on the reader if:
Accessing MoveNext() or Current in the reader throws an exception
The code using the iterator calls dispose on it
Note that a foreach statement calls Dispose on the iterator automatically, so if you wrote:
foreach (DbDataRecord record in ExecuteQuery())
{
if (someCondition)
{
break;
}
}
then that will call Dispose on the iterator at the end of the block, which will then call Dispose on the FbDataReader. In other words, it should all be working as intended.
If you need to add exception handling within the method, you would need to do something like:
using (FbDataReader DBReader = FBC.ExecuteReader())
{
using (var iterator = DBReader.GetEnumerator())
{
while (true)
{
DbDataRecord record = null;
try
{
if (!iterator.MoveNext())
{
break;
}
record = iterator.Current;
}
catch (FbException e)
{
// Handle however you want to handle it
}
yield return record;
}
}
}
Personally I'd handle the exception at the higher level though...
This line won't work, note the ; at the end, it is the entire scope of the using()
try
{
using (DBReader = FBC.ExecuteReader())
; // this empty statement is the scope of using()
}
The following would be the correct syntax except that you can't yield from a try/catch:
// not working
try
{
using (DBReader = FBC.ExecuteReader())
{
foreach (DbDataRecord record in DBReader)
yield return record;
}
}
catch (Exception e)
{
Log.ErrorException("Database Execute Reader Exception", e);
throw;
}
But you can stay a little closer to your original code:
// untested, ought to work
FbDataReader DBReader = null;
try
{
DBReader = FBC.ExecuteReader();
}
catch (Exception e)
{
Log.ErrorException("Database Execute Reader Exception", e);
throw;
}
using (DBReader)
{
foreach (DbDataRecord record in DBReader) // errors here won't be logged
yield return record;
}
To catch errors from the read loop as well see Jon Skeet's answer.