We Keep Coding

Reading CSV file with datetime column which can be invalid [duplicate]

I am using the class CsvReader successfully and am happy with it, however, the file that I consume is being produced by a group which changes column formats without letting me know.
So, one moment everything is working, then the next morning things break and the try catch block around csv.GetRecord<MyType>() catches the error and logs the error, however I can't gather any valuable info from the Exception instance. It just says: "The conversion cannot be performed." and the InnerException has nothing. Not very useful. I don't even know which one of my 150 columns are causing the problem.
Can you help me figure out how I can pinpoint which column in which row is causing the problem?
Thanks

Currently, there is no way to ignore errors at the field/property level. Your current options are these:
Look at the exception data.
catch( Exception ex )
{
// This contains useful information about the error.
ex.Data["CsvHelper"];
}
Ignore reading exceptions. This is on a row level, though, not field. It will allow the whole file to still be read, and just ignore the rows that don't work. You can get a callback when an exception occurs.
csv.Configuration.IgnoreReadingExceptions = true;
csv.Configuration.ReadingExceptionCallback = ( ex, row ) =>
{
// Do something with the exception and row data.
// You can look at the exception data here too.
};

First of all, it seems that I need to catch CsvTypeConverterException.
while (csv.Read())
{
try
{
var record = csv.GetRecord<MyType>();
}
catch (CsvTypeConverterException ex)
{
//ex.Data.Values has more info...
}
}
I now know how to investigate what went wrong, but how do I make sure that that field is skipped but the rest of the fields in that row are converted, so that not the entire row is thrown away?
Thanks

CsvHelper has public 'Context' field in the CsvReader and there are all what needed for display a problem:
try
{
var records = csvReader.GetRecords<MyType>().ToList();
}
catch (CsvHelperException e)
{
Console.WriteLine($"{e.Message} " + (e.InnerException == null ? string.Empty : e.InnerException.Message));
Console.WriteLine($"Row: {csvReader.Context.Row}; RawLine: {csvReader.Context.RawRecord}");
if (csvReader.Context.CurrentIndex >= 0 &&
csvReader.Context.CurrentIndex < csvReader.Context.HeaderRecord.Length)
{
Console.WriteLine($"Column: {csvReader.Context.CurrentIndex}; ColumnName: {csvReader.Context.HeaderRecord[csvReader.Context.CurrentIndex]}");
}
throw;
}

How to catch multiple exceptions on SaveChanges() or is it possible?

I get an XML file(from an infopath form), create an object and insert that object into the DB. Im using a C# webservice to do this parsing and it returns true if successful and currently it returns the exception message if the SaveChanges() fails. The main exceptions I'd like to return are the DbEntityValidationExceptions as I'll handle other exceptions differently. Some of the columns have max lengths so is the field exceeds that I want to return the field name that they need to edit. Can I catch all the DbEntityValidationException for all fields that failed or does entity only throw the first exception and then rollback the transaction? With 200 fields it'd be nice to tell the user which fields they need to change versus the first field and then continue to fail as they fix the single exception each time.
If it is not possible my proposed solution below is irrelevant and should be removed. If it is possible to return all the Exceptions, what am I doing wrong?
exceptionList = new List<string>();
try
{
db.SaveChanges();
}
catch (Exception ex)
{
if (ex.GetType() == new DbEntityValidationException().GetType())
{
DbEntityValidationException eValidEx = (DbEntityValidationException) ex;
foreach (DbEntityValidationResult vResult in eValidEx.EntityValidationErrors)
{
foreach (DbValidationError vError in vResult.ValidationErrors)
{
exceptionList.Add(vError.ErrorMessage);
}
}
result = false;
}
else
{
exceptionList.Add("Unknown. " + ex.Message);
}
}

You need custom exception which inherit Exception. In this exception you have property for example Errors. This property will be message collection.
When you are trying to save changes, you should know what kind of fields you are expecting. You should define in other class boundaries of the fields(example Price->decimal, mandatory, maxValue and so on) On save you should check the boundaries if one of them is not true you should add message string in Errors of the custom exception(example Price is not decimal field)
On the end if the Errors.Count > 0 thorw the CustomException. You should override the message property in the CustomException -> loop all the Errors and return the text of all of them. At the end you need only customException.Message and you will have all the problems shown to the user.
I Hope this helps.

So when you catch an exception like that, you'll only catch the first exception. Java doesn't let you keep going on when you have a bunch of Exceptions and to catch all of them.
What you could do is put the entire thing in a while loop that keeps going until you've saved all of the changes. Make sure in the try{} block that you iterate through each field instead of just trying the first field again and again.

Catch SQL raise error in C#

I generate the raise error in SQL procedure:
RAISERROR('Already exist',-10,-10)
but I can not catch it using the following code in C#
catch (SqlException ex)
{
bResult = false;
if (ex.Errors[0].Number == -10)
{
CommonTools.vAddToLog("bInsertNewUser", "ManageUsers", ex.Message);
if ((savePoint != null))
savePoint.Rollback();
}
}
How can I catch the raised error in C# ?

RAISERRORs with a SEVERITY value under or equal to 10 are not caught on the C# side because I suppose they are considered just warnings as you can see from the
list at Database Engine Error Severities.
SEVERITY values between 11 and 16 are errors that can be corrected by the user, so, for example, you can try with:
RAISERROR('Already exist',16,1)
Otherwise, you could choose another error code from the list above or, if you really need it, prepare your own custom error message using sp_addmessage.

Your if statement is where things fail. Assuming that the first error in the collection is indeed the one you are looking for (it might not be).
SqlError.Number is not the value that you set in RAISERROR.
Use SqlError.Class to retrieve the severity of the error, or SqlError.State to check the state (both are -10 in your example. so difficult to tell which out you mean):
catch (SqlException ex)
{
bResult = false;
if (ex.Errors[0].Class == -10)
{
CommonTools.vAddToLog("bInsertNewUser", "ManageUsers", ex.Message);
if ((savePoint != null))
savePoint.Rollback();
}
}

Use an error code within 11-16, or just use 16 for a "general" case.
RAISERROR('Already exists',16,1)
Why? Here's my summary of https://learn.microsoft.com/en-us/sql/relational-databases/errors-events/database-engine-error-severities?view=sql-server-2017:
SQL RAISERROR -> C#:
0-10 = fyi*
11-16 = you can fix it**
17-19 = get admin help
20-24 = definitely get admin help***
*(don't throw anything) ┬──┬
**(16 = general)
***(fatal error)

Is this a clear use of goto?

Just wondering if this is considered a clear use of goto in C#:
IDatabase database = null;
LoadDatabase:
try
{
database = databaseLoader.LoadDatabase();
}
catch(DatabaseLoaderException e)
{
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
goto LoadDatabase;
}
I feel like this is ok, because the snippet is small and should make sense. Is there another way people usually recover from errors like this when you want to retry the operation after handling the exception?
Edit: That was fast. To answer a few questions and clarify things a bit - this is part of a process which is essentially converting from a different kind of project. The _userInteractor.GetDatabaseConnector() call is the part which will determine if the user wants to retry (possibly with a different database than the one in the config they are loading from). If it returns null, then no new database connection was specified and the operation should fail completely.
I have no idea why I didn't think of using a while loop. It must be getting too close to 5pm.
Edit 2: I had a look at the LoadDatabase() method, and it will throw a DatabaseLoaderException if it fails. I've updated the code above to catch that exception instead of Exception.
Edit 3: The general consensus seems to be that
Using goto here is not necessary - a while loop will do just fine.
Using exceptions like this is not a good idea - I'm not sure what to replace it with though.

Is there another way people usually
recover from errors like this when you
want to retry the operation after
handling the exception?
Yes, in the calling code. Let the caller of this method decide if they need to retry the logic or not.
UPDATE:
To clarify, you should only catch exceptions if you can actually handle them. Your code basically says:
"I have no idea what happened, but whatever I did caused everything to
blow up... so lets do it again."
Catch specific errors that you can recover from, and let the rest bubble up to the next layer to be handled. Any exceptions that make it all the way to the top represent true bugs at that point.
UPDATE 2:
Ok, so rather than continue a rather lengthy discussion via the comments I will elaborate with a semi-pseudo code example.
The general idea is that you just need to restructure the code in order to perform tests, and handle the user experience a little better.
//The main thread might look something like this
try{
var database = LoadDatabaseFromUserInput();
//Do other stuff with database
}
catch(Exception ex){
//Since this is probably the highest layer,
// then we have no clue what just happened
Logger.Critical(ex);
DisplayTheIHaveNoIdeaWhatJustHappenedAndAmGoingToCrashNowMessageToTheUser(ex);
}
//And here is the implementation
public IDatabase LoadDatabaseFromUserInput(){
IDatabase database = null;
userHasGivenUpAndQuit = false;
//Do looping close to the control (in this case the user)
do{
try{
//Wait for user input
GetUserInput();
//Check user input for validity
CheckConfigFile();
CheckDatabaseConnection();
//This line shouldn't fail, but if it does we are
// going to let it bubble up to the next layer because
// we don't know what just happened
database = LoadDatabaseFromSettings();
}
catch(ConfigFileException ex){
Logger.Warning(ex);
DisplayUserFriendlyMessage(ex);
}
catch(CouldNotConnectToDatabaseException ex){
Logger.Warning(ex);
DisplayUserFriendlyMessage(ex);
}
finally{
//Clean up any resources here
}
}while(database != null);
}
Now obviously I have no idea what your application is trying to do, and this is most certainly not a production example. Hopefully you get the general idea. Restructure the program so you can avoid any unnecessary breaks in application flow.
Cheers,
Josh

maybe im missing something but why cant you just use a while loop? this will give you the same loop forever if you have an exception (which is bad code) functionality that your code gives.
IDatabase database = null;
while(database == null){
try
{
database = databaseLoader.LoadDatabase();
}
catch(Exception e)
{
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
//just in case??
database = null;
}
}
if you have to use goto in your normal code, you're missing logical flow. which you can get using standard constructs, if, while, for etc..

Personally, I would have this in a separate method that returns a status code of success or failure. Then, in the code that would call this method, I can have some magic number of times that I would keep trying this until the status code is "Success". I just don't like using try/catch for control flow.

Is it clear? Not really. What you actually want to do, I think, is first try to load the database and then, if that didn't work, try to load it a different way. Is that right? Let's write the code that way.
IDatabase loadedDatabase = null;
// first try
try
{
loadedDatabase = databaseLoader.LoadDatabase();
}
catch(Exception e) { } // THIS IS BAD DON'T DO THIS
// second try
if(loadedDatabase == null)
{
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
loadedDatabase = databaseLoader.LoadDatabase()
}
This more clearly illustrates what you're actually doing. As an added bonus, other programmers won't gouge out your eyes. :)
NOTE: you almost certainly don't want to catch Exception. There's likely a more specific exception that you would rather be catching. This would also catch TheComputerIsOnFireException, after which it isn't really worth retrying.

No, it's not okay: http://xkcd.com/292/

On a side note, I think there is potential for an endless loop if you always get an exception.
Technically there is nothing wrong with your goto structure, but for me, I would opt for using a while loop instead. Something like:
IDatabase database = null;
bool bSuccess = false;
int iTries = 0
while (!bSuccess) // or while (database == null)
{
try
{
iTries++;
database = databaseLoader.LoadDatabase();
bSuccess = true;
}
catch(DatabaseLoaderException e)
{
//Avoid an endless loop
if (iTries > 10)
throw e;
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
}
}

Try, Catch Problem

I've noticed this problem happening a lot in most things I do, so I'm thinking there must be a design pattern for this.
Basically if an exception is thrown, attempt to solve the problem and retry. If I place it in the try, all it will do is catch the exception, but I want to retry whatever it was doing and if it fails again, retry again a certain number of times.
Is there a common pattern for this sort of stuff?

check this SO answer.. hope that helps u
Cleanest way to write retry logic?
public static class RetryUtility
{
public static void RetryAction(Action action, int numRetries, int retryTimeout)
{
if(action == null)
throw new ArgumenNullException("action");
do
{
try
{
action();
return;
}
catch
{
if(numRetries <= 0)
throw; // Avoid silent failure
else
{
Thread.Sleep(retryTimeout);
numRetries--;
}
}
}
while(numRetries > 0);
}
}
Call
RetryUtility.RetryAction( () => SomeFunctionThatCanFail(), 3, 1000 );
Credit goes to LBushkin

This runs indefinately but it would be easy to add a loop counter to the while clause
var solved = false;
var tries = 0;
while (!solved)
{
try
{
//Do Something
solved = true;
}
catch
{
//Fix error
}
finally
{
if(solved || IsRediculous(tries))
break;
tries++;
}
}

try/catch inside a loop, with a counter for retries?
EDIT: And your requirement of "retry whatever it was doing," you need custom logic for that, how to retry varies wildly (ie, reopen a stream, recreate the object, pause for X milliseconds, etc...), so you need it's own try/catch inside a loop for every atomic operation.
By "atomic operation" I mean a set of related statements, such as read a file. The whole file read into memory might be an atomic operation, for example.

On some limited basis, you might want to put your try/catch into a loop, and force break if is ultimately successful. Such might be for internet access testing and you want user to have another attempt at connection.

Something like this, maybe:
int MAX_RETRIES = 5;
for (var attempt=1; attempt <= MAX_RETRIES; attempt++) {
try {
DoSomethingThatMightThrow();
}
catch (AnExceptionIKnowHowToHandle) {
if (attempt < MAX_RETRIES)
continue;
throw;
}
}

Depends what you are trying, but typically you want to check for the possibility of an exception happening PRIOR to executing the code that could cause an exception.
For example, check that a file exists before accessing it, and create it (or whatever) if it doesn't.

Are you sure exception handling is the proper methodology here? If you can "solve the problem" you can probably detect the error condition prior to calling the exception-generatiing code.
Exception handling is most natural for things which are truly exceptional. A failed Internet connection (as in the previous answer) is something that can be detected and handled before calling exception-throwing code.

Yes, it is quite common to have a loop with a number of retries where you break out of the loop on success. A couple of things:
You might want to add a delay before retrying so that you don't use up all your retries in just a few milliseconds before the temporary problem had time to fix itself.
If you eventually fail, you should throw the first exception you caught, not the last one. The second exception could be the result of failing to recover correctly from the first failure and might not help to debug the original problem.

Coding what others have already mentioned:
var success = false;
var attempts = 0;
var maxAttempts = 0;
do {
attempts++;
try {
/* your code */
success = condition;
} catch(SuperciliousException e) {
/* recover */
}
} while(!success && attempts < maxAttempts);