Reading CSV file with datetime column which can be invalid [duplicate] - c#

I am using the class CsvReader successfully and am happy with it, however, the file that I consume is being produced by a group which changes column formats without letting me know.
So, one moment everything is working, then the next morning things break and the try catch block around csv.GetRecord<MyType>() catches the error and logs the error, however I can't gather any valuable info from the Exception instance. It just says: "The conversion cannot be performed." and the InnerException has nothing. Not very useful. I don't even know which one of my 150 columns are causing the problem.
Can you help me figure out how I can pinpoint which column in which row is causing the problem?
Thanks

Currently, there is no way to ignore errors at the field/property level. Your current options are these:
Look at the exception data.
catch( Exception ex )
{
// This contains useful information about the error.
ex.Data["CsvHelper"];
}
Ignore reading exceptions. This is on a row level, though, not field. It will allow the whole file to still be read, and just ignore the rows that don't work. You can get a callback when an exception occurs.
csv.Configuration.IgnoreReadingExceptions = true;
csv.Configuration.ReadingExceptionCallback = ( ex, row ) =>
{
// Do something with the exception and row data.
// You can look at the exception data here too.
};

First of all, it seems that I need to catch CsvTypeConverterException.
while (csv.Read())
{
try
{
var record = csv.GetRecord<MyType>();
}
catch (CsvTypeConverterException ex)
{
//ex.Data.Values has more info...
}
}
I now know how to investigate what went wrong, but how do I make sure that that field is skipped but the rest of the fields in that row are converted, so that not the entire row is thrown away?
Thanks

CsvHelper has public 'Context' field in the CsvReader and there are all what needed for display a problem:
try
{
var records = csvReader.GetRecords<MyType>().ToList();
}
catch (CsvHelperException e)
{
Console.WriteLine($"{e.Message} " + (e.InnerException == null ? string.Empty : e.InnerException.Message));
Console.WriteLine($"Row: {csvReader.Context.Row}; RawLine: {csvReader.Context.RawRecord}");
if (csvReader.Context.CurrentIndex >= 0 &&
csvReader.Context.CurrentIndex < csvReader.Context.HeaderRecord.Length)
{
Console.WriteLine($"Column: {csvReader.Context.CurrentIndex}; ColumnName: {csvReader.Context.HeaderRecord[csvReader.Context.CurrentIndex]}");
}
throw;
}

Related

Linq to SQL - SubmitChanges() - Check if insert was successful

I have following code -
public bool InsertUser(params)
{
User objUser = new User(params);
objDataContext.Users.InsertOnSubmit(objUser);
objDataContext.SubmitChanges();
return objUser.Id >= 0;
}
Calling method -
if (!_database.InsertUser(params))
{
//WriteErrorMessage
}
As I understand from various posts that if we want to know whether an insert was successful or not, we should check if any exception is being thrown.
However above code is relying on whether newly inserted Id is >=0 or not.
Can please guide -
If I should change above code and add a try-catch instead?
Is there any possible scenario where no error is thrown by SubmitChanges() but newly inserted Id <= 0 ?
Thank you!
If I should change above code and add a try-catch instead?
No, don't do that as in that case you will not be able to get the exact reason of failure. If you catch it then the information of failure will be lost and will not propogate to the user.
However if you think that giving exception to the user is not a good idea and you need to catch it then simply place it inside the try catch like this:
public bool SubmitChanges()
{
try{
//your code
db.SubmitChanges();
return true;
}
catch(Exception e)
{
// some code to catch exception
return false;
}
}
Is there any possible scenario where no error is thrown by SubmitChanges() but newly inserted Id <= 0 ?
If you are getting the value >=0, then there is no point to worry about this.
You can also use the GetChangeSet like this:
Gets the modified objects tracked by DataContext.
ChangeSet cs = db.GetChangeSet();
Console.Write("Changes: {0}", cs);
if your code execute the line objDataContext.SubmitChanges(); and comes on return objUser.Id >= 0; your insert will be successfull. you don't need to worry after that. let sql server and your compiler take care of the rest
Yes can use try catch to efficiently catch the error and display appropriate messages.

How to catch multiple exceptions on SaveChanges() or is it possible?

I get an XML file(from an infopath form), create an object and insert that object into the DB. Im using a C# webservice to do this parsing and it returns true if successful and currently it returns the exception message if the SaveChanges() fails. The main exceptions I'd like to return are the DbEntityValidationExceptions as I'll handle other exceptions differently. Some of the columns have max lengths so is the field exceeds that I want to return the field name that they need to edit. Can I catch all the DbEntityValidationException for all fields that failed or does entity only throw the first exception and then rollback the transaction? With 200 fields it'd be nice to tell the user which fields they need to change versus the first field and then continue to fail as they fix the single exception each time.
If it is not possible my proposed solution below is irrelevant and should be removed. If it is possible to return all the Exceptions, what am I doing wrong?
exceptionList = new List<string>();
try
{
db.SaveChanges();
}
catch (Exception ex)
{
if (ex.GetType() == new DbEntityValidationException().GetType())
{
DbEntityValidationException eValidEx = (DbEntityValidationException) ex;
foreach (DbEntityValidationResult vResult in eValidEx.EntityValidationErrors)
{
foreach (DbValidationError vError in vResult.ValidationErrors)
{
exceptionList.Add(vError.ErrorMessage);
}
}
result = false;
}
else
{
exceptionList.Add("Unknown. " + ex.Message);
}
}
You need custom exception which inherit Exception. In this exception you have property for example Errors. This property will be message collection.
When you are trying to save changes, you should know what kind of fields you are expecting. You should define in other class boundaries of the fields(example Price->decimal, mandatory, maxValue and so on) On save you should check the boundaries if one of them is not true you should add message string in Errors of the custom exception(example Price is not decimal field)
On the end if the Errors.Count > 0 thorw the CustomException. You should override the message property in the CustomException -> loop all the Errors and return the text of all of them. At the end you need only customException.Message and you will have all the problems shown to the user.
I Hope this helps.
So when you catch an exception like that, you'll only catch the first exception. Java doesn't let you keep going on when you have a bunch of Exceptions and to catch all of them.
What you could do is put the entire thing in a while loop that keeps going until you've saved all of the changes. Make sure in the try{} block that you iterate through each field instead of just trying the first field again and again.

C# if exception is caught will it reach my return statement?

In System.IO there is a function:
string File.ReadAllText( string path );
I am trying to write a function that would call File.ReadAllText, take care of all possible exceptions and return true/false and store error message.
What I have is this:
public static class FileNoBS
{
public static bool ReadAllText( string path, out string text, out string errorMessage )
{
errorMessage = null;
text = null;
bool operationSuccessful = false;
try
{
text = System.IO.File.ReadAllText( path );
operationSuccessful = true;
}
catch ( ArgumentNullException e )
{
errorMessage = "Internal software error - argument null exception in FileNoBs.ReadAllText\nMessage: " + e.Message;
}
catch ( ArgumentException e )
{
errorMessage = "Internal software error - path is a zero-length string, contains only white space, or contains one or more invalid characters as defined by InvalidPathChars in FileNoBs.ReadAllText.\nMessage: " + e.Message;
}
catch ( PathTooLongException e )
{
errorMessage = "The specified path was too long.\nMessage: " + e.Message;
}
catch ( DirectoryNotFoundException e )
{
errorMessage = "The specified directory was not found.\nMessage: " + e.Message;
}
catch ( FileNotFoundException e )
{
errorMessage = "The file specified in path was not found.\nMessage: " + e.Message;
}
catch ( IOException e )
{
errorMessage = "An I/O error occurred while opening the file.\nMessage: " + e.Message;
}
catch ( UnauthorizedAccessException e )
{
errorMessage = #"UnauthorizedAccessException
path specified a file that is read-only.
-or-
This operation is not supported on the current platform.
-or-
path specified a directory.
-or-
The caller does not have the required permission.\nMessage: " + e.Message;
}
catch ( NotSupportedException e )
{
errorMessage = "path is in an invalid format.\nMessage: " + e.Message;
}
catch ( SecurityException e )
{
errorMessage = "You do not have the required permission.\nMessage: " + e.Message;
}
return operationSuccessful;
}
}
I don't understand how how control flow goes with functions that return value.
Let's say UnauthorizedAccessException gets caught, errorMessage is set to
errorMessage = "You do not have the required permission..."
I know that finally gets executed every time, but compiler won't let me do return inside finally block. So will my return get reached or not?
Another question is how to simplify this while still following official guidelines:
"In general, you should only catch those exceptions that you know how to recover from. "
I dread going through all functions that I will need from File class (Move, Copy, Delete, ReadAllText, WriteAllText) and then Directory class and doing all these long blocks of code just to catch all exceptions I don't care about and not catch too many of them cause Microsoft says it's bad.
Thank you.
EDIT: I got comments like this is not handling exceptions this is "something else".
I am client for my code and I want to do something like this:
if ( !FileNoBS.ReadAllText( path, text, errorMessage ) ) {
MessageBox.Show( errorMessage );
return;
}
// continue working with all errors taken care of - don't care for whatever reason file wasn't opened and read, user is notified and I am moving on with my life
Your return will be reached as there isn't a return in the try block or the catch block.
Generally, you only want to catch exceptions that you expect may occur and have a way of handling them. For example, you may want to handle the file not being found from the given path and return a default file instead. You should allow other exceptions not to be caught so you know that something unexpected has happened and not hide it by catching all exceptions.
As I said in my comment, you are better off handling the exceptions at a higher level and simply displaying the exception message rather than manually setting each message. I think in this case the message from the exception will be descriptive enough.
public static class FileNoBS
{
public static string ReadAllText(string path)
{
return System.IO.File.ReadAllText( path );
}
}
then use it like this at some higher level in your application. I typically have a general handler to handle all application exceptions and log them and display a message box if necessary.
try
{
var text = FileNoBS.ReadAllText("file.ext");
}
catch(Exception e)
{
Console.WriteLine(e.Message);
}
Instead of catching the exceptions you should try to avoid the situation that will lead to those exceptions being thrown in the first place. In your case you should have some input validation before calling ReadAllText
never accept a path that is null - you know this will lead to an exception so handle it before it does
never accept a path that leads to a file that does not exist - use File.Exists(path) prior to the call
never accept a malformed path E.g. the empty string or one with invalid characters - this will lead to an exception
These tests should be performed where the input originates. That is if the user types them in, validate them before using them. If they come from a DB or somewhere else validate there before use. If it's not user input they are all indications of a system error and should be treated as such, not as something the user should worry about.
Security exceptions can be somewhat harder to test up front and in many cases it is exceptional to get a violation and therefor perfectly ok to get an exception. It shouldn't crash the program of course but be handled with an errormessage to the user (if it's based on user input, if it's system generated data that leads to this, it's an idication of a system error that should be fixed at code level). It's often more appropriate to do this where the call happens than in some library method.
for IOExceptions they can be put into two buckets. Recoverable once (usually a retry) and unrecoverable once. As a minimum give the user feedback on the exception, so the user might have the option of retrying.
A very general rule that should be part of the error correction logic is to never have invalid data floating around the system. Make sure that all objects manage the invariants (Tools are available for this such as code contracts). Reject invalid input from the user (or other systems) when they are received instead of when they result in an exception.
If you do all the input validation and still have E.g. ArgumentNullException then that points to an error in the logic of the program, something that you want to be able to easily find in a test and correct before you release the bug. You shouldn't try and mask this error.
Provided no other error occurs, yes.
I'd add at the end:
catch (Exception e)
{
errormessage = "An unexpected error has occured.";
}
return operationSuccessful;
Though, this will always return the successful even if you got an error. I'm not sure if that's what you want, or if your variables are badly named.
The return statement is going to be called in case of any exception in your code, before it is placed at the end of the program before it exits.
I will suggest placing a single exception handler with a high level Exception type, like the 'Exception' type itself, and print or log the exception message. Specifying so many exception handlers in each method is going to take a lot of energy which your should actually put in the method itself.
try
{
return ReadAllText("path", "text", "error");
}
catch (Exception exception)
{
Console.WriteLine(exception);
}
return false;
So if the method gets called, it will return immediately, otherwise the exception gets printed/logged and the method will return false.
You can however, mention a couple or few explicit exception handlers in some cases, where you think it will be beneficial.
Yes It will return the value.
But, better you handle return value in finally statement.
If in any case you want to return operationSuccessful value, then write finally block after catch blocks as follows,
finally
{
return operationSuccessful;
}

Is this a clear use of goto?

Just wondering if this is considered a clear use of goto in C#:
IDatabase database = null;
LoadDatabase:
try
{
database = databaseLoader.LoadDatabase();
}
catch(DatabaseLoaderException e)
{
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
goto LoadDatabase;
}
I feel like this is ok, because the snippet is small and should make sense. Is there another way people usually recover from errors like this when you want to retry the operation after handling the exception?
Edit: That was fast. To answer a few questions and clarify things a bit - this is part of a process which is essentially converting from a different kind of project. The _userInteractor.GetDatabaseConnector() call is the part which will determine if the user wants to retry (possibly with a different database than the one in the config they are loading from). If it returns null, then no new database connection was specified and the operation should fail completely.
I have no idea why I didn't think of using a while loop. It must be getting too close to 5pm.
Edit 2: I had a look at the LoadDatabase() method, and it will throw a DatabaseLoaderException if it fails. I've updated the code above to catch that exception instead of Exception.
Edit 3: The general consensus seems to be that
Using goto here is not necessary - a while loop will do just fine.
Using exceptions like this is not a good idea - I'm not sure what to replace it with though.
Is there another way people usually
recover from errors like this when you
want to retry the operation after
handling the exception?
Yes, in the calling code. Let the caller of this method decide if they need to retry the logic or not.
UPDATE:
To clarify, you should only catch exceptions if you can actually handle them. Your code basically says:
"I have no idea what happened, but whatever I did caused everything to
blow up... so lets do it again."
Catch specific errors that you can recover from, and let the rest bubble up to the next layer to be handled. Any exceptions that make it all the way to the top represent true bugs at that point.
UPDATE 2:
Ok, so rather than continue a rather lengthy discussion via the comments I will elaborate with a semi-pseudo code example.
The general idea is that you just need to restructure the code in order to perform tests, and handle the user experience a little better.
//The main thread might look something like this
try{
var database = LoadDatabaseFromUserInput();
//Do other stuff with database
}
catch(Exception ex){
//Since this is probably the highest layer,
// then we have no clue what just happened
Logger.Critical(ex);
DisplayTheIHaveNoIdeaWhatJustHappenedAndAmGoingToCrashNowMessageToTheUser(ex);
}
//And here is the implementation
public IDatabase LoadDatabaseFromUserInput(){
IDatabase database = null;
userHasGivenUpAndQuit = false;
//Do looping close to the control (in this case the user)
do{
try{
//Wait for user input
GetUserInput();
//Check user input for validity
CheckConfigFile();
CheckDatabaseConnection();
//This line shouldn't fail, but if it does we are
// going to let it bubble up to the next layer because
// we don't know what just happened
database = LoadDatabaseFromSettings();
}
catch(ConfigFileException ex){
Logger.Warning(ex);
DisplayUserFriendlyMessage(ex);
}
catch(CouldNotConnectToDatabaseException ex){
Logger.Warning(ex);
DisplayUserFriendlyMessage(ex);
}
finally{
//Clean up any resources here
}
}while(database != null);
}
Now obviously I have no idea what your application is trying to do, and this is most certainly not a production example. Hopefully you get the general idea. Restructure the program so you can avoid any unnecessary breaks in application flow.
Cheers,
Josh
maybe im missing something but why cant you just use a while loop? this will give you the same loop forever if you have an exception (which is bad code) functionality that your code gives.
IDatabase database = null;
while(database == null){
try
{
database = databaseLoader.LoadDatabase();
}
catch(Exception e)
{
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
//just in case??
database = null;
}
}
if you have to use goto in your normal code, you're missing logical flow. which you can get using standard constructs, if, while, for etc..
Personally, I would have this in a separate method that returns a status code of success or failure. Then, in the code that would call this method, I can have some magic number of times that I would keep trying this until the status code is "Success". I just don't like using try/catch for control flow.
Is it clear? Not really. What you actually want to do, I think, is first try to load the database and then, if that didn't work, try to load it a different way. Is that right? Let's write the code that way.
IDatabase loadedDatabase = null;
// first try
try
{
loadedDatabase = databaseLoader.LoadDatabase();
}
catch(Exception e) { } // THIS IS BAD DON'T DO THIS
// second try
if(loadedDatabase == null)
{
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
loadedDatabase = databaseLoader.LoadDatabase()
}
This more clearly illustrates what you're actually doing. As an added bonus, other programmers won't gouge out your eyes. :)
NOTE: you almost certainly don't want to catch Exception. There's likely a more specific exception that you would rather be catching. This would also catch TheComputerIsOnFireException, after which it isn't really worth retrying.
No, it's not okay: http://xkcd.com/292/
On a side note, I think there is potential for an endless loop if you always get an exception.
Technically there is nothing wrong with your goto structure, but for me, I would opt for using a while loop instead. Something like:
IDatabase database = null;
bool bSuccess = false;
int iTries = 0
while (!bSuccess) // or while (database == null)
{
try
{
iTries++;
database = databaseLoader.LoadDatabase();
bSuccess = true;
}
catch(DatabaseLoaderException e)
{
//Avoid an endless loop
if (iTries > 10)
throw e;
var connector = _userInteractor.GetDatabaseConnector();
if(connector == null)
throw new ConfigException("Could not load the database specified in your config file.");
databaseLoader = DatabaseLoaderFacade.GetDatabaseLoader(connector);
}
}

Try, Catch Problem

I've noticed this problem happening a lot in most things I do, so I'm thinking there must be a design pattern for this.
Basically if an exception is thrown, attempt to solve the problem and retry. If I place it in the try, all it will do is catch the exception, but I want to retry whatever it was doing and if it fails again, retry again a certain number of times.
Is there a common pattern for this sort of stuff?
check this SO answer.. hope that helps u
Cleanest way to write retry logic?
public static class RetryUtility
{
public static void RetryAction(Action action, int numRetries, int retryTimeout)
{
if(action == null)
throw new ArgumenNullException("action");
do
{
try
{
action();
return;
}
catch
{
if(numRetries <= 0)
throw; // Avoid silent failure
else
{
Thread.Sleep(retryTimeout);
numRetries--;
}
}
}
while(numRetries > 0);
}
}
Call
RetryUtility.RetryAction( () => SomeFunctionThatCanFail(), 3, 1000 );
Credit goes to LBushkin
This runs indefinately but it would be easy to add a loop counter to the while clause
var solved = false;
var tries = 0;
while (!solved)
{
try
{
//Do Something
solved = true;
}
catch
{
//Fix error
}
finally
{
if(solved || IsRediculous(tries))
break;
tries++;
}
}
try/catch inside a loop, with a counter for retries?
EDIT: And your requirement of "retry whatever it was doing," you need custom logic for that, how to retry varies wildly (ie, reopen a stream, recreate the object, pause for X milliseconds, etc...), so you need it's own try/catch inside a loop for every atomic operation.
By "atomic operation" I mean a set of related statements, such as read a file. The whole file read into memory might be an atomic operation, for example.
On some limited basis, you might want to put your try/catch into a loop, and force break if is ultimately successful. Such might be for internet access testing and you want user to have another attempt at connection.
Something like this, maybe:
int MAX_RETRIES = 5;
for (var attempt=1; attempt <= MAX_RETRIES; attempt++) {
try {
DoSomethingThatMightThrow();
}
catch (AnExceptionIKnowHowToHandle) {
if (attempt < MAX_RETRIES)
continue;
throw;
}
}
Depends what you are trying, but typically you want to check for the possibility of an exception happening PRIOR to executing the code that could cause an exception.
For example, check that a file exists before accessing it, and create it (or whatever) if it doesn't.
Are you sure exception handling is the proper methodology here? If you can "solve the problem" you can probably detect the error condition prior to calling the exception-generatiing code.
Exception handling is most natural for things which are truly exceptional. A failed Internet connection (as in the previous answer) is something that can be detected and handled before calling exception-throwing code.
Yes, it is quite common to have a loop with a number of retries where you break out of the loop on success. A couple of things:
You might want to add a delay before retrying so that you don't use up all your retries in just a few milliseconds before the temporary problem had time to fix itself.
If you eventually fail, you should throw the first exception you caught, not the last one. The second exception could be the result of failing to recover correctly from the first failure and might not help to debug the original problem.
Coding what others have already mentioned:
var success = false;
var attempts = 0;
var maxAttempts = 0;
do {
attempts++;
try {
/* your code */
success = condition;
} catch(SuperciliousException e) {
/* recover */
}
} while(!success && attempts < maxAttempts);

Categories