We Keep Coding

Is it normal to use many business exceptions if needed?

Is it a good practice to define and throw custom exceptions even if the application needs lots of them?
EntityNotFoundException
EntityAlreadyExistsException
EntityNotUniqueException
EntityNotAddedException
EntityNotUpdatedException
EntityNotDeletedException
QuizOverException
QuizExpiredException
TableAlreadyBookedException
EndDateMustBeGreaterThanStartDateException
I tried to name these sample exception names to describe their purpose as good as I could. I hope they could form an idea of what I am trying to ask.
Don't limit your imagination with only these exceptions but all that could arise during you application's life. Consider both CRUD and business exceptions.
I know that throwing and catching exceptions is an expensive process in terms of performance but don't they provide a more elegant way to develop you app?
Isn't it better to throw a EntityNotFoundException instead of writing an if statement to check whether the entity is null?
Isn't it better to throw a EntityAlreadyExistsException instead of writing an additional if statement which will call a method to check whether the entity with the given Id already exists?
Isn't it better to throw a EntityNotAddedException instead of checking the return value of type bool specifying whether the transaction was successful or not? What if we want to return an object?
I feel that the answer will be like "you should not use EntityNotFoundException but instead check if null, but you should use EntityAlreadyExistsException", "there is no holy grail".
I am wondering what is the elegant way of doing this?

Keeping in mind that exceptions are supposed to represent exceptional circumstances all of your questions can only really be answered with - it depends.
The context of when & where you intend on throwing a particular exception will naturally decide whether it makes sense. For example, if you attempt to retrieve an entity that should exist but doesn't, then throwing an EntityNotFoundException would be considered appropriate because we now have an exceptional circumstance. On the other hand, if you are checking whether the entity already exists before creating a new one then you could argue that because we know there is a chance the entity may or may not exist then it's not really an exceptional circumstance.
Like I said, it really depends on the context of the situation and the nature of your application whether you should throw an exception or not, however, the one thing you don't want to end up doing is controlling the program flow with exceptions.
To help make the distinction between when it's suitable to use an exception vs business logic, simply ask yourself "is this particular situation valid?" or in other words "is it OK for the application to find itself in this state?". If the answer is yes, use logic to control the flow of the application and deal with the situation, otherwise you want to throw an Exception to effectively interrupt the program flow and inform the user that something isn't quite right.

When creating exceptions ask about their added value.
Will someone care about specific types of exception to be caught?
Will exceptions have different fields to help exception handling?
More abstract exception with custom message can save you time writing exceptions with no value.
Using exceptions to control program flow is considered bad idea. Here are some reasons:
Exceptions have been created for concept of error handling
Performance as you said
you can forget to handle some exceptions
If you use checked exceptions you may need to write handlers for exceptions you do not care about or don't know how to handle.
Exceptions increase uncertainty in programs (exceptions thrown while exception handling, finally statements,...)
There are other ways to solve conditional statements, take a look at scala programming language and its use of "monads".

One major limitation with the exception-handling paradigm in C++, which was inherited by Java and subsequently .NET, is that if a call to Foo throws a BozException, that can have two very different meanings:
Some condition was detected during the execution of Foo which implied that it should throw a BozException.
Foo called some method which it was not expecting to throw a BozException, but the method threw one anyway and Foo didn't catch it, the exception was thrown out of Foo.
Even though the Framework guidelines discourage the use of custom exceptions when an existing exception would seem to "fit", the lack of any standard means for code that catches a Framework-defined exception to know whether it actually represents an expected circumstance is a pretty big disadvantage to using a Framework exception (like InvalidOperationException) to report business-logic circumstances a caller might want to handle (e.g. trying to add to a Dictionary to records with the same ID). If you can tolerate the boilerplate associated with defining exceptions, I'd suggest that it's better to err on the side of too many rather than too few (though you may wish to use inheritance for exceptions that will likely be handled the same way).

As with almost everything in software development, given a general rule or best practise the skill is knowing when and how to apply it.
Should I program to an interface? YES! Should I make every single class I write implement a corresponding interface and only ever program to that abstraction? I could do but writing a very simple application would take me an age and my productivity would grind to a halt.
Single Responsibility - is it a good thing? YES! Does every class I write have exactly one and only one responsibility. No - partly because of my own failings as a programmer, but also because I'd end up with a plethora of single-method classes and an unmanageably disconnected code base.
Now let us turn to error handling:
Firstly, by looking at your comment to James' answer, let us clarify that using error codes and exception handling represent two distinct models of handling errors in your application and as a general rule they should not be mixed.
Let us assume you are using exception handling and make the following argument:
Why do we throw exceptions?
We throw an exception because something bad has happened. If we are expecting a scenario to occur some point, then it is within the flow of our application, and is therefore not exceptional - so throwing an Exception would not be appropriate!
With this argument, it makes little sense to throw anything other than the top level Exception class, as any other exception implies that we have foreseen this scenario and have been able to attach extra metadata to the exception.
Why handle exception?
Having thrown an exception, we expect someone somewhere to deal with it, hopefully with the intention of restoring the application to a consistent state. However, if the exception is truly exceptional - we can have no way of knowing what state has been corrupted so exception handling is futile.
Isn't this taking things a bit too far?
Well - this is exactly how error handling works in a service orientated architecture. A robust service should not reveal any incriminating details of what has happened when an exception occurs (it's a security breach and introduces coupling between the service and the client) - all the client needs to know is something bad happened.
However, I'm guessing that we are working within an Object Orientated environment, where we are prepared to accept a slightly higher degree of coupling between an implementation and it's consumers. And it is this that is the important point - by introducing an exception hierarchy which exposes detailed information about the exception you increase the coupling in your application (the consumers need to have detailed knowledge about all the exceptions you could throw).
In software we strive for a loosely coupled code base, making it easy to maintain and extend. I have no doubt there is a happy medium between the SOA approach to exceptions and the one you describe in your question - finding that sweet spot is the skill of the developer.

How to descern fatal from non fatal .NET Exceptions

I'm writing a general discrete event system simulation library in C#. I'm going to write another library on top of that which will implement a specific kind of discrete event sim. Here is an integral version of the code.
static class Engine
{
[ThreadStatic] internal static uint time;
//...
public static void Run(OnException onException = OnException.PauseAndRethrow,
IList<Type> exceptionsToPass = null)
{
//...
while (!stop)
{
Event e = currentTimeEventQueue.PopEvent();
if (e == null) break;
try {
e.Activate();
}
catch (EngineException ex)
{
// An engine method that shouldn't have been called
// was called, and Activate didn't handle that
// handle this situation...
}
catch (/* fatal exception type*/ ex)
{
throw;
}
catch (Exception ex)
{
// code to decides whether to dismiss exception
// pause, stop, reset based on parameters to this method
}
}
}
}
The question is: Should I specifically catch Exception types that are known to be unrecoverable (that I shouldn't try to handle in any way). What are those Exceptions (I can think of OutOfMemoryException and StackOverflowException). Is there a list of fatal Exceptions? As I remember some of them are uncatchable. So I am interested in a list of fatal Exceptions that can be caught. I just want to rethrow them, and not do anything. On the other hand I want to handle any other type of Exception. Or maybe I need another angle on this.
EDIT: OK, I've made a huge oversight when writing the question. Activate() is abstract. I am writing a general purpose discrete event system simulation library. The engine is working with totally unknown subclasses of Event. So it is calling an absolutely unknown method Activate(), that could throw any kind of exception. I could just ignore this issue, but I want to give control of the process to the caller. As you can see from the parameters to the Run() method, the caller decides what the engine will do if an exception comes from a call to Activate() (it can instruct the engine to ignore and continue, or pause and rethrow, or ...). That is why I am trying to separate fatal from all other Exceptions. If the caller has instructed the engine to ignore any exception that comes from Activate() it would be unwise to catch and ignore fatal exceptions. (there be dragons :) )

Should I specifically catch Exception types that are known to be unrecoverable
No. You shouldn't. If they are unrecoverable you shouldn't try to recover from them.
The rule regarding exceptions is - catch and handle exceptions that you know how to recover from. Let any other bubble up - if this means an application crash, this is probably for the best.
The following is a code smell and shouldn't be coded:
catch (/* fatal exception type*/ ex)
{
throw;
}

No throwable is really "uncatchable"; anything that can be thrown in .NET derives from Exception and so can be caught. However, many things shouldn't be caught. Hence your question; how to tell the difference?
The general rule I follow is "Catch the exceptions that you expect and know how to deal with". This requires you to first know what your code could throw. The MSDN docs are usually pretty good at stating what various framework methods will throw in what situations; your own codebase is likely less well-documented unless you're developing an intermediate library to be consumed by other coders (or your management/leads are anal about proper documentation).
Once you know what the code can throw, determine what, if anything, you should catch. Exception trapping, aka "Pokemon handling" (Gotta catch 'em all) is generally a bad thing because there are legitimate reasons to let your application die a fiery death and make the user restart it. Examples include StackOverflowExceptions, OutOfMemoryExceptions, and various Win32Exceptions detailing some internal failure to provide your program with the requested resources. You usually cannot recover from these in any meaningful way.
Most errors, however, are not that serious. Exceptions are thrown when a file isn't found, or when a network connection is refused or unexpectedly closed, or when you attempt to do something with a type without checking for null (and what happens when the check for null fails? Often you can't continue and must throw an exception of your own). These are things that you should be expecting, catching, and at the very least communicating to the end user in some understandable way.
In many cases, try/throw/catch is a useful tool for expected but not everyday situations; if you receive a timeout exception while waiting for results when usually there's no problem getting results, then don't even tell the user there was a problem; just try again. If you can (should?) plug in some default value when a calculation can't be evaluated (divide by zero, math that could produce non-real results but the code can't handle them, etc etc), then do so.
There is, however, one scenario I can think of where you simply must catch and rethrow, and that's in situations involving database transactions. If you're performing a large "unit of work" with a database transaction handling several persistence operations along the way, then if anything goes wrong, you should roll back the DB transaction. In that case, the operation should be surrounded in a try-catch(Exception) block that catches the exception, rolls back the transaction, and rethrows. Any exception that you can proceed through normally should be handled with a nested try-catch or the condition should be checked for prior to the operation that could fail in that way.

Your question is a reasonable one to ask, but alas there is often no really good answer. A major weakness with the exception paradigm in C++, which unfortunately follows through into other languages and frameworks which borrow from it, is that it encapsulates way too much in the type of an exception object. In C++, this is understandable, since Bjarne Stroustrup wanted to avoid having any non-primitive types be "hard-coded" into the language; given that constraint, the C++ design may have been the best possible. Nonetheless, such a design imposes some serious limitations.
The biggest problem is that code which is considering catching and processing an exception, and then either swallowing or rethrowing it, may be interested in a number of things:
What happened that prevented code from behaving as expected
What actions should be taken in response to the condition
Once various actions have been taken, should the condition be considered "resolved"
Should one adjust one's assumptions about any aspects of the system state beyond what is implied by the method's failing (e.g. if file "foo/bar" couldn't be read, the nature of the failure may impact a decision whether to try reading "foo/boz").
In Java and .net, the expectation is that the exception type should be used to indicate #1 above, and there's no standard means for exceptions to answer #2 and #3, even though they are in many cases far more important than "what happened". If one tries to load a document and something goes wrong, 99% of the time the system will be in the same state as it was before the load attempt, so the exception, regardless of its type, will basically mean "file isn't readable and wan't loaded". Further, in that remaining 1% of the time where something "bad" happens beyond the failure to load the file, it's entirely possible that the exception type will be the same as one that's thrown in the 99%.
Absent some new exception-handling features, one's best bet is probably, to the extent possible, have exceptions that indicate state corruption invalidate the corrupted parts of system state such that all future operations on those parts will likewise throw exceptions. Such behavior will mean that exceptions that code can't handle will end up bringing down the system (since the code will keep trying to use parts of the state that have become corrupted), while allowing code to recover from things it should recover from (e.g. if an exception occurs because of corruption in an object which was being built, and if the exception causes code to abandon the object under construction, such abandonment would suffice to "handle" the exception, and so execution can and should continue).

What can we do with custom exception?

I don't know what we can do with a custom exception, what we can't do with a built-in one. It seems a naive question but I really have no idea about that.
What do you think?

The reason for the different types of exceptions is to allow you to be able to catch just the ones you want with your handlers, letting the others go on up the stack. So you can arrange to catch exceptions for certain, occasionally-expected situations just by the type of the exception.
You may not need to create your own very often at all, actually. But if you do, it would be because you need to be able to throw and capture an exception type more specific than what is available, and perhaps with additional information attached.

It is useful to create a custom exception if:
There is no built-in exception that expresses the type of error condition you have.
You want to catch only that specific type of exception and not exceptions coming from the framework.
But usually if there already is an exception in the framework that you could use then it is better to use it instead of creating your own exception for the same thing.

You could use it for implementing special error handling for things related to your application. Suppose you build a banana application, then you could have an OutOfBananasException. If your application gets out of bananas you can throw the exception and catch it later on with special error handling.
try
{
EatBananas();
}
catch(OutOfBananasException oobe)
{
GetMoreBananas();
}
catch(Exception e)
{
TellUserAndAbort();
}
Edit:
The reason to use your own Exceptions instead of the built in is to make it clear to everyone reading you code or using your library what type of error has occurred. You should only create your own exceptions when you can not find any suitable built in exception.
Edit2:
One thing you can do with your own exceptions that you can not do with built in is to add properties describing things about the error condition that the error handler might use. If you have an exception relating to customers you exception could have properties for customer name and customer id and thus make it possible for the error handler to display informative error messages to the user.

There's only a few exceptions in .NET that are treated in a special manner, like ThreadAbortException, which can't (normally) be caught and handled and swallowed.
Other than that, exception types are just exception types. You can do pretty much the same with your own exceptions that you can do with the ones defined in the framework.

The benefits of custom exceptions have been outlined here, but before you create your own make sure the BCL doesn't already have one fitting your needs:
http://mikevallotton.wordpress.com/2009/07/08/net-exceptions-all-of-them/
(There's 141 of them!)

One annoyance with the built-in exceptions is that there is no systematic distinction made between exceptions that indicate that
an operation failed but a retry might succeed; the system state is as it was before the operation was attempted.
an operation failed, and a retry isn't likely to help; nonetheless, the system state is as it was before the operation was attempted;
an operation has failed in such a way as to potentially corrupt something else.
It may be useful to catch exceptions and rethrow one of three custom exceptions (meanings defined above) based upon where the exception was caught. When the exception is rethrown, pass the original exception as the InnerException parameter.
Incidentally, it's possible to define generic exceptions. I'm not quite sure of the pros and cons of doing so, and I've never seen anyone else do it. One could define e.g. a TransientFaultException(of T) which inherits from a (custom) TransientFaultException; an application which catches a TimeoutException could rethrow as a TransientFaultException(of TimeOutException) and have it caught as either a TransientFaultException(of TimeoutException) or a TransientFaultException. Unfortunately, one would have to know the type of the exception to be taught to create the proper generic. If one were to catch an Exception and pass it to a factory method for TransientFaultException, the new exception would be of type TransientFaultException(of Exception), regardless of what type of exception was originally thrown.

Custom exceptions allow you to do 2 things:
Capture custom typed-data in the exception
Capture a custom exception occurrence
You should only create a custom exception when there isn't a built in exception to handle as necessary.
As an example, in our application we have DataLayerException which is thrown when the datalayer encounters an error (and includes the specific DBMS exception as an inner exception).
We also have DataLayerSingleResultNoneException - which is when we expect a single result back but there is no result, and DataLayerSingleResultManyException which is when we expect a single result but get many back. This allows us to catch the different problems an action them accordingly.

Create custom exception or use built-in exceptions?

Currently I'm in the process of writing a client class that utilizes DNS, Sockets, and SSL among other classes that love to throw exceptions. Other people will be implementing this class, so I was wondering what the best practice is for throwing exceptions.
Should I create my own custom exception so they know that it is my class throwing the exception or should I allow the classes and methods I call (DNS, Sockets, etc.) to throw their own exceptions? Currently, the code is in the hundreds of lines and growing with many different method calls. What is the best practice for throwing exceptions in this situation?

If the BCL contains classes that already convey the meaning you want (ArgumentNullException, for instance), use those.
Reserve using your own exception classes for things that are specific to your API.
If you feel that you can add information, by all means raise your own exception but do not swallow exceptions - propagate them up as inner exceptions of your own.

It is fine to throw your own exceptions if they add information, but don't swallow the exceptions from the underlying services unless there is a security concern.
If you are catching one exception and and throwing a new one, assign the old one to the InnerException property of the new one. You can nest as many Exceptions as necessary in this way, which creates a hierarchical "view" into how the exception propagated, which is very helpful for debugging.

The worst thing you can do is throw an ApplicationException with details in the message string. IF you find yourself needing to do that, it's time for a custom exception.

It really depends on your audience, i.e. the consumers of your class.
For example, if you are essentially wrapping a lot of different exceptions, it may be a good idea to create custom exceptions that would simplify the error handling in the consumer.
If you do create custom exceptions, be sure to include the original exception into the InnerException of your custom exception, unless you explicitly have a reason to hide it. This will people using your class the most information available, as well as cover you if an exception comes back that your class doesn't completely cover.

If you want anyone to catch and recover from your exceptions, it's probably best to use a small hierarchy of custom exception types, probably organized by the expected degree of recoverability (e.g. have an exception type indicating 'something unexpected happened, but the socket is probably still good', another for 'the socket state cannot be trusted, but starting over with a new socket connection might work', and another for 'the host says what you're doing won't work; don't even bother retrying unless you have reason to believe something has changed'). The particulars of what caused an exception are often less important to the 'catching' code than the nature of the violated post-conditions.

Defining exceptions that can be thrown from a toolkit API

Is there a best-practice or industry standard for throwing exceptions from a toolkit API?
Should the user facing methods catch and wrap up Exception in some sort of CustomException so that users only have to worry about CustomExceptions coming out of the API?
Or is the convention to just let those bubble up?
We're concerned with being able to document all possible exceptions our API methods might throw. (For example, if our API method calls Stream.Write() which throws 4 or 5 exceptions, we'd have to document all of those in addition to other exceptions that other called methods might throw.)
We were thinking of doing something like this:
public void customerFacingApiMethod(){
try {
//api functionality goes here
} catch (Exception e) {
throw new CustomException(e);
}
}

In my opinion having the API throwing only one exception type is a bad idea. One of the good things with different exceptions is that you can choose to catch different types of exceptions and handle them differently. Wrapping up exceptions into a single exception type would remove that facility.
Use the exception types provided by the framework where appropriate, and create your own custom exception types for specific situations where appropriate. And above all, make sure to document for each method which exceptions they may throw.

When throwing exceptions, make sure that the user-facing exceptions are all relevant to what the user actually did wrong with regards to your toolkit. That might mean catching or merging different filesystem exceptions into a single exception but you shouldn't ever just catch Exception and throw a new one - that's not actually telling the toolkit user what they did wrong.

You should follow the same concepts as the .NET Framework. Your users already use the framework, so they know how it works and expect certain behavior.
Throw an ArgumentException-derived exception if an argument from the client is invalid (use ArgumentNullException, ArgumentOutOfRangeException, etc. as appropriate).
Throw an IOException-derived exception for IO errors.
Etc...
In your documentation, clearly state the preconditions for your public API and state the exceptions that will be thrown when they fail (like MSDN does).

I don't like it. If you wrap all exceptions into a CustomException it will be difficult to catch specific exceptions without searching inside the object for the InnerExcpetion (possibly multiple levels deep if your API also uses other APIs that do the same thing). If there's an OutOfMemoryException, I'd like to know that without having to go searching for it.

Don't use the same exception for all.
You need the different exception types to be able to separate one case from another. For anything that you'd let it flow up until a generic exception handler block catches it would seem k, but you are preventing any other code recovery - extra actions that operate on specific cases.
You can wrap a set of exceptions, if those fit into a specific scenario that can be handled by the calling code. Don't forget that some framework exceptions already communicate the situation well.

What Fredrik Mork says is very true. I also like to add to that that you can easily document the exceptions that are thrown with xml comments (and GhostDoc).
So an API user can lookup in the documentation to see what exceptions can be thrown. However, there is a downside.
If you have a deep callstack inside your API and especially if the cyclomatic complexity becomes high (i.e. num of possible branches) you cannot recover all possible exceptions that can be thrown (perhaps by the runtime?)
So i would recommend to only throw if something if recovery is not possible and only throw in the upper layer of your API, i.e. no deeper than 2 deep. And catch all exceptions that are thrown deeper in your callstack and return them as InnerException or something like that.

There are two types of exceptions to consider (ignoring StackOverflowException, etc., which you can't do anything about anyway):
Those that are caused by external issues (like file IO), which have to be handled.
Those that can always be avoided with properly validated input.
It usually makes sense for the IO-type exceptions to be passed up the call stack unmodified, since there isn't anything you can really do about it anyway. Exceptions to this would be anticipated errors for which you might do retries, or asking the user what to do, if your API operates at the GUI level (which would be rather unusual).
Just be sure to document which exceptions of this type your API methods can throw.
Of the second type (exceptions that can always be prevented), these exceptions may be generated by your API on bad input, but should never be passed up the call-stack from the lower levels. Input to anything you call should be validated so that these errors do not occur, and if, for some reason, they can occur, then these exceptions should be wrapped up. Otherwise, the code that uses your API will have to deal with exceptions at the wrong level of abstraction.
And once again, be sure to document what sort of input will not generate exceptions, and what kinds of exceptions to expect if these rules are broken.
In all cases, throw exceptions that make sense to the user of your API, not the person programming the API itself.

The problem with catching System.Exception in the general case is that by doing so, you're effectively saying "I don't have any idea of why this failed, and that's okay, because I know I can continue anyway!"
That's rarely, if ever, true.
Wrapping up all exceptions in a single custom exception type means that your consumers will just catch that type - which is then the moral equivalent of catching System.Exception. While you may satisfy some policy/FXCop rule by doing that, you're really still just catching System.Exception.
For your exception policy, I'd start with asking this: "If I didn't know what this API actually talked to, what kind of exception would I expect to get back?" And remember that a method is a request to an object to do something, and an exception is the object's way of letting you know that it couldn't, and why.

If you create a single exception type for use throughout your API, its meaning will be vague to client applications that encounter that exception type. Instead, consider creating an exception class hierarchy for your API.
To do so, first define an abstract base exception type:
CustomException
...and then derive more specific exception types from it. For example:
CustomFooException
CustomBarException
CustomFizException
CustomFuzException
The abstract base type CustomException will never be directly thrown from your API. However, a client of your API can catch the base exception type CustomException if it doesn't care about which specific derived exception was thrown from your API.
For more on the idea of developing an exception class hierarchy, see this answer and other answers to this question: Why create custom exceptions in .NET?
Also see this answer which introduces the idea of making the base class of your exception hierarchy abstract.