We Keep Coding

Calling an external API whenever a field in the database is updated

My ApplicationUser model contains a property:
public bool SubscribedToNewsletter { get;set; }
I would like to make sure that whenever I update its value in the database, an external API will be called to add or remove the user from a list in my email automation system, without manually calling the method myself to ensure synchronization regardless of programmer's intention.
Is there a built-in functionality provided in ASP.NET? Or do I have to extend the UserManager class and centralize all the calls updating the database?

Calling an external API to keep in sync with your application data is a little more complicated than making a simple change in a domain model.
If you did this, would you call the API before or after you persist changes to the database? If before:
How do you make sure that the change is going to be accepted by the DB?
What if the API call fails? Do you refuse to update the DB?
What if the API call succeeds but the application crashes before updating the DB or the DB connection is temporarily lost?
If after:
The API could be unavailable (e.g. outage). How do you make sure this gets called later to keep things in sync?
The application crashes after updating the DB. How do you make sure the API gets called when it restarts?
There are a few different ways you could potentially solve this. However, bear in mind that by synchronising to an external system that you have lost the ACID semantics you may be used to and your application will have to deal with eventual consistency.
A simple solution would be to have another database table that acts as a queue of API calls to be made (it's important this is ordered by time). When the user's email is updated, you add a row as part of the DB transaction with the relevant details needed. This ensures the request to call the API is always recorded with an update.
Then you would have a separate process (or thread) that polls this table. You could use pg_notify to support push notifications rather than polling.
This process can read the row (in order) then call the relevant API to make the change in the external system. If it succeeds, it can remove the row. If it fails, it can try again using an exponential back-off. Continued failures should be logged for investigation.
The worst case scenario now is that you have at-least-once delivery semantics for updating the system (e.g. if API call succeeded but process crashed before removing the row then the call would be made again when process restarted). If you needed at-most-once, you would remove the row before attempting to make the call.
This is obviously glossing over some of the details and would need modified for a high through-put system but should hopefully explain some of the principles.

I usually tackle this sort of thing with LISTEN and NOTIFY plus a queue table. You send a NOTIFY from a trigger when there's a change of interest, and insert a row into a queue table. A LISTENing connection notices the change, grabs the new row(s) from the queue table, actions them, and marks them as completed.
Instead of listen and notify you can just poll a queue table, listen and notify are an optimisation.
To make this reliable, either the actions you take must be in the same DB and done on the same connection as the update to the queue, or you need to use two-phase commit to synchronise actions. That's beyond the scope of this sort of answer, as you need a transaction resolver for crash recovery etc.
If it's safe to call the API multiple times (it's idempotent), then on failure midway through an operation it becomes fine to just execute all entries in the pending queue table again on crash recovery/restart/etc. You generally only need 2PC etc if you cannot safely repeat one of the actions.

How do I minimize or inform users of database connection lag / failure?

I'm maintaining a ASP/C# program that uses an MS SQL Server 2008 R2 for its database requirements.
On normal and perfect days, everything works fine as it is. But we don't live in a perfect world.
An Application (for Leave, Sick Leave, Overtime, Undertime, etc.) Approval process requires up to ten separate connections to the database. The program connects to the database, passes around some relevant parameters, and uses stored procedures to do the job. Ten times.
Now, due to the structure of the entire thing, which I can not change, a dip in the connection, or heck, if I put a debug point in VS2005 and let it hang there long enough, the Application Approval Process goes incomplete. The tables are often just joined together, so a data mismatch - a missing data here, a primary key that failed to update there - would mean an entire row would be useless.
Now, I know that there is nothing I can do to prevent this - this is a connection issue, after all.
But are there ways to minimize connection lag / failure? Or a way to inform the users that something went wrong with the process? A rollback changes feature (either via program, or SQL), so that any incomplete data in the database will be undone?
Thanks.

But are there ways to minimize connection lag / failure? Or a way to
inform the users that something went wrong with the process? A
rollback changes feature (either via program, or SQL), so that any
incomplete data in the database will be undone?
As we discussed in the comments, transactions will address many of your concerns.
A transaction comprises a unit of work performed within a database
management system (or similar system) against a database, and treated
in a coherent and reliable way independent of other transactions.
Transactions in a database environment have two main purposes:
To provide reliable units of work that allow correct recovery from failures and keep a database consistent even in cases of system
failure, when execution stops (completely or partially) and many
operations upon a database remain uncompleted, with unclear status.
To provide isolation between programs accessing a database concurrently. If this isolation is not provided, the program's outcome
are possibly erroneous.
Source
Transactions in .Net
As you might expect, the database is integral to providing transaction support for database-related operations. However, creating transactions from your business tier is quite easy and allows you to use a single transaction across multiple database calls.
Quoting from my answer here:
I see several reasons to control transactions from the business tier:
Communication across data store boundaries. Transactions don't have to be against a RDBMS; they can be against a variety of entities.
The ability to rollback/commit transactions based on business logic that may not be available to the particular stored procedure you are calling.
The ability to invoke an arbitrary set of queries within a single transaction. This also eliminates the need to worry about transaction count.
Personal preference: c# has a more elegant structure for declaring transactions: a using block. By comparison, I've always found transactions inside stored procedures to be cumbersome when jumping to rollback/commit.
Transactions are most easily declared using the TransactionScope (reference) abstraction which does the hard work for you.
using( var ts = new TransactionScope() )
{
// do some work here that may or may not succeed
// if this line is reached, the transaction will commit. If an exception is
// thrown before this line is reached, the transaction will be rolled back.
ts.Complete();
}
Since you are just starting out with transactions, I'd suggest testing out a transaction from your .Net code.
Call a stored procedure that performs an INSERT.
After the INSERT, purposely have the procedure generate an error of any kind.
You can validate your implementation by seeing that the INSERT was rolled back automatically.
Transactions in the Database
Of course, you can also declare transactions inside a stored procedure (or any sort of TSQL statement). See here for more information.

If you use the same SQLConnection, or other connection types that implement IDbConnection, you can do something similar to transactionscopes but without the need to create the security risk that is a transactionscope.
In VB:
Using scope as IDbTransaction = mySqlCommand.Connection.BeginTransaction()
If blnEverythingGoesWell Then
scope.Commit()
Else
scope.Rollback()
End If
End Using
If you don't specify commit, the default is to rollback the transaction.

How to rollback transaction at later stage?

I have a data entry ASP.NET application. During a one complete data entry many transactions occur. I would like to keep track of all those transactions so that if the user wants to abandon the data entry, all the transaction of which I have been keeping record can be rolled back.
SQL 2008 ,Framework version is 4.0 and I am using c#.

This is always a tough lesson to learn for people that are new to web development. But here it is:
Each round trip web request is a separate, stand-alone thread of execution
That means, simply put, each time you submit a page request (click a button, navigate to a new page, even refresh a page) then it can run on a different thread than the previous one. What's more, even if you do get the same thread twice, several other web requests may have been processed by the thread in the time between your two requests.
This makes it effectively impossible to span simple transactions across more than one web request.
Here's another concept that you should keep in mind:
Transactions are intended for batch operations, not interactive operations.
What this means is that transactions are meant to be short-lived, and to encompass several operations executing sequentially (or simultaneously) in which all operations are atomic, and intended to either all complete, or all fail. Transactions are not typically designed to be long-lived (meaning waiting for a user to decide on various actions interactively).
Web apps are not desktop apps. They don't function like them. You have to change your thinking when you do web apps. And the biggest lesson to learn, each request is a stand-alone unit of execution.
Now, above, I said "simple transactions", also known as lightweight or local transactions. There's also what's known as a Distributed Transaction, and to use those requires a Distributed Transaction Coordinator. MSDTC is pretty commonly used. However, DT's perform much more slowly than LWT's. Also, they require that the infrastructure be setup to use a DTC.
It's possible to span a transaction over web requests using a DTC. This is done by "Enlisting" in a Distribute Transaction, and then somehow sharing this transaction identifier between requests. But this is a lot of work to setup, and deal with, and has a lot of error prone situations. It's not something you want to do if you have other options.
In general, you're better off adding the data to a temporary table or tables, and then when the final save is done, transfer that data to the permanent tables. Another option is to maintain some state (such as using ViewState or Session) to keep track of the changes.
One popular way of doing this is to perform operations client-side using JavaScript and then submitting all the changes to the server when you are done. This is difficult to implement if you need to navigate to different pages, however.

From your question, it appears that the transactions are complete when the user exercises the option to roll them back. In such cases, I doubt if the DBMS's transaction rollback semantics would be available. So, I would provide such semantics at the application layer as follows:
Any atomic operation that can be performed on the database should be encapsulated in a Command object. Each command will implement the undo method that would revert the action performed by its execute method.
Each transaction would contain a list of commands that were run as part of it. The transaction is persisted as is for further operations in future.
The user would be provided with a way to view these transactions that can be potentially rolled back. Upon selection of a transaction by user to roll it back, the list of commands corresponding to such a transaction are retrieved and the undo method is called on all those command objects.
HTH.

You can also store them on temporary Table and move those records to your original table 'at later stage'..

If you are just managing transactions during a single save operation, use TransactionScope. But it doesn't sound like that is the case.
If the user may wish to abandon n number of previous save operations, it suggests that an item may exist in draft form. There might be one working draft or many. Subsequently, there must be a way to promote a draft to a final version, either implicitly or explicitly. Think of how an email program saves a draft. It doesn't actually send your message, you may abandon it at any time, and you may recall it at a later time. When you send the message, you have "committed the transaction".
You might also add a user interface to rollback to a specific version.
This will be a fair amount of work, but if you are willing to save and manage multiple copies of the same item it can be accomplished.
You may save the a copy of the same data in the same schema using a status flag to indicate that it is a draft, or you might store the data in an intermediate format in separate table(s). I would prefer the first approach in that it allows the same structures to be used.

How is rolling back transaction related to LINQ to SQL?

The question is solely about rolling back the changes, not commiting.
Let's say I fetch some data, I change them, I submit changes (optional step) and I roll back transaction. Wherever you look every author writes, this cancels the changes.
But I found out that is half true -- LINQ DataContext will keep the changed data! I tested this using TransactionScope and DataContext.Transaction. In both cases I got the same behaviour.
A workaround would be to recreate DataContext after roll back (however this leads to other problems like caching data and handling nested transactions) or manually discarding the changes in DataContext. Nevertheless those are just workarounds.
Questions
So what am I missing? Is LINQ to SQL not suited for transactions? How to use transactions so they would REALLY roll back changes?
Example
MyTable record = null;
db.Connection.Open();
using (db.Transaction = db.Connection.BeginTransaction())
{
record = db.MyTable.First();
record.BoolField = !record.BoolField; // changed
db.SubmitChanges();
db.Transaction.Rollback();
}

A data-context should be considered as a unit-of-work. How granular you make that is up to you - it could be a page request, or a single operation; but - if you get an exception (or pretty much anything unexpected) - stop; abandon the data-context and rollback. After a rollback, your data-context is going to be confused, so just don't keep it.
Additionally; don't keep a data-context for longer than necessary. It is not intended as an app-long data cache.

What you seem to be asking for is an in-memory cache of the database (or some part of it) rather than a lightweight ORM. I would say that LINQ to SQL is just fine for transactions and as a lightweight ORM, but not so good to use out of the box as a database cache. The data context functions best, in my opinion, using the Unit of Work pattern. Create the context for a particular task, perform the task, then dispose of the context. If the task happens to include a failed transaction, then you need to figure out how to respond to the failure. This could be by either correcting the errors and retrying with the existing context or, as in a web context, passing back the attempted changes to the user, then trying again with a new context when the data is resubmitted.

Two things:
1) stale datacontext
What you observe is commonly refered to as a 'stale' datacontext. The entities in the datacontext do not notice your rollbak. You would get simular behaviour if you would execute a stored procedure after your submitchanges. That will also not be noticed by the datacontext. However, your transactions will be rolled back in the DB! (and likewise the stored procedure will be executed)
2) about transactions
There is no need to manage the transaction. Linq2Sql already creates a transaction for you in the Submitchanges.
If you really want to manage the transactions (e.g. over multiple datacontexts or a stored procedure combined with some linq2sql, wrap the whole thing in a TransactionScope. Call transaction.Complete() at the point where you want to commit.

NHibernate session management?

Firstly, let me give a brief description of the scenario. I'm writing a simple game where pretty much all of the work is done on the server side with a thin client for players to access it. A player logs in or creates an account and can then interact with the game by moving around a grid. When they enter a cell, they should be informed of other players in that cell and similarly, other players in that cell will be informed of that player entering it. There are lots of other interactions and actions that can take place but it's not worth going in to detail on them as it's just more of the same. When a player logs out then back in or if the server goes down and comes back up, all of the game state should persist, although if the server crashes, it doesn't matter if I lose 10 minutes or so of changes.
I've decided to use NHibernate and a SQLite database, so I've been reading up a lot on NHibernate, following tutorials and writing some sample applications, and am thoroughly confused as to how I should go about this!
The question I have is: what's the best way to manage my sessions? Just from the small amount that I do understand, all these possibilities jump out at me:
Have a single session that's always opened that all clients use
Have a single session for each client that connects and periodically flush it
Open a session every time I have to use any of the persisted entities and close it as soon as the update, insert, delete or query is complete
Have a session for each client, but keep it disconnected and only reconnect it when I need to use it
Same as above, but keep it connected and only disconnect it after a certain period of inactivity
Keep the entities detached and only attach them every 10 minutes, say, to commit the changes
What kind of strategy should I use to get decent performance given that there could be many updates, inserts, deletes and queries per second from possibly hundreds of clients all at once, and they all have to be consistent with each other?
Another smaller question: how should I use transactions in an efficient manner? Is it fine for every single change to be in its own transaction, or is that going to perform badly when I have hundreds of clients all trying to alter cells in the grid? Should I try to figure out how to bulk together similar updates and place them within a single transaction, or is that going to be too complicated? Do I even need transactions for most of it?

I would use a session per request to the server, and one transaction per session. I wouldn't optimize for performance before the app is mature.
Answer to your solutions:
Have a single session that's always opened that all clients use: You will have performance issues here because the session is not thread safe and you will have to lock all calls to the session.
Have a single session for each client that connects and periodically flush it: You will have performance issues here because all data used by the client will be cached. You will also see problems with stale data from the cache.
Open a session every time I have to use any of the persisted entities and close it as soon as the update, insert, delete or query is complete: You won't have any performance problems here. A disadvantage are possible concurrency or corrupt data problems because related sql statements are not executed in the same transaction.
Have a session for each client, but keep it disconnected and only reconnect it when I need to use it: NHibernate already has build-in connection management and that is already very optimized.
Same as above, but keep it connected and only disconnect it after a certain period of inactivity: Will cause problems because the amount of sql connections is limited and will also limit the amount of users of your application.
Keep the entities detached and only attach them every 10 minutes, say, to commit the changes: Will cause problems because of stale data in the detached entities. You will have to track changes yourself, which makes you end up with a piece of code that looks like the session itself.
It would be useless to go into more detail now, because I would just repeat the manuals/tutorials/book. When you use a session per request, you probably won't have problems in 99% of the application you describe (and maybe not at all). Session is a lightweight not threadsafe class, that to live a very short. When you want to know exactly how the session/connection/caching/transaction management works, I recommend to read a manual first, and than ask some more detailed questions about the unclear subjects.

Read the 'ISessionFactory' on this page of NHibernate documentation. ISessions are meant to be single-threaded (i.e., not thread-safe) which probably means that you shouldn't be sharing it across users. ISessionFactory should be created once by your application and ISessions should be created for each unit of work. Remember that creating an ISessions does not necessarily result in opening a database connection. That depends on how your SessionFactory's connection pooling strategy is configured.
You may also want to look at Hibernate's Documentation on Session and Transaction.

I would aim to keep everything in memory, and either journal changes or take periodic offline snapshots.

Have a read through NHibernate Best Practices with ASP.NET, there are some very good tips in here for a start. As mentioned already be very careful with an ISession as it is NOT threadsafe, so just keep that in mind.
If you require something a little more complex then take a look into the NHibernate.Burrow contrib project. It states something like "the real power Burrow provides is that a Burrow conversation can span over multiple http requests".