We Keep Coding

How Does One Fill a Typed DataSet, Keep it Synchronized, and Receive Updates When the Data Changes?

So I'm developing an application that works as sort of a "sidekick" to a large proprietary application which I do not have the source code for nor the rights to modify. The proprietary application does store all of its data in a Microsoft SQL database (version 2008 R2 or higher, I believe), however, and I have a good idea what the data represents. What I need my application to do is to constantly monitor the data as it is being added, updated, and deleted, and then act on the data automatically (such as raising alerts).
The issue is figuring out the best approach to receiving changes made to the database by the other application as they're happening, because I don't wanna miss a beat.
Here is what I have done so far:
LINQ to SQL: As far as I know, each time I run a query, I receive a new set of data, but I do not get the ability to receive the changes only or be notified of changes.
Typed DataSet using DataSet.Load:
using (IDataReader reader = dataSetInstance.CreateDataReader())
{
dataSetInstance.Load(reader, LoadOption.OverwriteChanges, dataSetInstance.Table1, dataSetInstance.Table2, dataSetInstance.Table3);
}
This didn't work out too well when I did it. dataSetInstance only contained a set of unfilled tables after calling the Load method. I was hoping to call dataSetInstance.GetChanges and dataSetInstance.AcceptChanges at regular intervals after the first call to dataSetInstance.Load to get only the changes. Am I doing it wrong?
Typed DataSet with tables filled individually using their associated table adapters:
using (Table1TableAdapter adapter = new Table1TableAdapter())
{
adapter.Fill(dataSetInstance.Table1);
}
using (Table2TableAdapter adapter = new Table2TableAdapter())
{
adapter.Fill(dataSetInstance.Table2);
}
using (Table3TableAdapter adapter = new Table3TableAdapter())
{
adapter.Fill(dataSetInstance.Table3);
}
Of course, the problem is that there are actually way more than 3 tables which can add up to quite a lot of repetitive code (and maintenance work), but the real problem is that I will not receive any change notifications since I'm not using the Load/AcceptChanges methods (according to the documentation).
Row retrieval by date/time field: This was something I started work on, but something I stopped after observing the other application modify fields in the rows after creating them. Consider this:
There is a row with a time stamp of a transaction and a boolean field that specifies if the transaction was canceled later on. If it is canceled, the other application simply goes back to that row and toggles the value. The time stamp remains the same, and my application will never know of the news. There is no statute of limitations; the other application can change this field any time in the future.
By the way, I should mention that this other application does not implement any constraints within the database such as foreign and primary keys. I believe I read somewhere in the documentation that for row update events and such to fire on the typed DataTable classes, some sort of primary key is needed.
There must be some way to do this!!!

Have you considered SQL Server Query Notifications? This uses SQL Server Service Broker under the covers.
SqlDependency is the C# class to look at.
Using SqlDependency in a Windows Application (.NET Framework 2.0 example: should be very similar to later versions.)
SqlDependency in an ASP.NET Application

I’d consider solving this at SQL Server level by implementing auditing triggers or SQL Server traces.
Triggers – idea is to add triggers to all tables you want to monitor. Triggers will catch all changes and store the data in some other “history” table. Once this is setup all your application needs to do is to read from these tables.
Check this for more details Creating audit triggers in SQL Server
Traces – you can setup SQL Server traces that will store all info in trace files and then your app can parse trace files and see what’s going on.

There appears to be no silver bullet to the problem given the conditions, but anything is better than polling the database for changes every minute. What I will probably do now is take Mitch Wheat's suggestion and work from there:
Some tables have rows that are highly likely to change. A recent purchase, for example, is more likely to be cancelled than one from 7 days ago, or 6 months ago, or in the case of 1 year—probably never. The application will only need to monitor queries restricted to a certain time range. Older (in terms of creation time) rows will simply be refreshed at a much slower rate and without prompting from SQL Server query notifications. The application is going to have to tolerate some stale data in order to not needlessly pull entire tables from the database every minute.
For tables without chronological information, the application will have to receive notifications for queries on conditions that are important or have to be acted on right away such as WHERE Quantity < 0.
Some more clever approaches will need to be taken for the rest of the tables. Some tables are never updated nor their rows deleted, but they will gain new rows whenever some other table's rows changes. For example: every time the NumberOfPeople value changes for a row in table Room, another row is added to one of the tables CheckIn or CheckOut.
A lot more code needs to be written, but the application is probably going to be doing a lot less unnecessary work when it's running.

C#/SQL changing logging - best methods

Not sure if this question is suitable for StackOverflow as it's much more 'general'. Basically, I have a database driven business application made in ASP.NET and C#, which'll be used by around 20 members of a company. A crucial aspect of this is auditing - I need to log on any changes to any of the tables, and have them viewable by senior members of the staff.
My current solution uses SQL triggers, but I need to create something much more robust and user friendly. The database is gigantic, with a lot of tables with relations etc, and the audits currently are very uninformative to the users - telling the staff that x user modified an order to have a customer of ID of 837 is near enough useless - I need to be able to dictate which field is displayed in the audit log.
My idea is to create a class in my code that'll handle all these, and somehow map out what fields to display to the user, and also somehow tell the code which table was modified and which record.
Can anyone offer any general advice on how to do what I want, and whether it's actually possibile? I'm a heavy user of LINQ-to-SQL in my code, so I'm hoping that'll help...

You could also try using DoddleAudit for your needs. It provides automatic auditing of all inserts/updates/deletes for any table in your database with a single line of code, including:
What table was modified?
What fields changed?
Who made the change?
When did it occur?
You can find it here: http://doddleaudit.codeplex.com/

I've had similar audit requirements for a healthcare application, which used linq-to-sql for data access.
One way to do it centrally in Linq-to-sql is to override SubmitChanges in the data context class. Before submitting the changes, call GetChangeSet() to get data about the pending changes. Then add change tracking information as appropriate to a relevant log table before calling base.SubmitChanges(). In my application I used an xml column to be able to store change data for different tables in a structured manner, without having to create special history tables for each table in the system.

You could also try using SQL Server 2008's Change Data Capture feature. It basically captures inserts, updates and deletes on the desired tables, and stores changes made into a separate set of relational tables.
http://www.mssqltips.com/sqlservertip/1474/using-change-data-capture-cdc-in-sql-server-2008/

Entity Framework savechanges - can't see changes in other clients

I'm using EF 4.0 with VS2010. I have 2 clients running my applicaion.
When I save the changes in one client, I see them in the SQL server, but the 2nd client doesn't see them.
I need to restart the application to see the changes.
I'm using a Data layer for all the DB stuff, I leave my connection open all the time (as suggest in some post I read) might it be the problem??? any workaround I can't write the DL from scratch again.
10x

By default if an entity is loaded to the context that instance is returned when you query the database for a set of entities which will include the above entity.
You need to set the MergeOption to OverwriteChanges to get the changes in the database.
context.Products.MergeOption = MergeOption.OverwriteChanges;
var products = context.Products.Where(/**/);
Its better to create short lived to contexts to avoid such problems.

EntityFramwork isn't updating data when you change it on other connection. To get new state you have to recreate Context and load all data again.

C# Backing up data to file which can be restored later

I'm looking for some feedback as to what is the best way to back data up from a sql server database which can be restored at a later date. This back up needs to be in a file which the user can use to restore data at a later date.
Does anyone have any ideas or examplea as to the best way to do this using C# and Sql Server?
EDIT:
This shouldn't back the whole database, just a set of data specified by the user using dates.
Thanks

If I understand properly, you want to audit just some of your data for changes (as opposed to having a full DB backup) and to be able to selectively roll them back to a previous state.
If that's what you want to do, you may do that from within sql by using a "audit" table and populating it via triggers on the table where your original data is stored (i.e. each time a row is inserted/updated/deleted the trigger will write on the "audit" table what the previous value was and when it was changed).
See here for an example.

Try this bit of Code. i have used this before.
http://www.daniweb.com/forums/thread202843.html

Best way to track changes and make changes from Mysql -> MSSQL

So I need to track changes that happen on a Mysql table. I was thinking of using triggers to log all the changes made to it and then save these changes in another table. Then I will have a cron script get all these changes and propagate the changes into the Mssql database.
I really dont expect a lot of information to be proporgated, but the data is very time sensitive. Ideally the MSSQL will see these changes within a minute, but I know that this requirement may be too high.
I was wondering if anyone had a better solution.
I have the bulk of the site written in .net but use vbulletin as the forums (sorry but there are no .net forums as powerful or feature rich like vbulletin)

The majority of the replicator tools use this technique. Fill another table on insert/update/delete triggers that containt the tablename and the PK or a unique key.
Then a reader reads this table, do the proper "select" if insert/update to get the data, then updates the other database.
HTH