I need to get changes from several tabases into my 1 database, so I guess sql notifier will be best way.
I found interesting application: http://www.codeproject.com/Articles/144344/Query-Notification-using-SqlDependency-and-SqlCach
that works, but only in one database.
Im not asking for full-solution, maybe just some of you work with project like that, and know how can I use sqlnotifier to read data from different databases?
Depending on what information you want to track, the SQL Server 2008 developer certification suggests using either CDC (change data capture) or CT (change tracking).
Look here for the details.
You may also consider using the auditing feature built-in.
As an alternative you might want to investigate Microsoft Sync Framework to implement what you want, since it looks like you really need a sync solution.
Using this you could sync between two or multiple sql server databases. And since you can define a sync scope, you could restrict it to just the table that you're interested in.
Here are some links where you can read more about this:
MSDN - How to Choose a Data Synchronization Technology
MSDN - Microsoft Sync Framework
Wikipedia - Microsoft Sync Framework
Related
I have been working on an application for a couple of years that I updated using a back-end database. The whole key is that everything is cached on the client, so that it never requires an network connection to operate, but when it does have a connection it will always pickup the latest updates. Every application updated is shipped with the latest version of the database and I wanted it to download only the minimum amount of data when the database has been updated.
I currently use a table with a timestamp to check for updates. It looks something like this.
ID - Name - Description- Severity - LastUpdated
0 - test.exe - KnownVirus - Critical - 2009-09-11 13:38
1 - test2.exe - Firewall - None - 2009-09-12 14:38
This approach was fine for what I previously needed, but I am looking to expand more function of the application to use this type of dynamic approach. All the data is currently stored as XML, but I do not want to store complete XML files in the database and only transmit changed data.
So how would you go about allowing a fairly simple approach to storing dynamic content (text/xml/json/xaml) in a database, and have the client only download new updates? I was thinking of having logic that can handle XML inserted directly
ID - Data - Revision
15 - XXX - 15
XXX would be something like <Content><File>Test.dll<File/><Description>New DLL to load.</Description></Content> and would be inserted into the cache, but this would obviously be complicated as I would need to load them in sequence.
Another approach that has been mentioned was to base it on something similar to Source Control, storing the version in the root of the file and calculating the delta to figure out the minimal amount of data that need to be sent to the client.
Anyone got any suggestions on how to approach this with no risk for data corruption? I would also to expand with features that allows me to revert possibly bad revisions, and replace them with new working ones.
It really depends on the tools you are using and the architecture you already have. Is there already a server with some logic and a data access layer?
Dynamic approaches might get complicated, slow and limit the number of solutions. Why do you need a dynamic structure? Would it be feasible to just add data by using a name-value pair approach in a relational database? Static and uniform data structures are much easier to handle.
Before going into detail, you should consider the different scenarios.
Items can be added
Items can be changed
Items can be removed (I assume)
Adding is not a big problem. The client needs to remember the last revision number it got from the server and you write a query which get everything since there.
Changing is basically the same. You should care about identification of the items. You need an unchangeable surrogate key, as it seems to be the ID you already have. (Guids may be useful here.)
Removing is tricky. You need to either flag items as deleted instead of actually removing them, or have a list of removed IDs with the revision number when they had been removed.
Storing the data in the client: Consider using a relational database like SQLite in the client. (It doesn't need installation, it is just storing in a file. Firefox for instance stores quite a lot in SQLite databases.) When using the same in the server, you can probably reuse some code. It is also transaction based, which helps to keep it consistent (rollback in case of error during synchronization).
XML - if you really need it - can be stored just as a string in the database.
When using an abstraction layer or ORM that supports SQLite (eg. NHibernate), you may also reuse some code even when there is another database used by the server. Note that the learning curve for such an ORM might be rather steep. If you don't know anything like this, it could be too much.
You don't need to force reuse of code in the client and server.
Synchronization itself shouldn't be very complicated. You have a revision number in the client and a last revision in the server. You get all new / changed and deleted items since then in the client and apply it to the local store. Update the local revision number. Commit. Done.
I would never update only a part of a revision, because then you can't really know what changed since the last synchronization. Because you do differential updates, it is essential to have a well defined state of the client.
I would go with a solution using Sync Framework.
Quote from Microsoft:
Microsoft Sync Framework is a comprehensive synchronization platform enabling collaboration and offline for applications, services and devices. Developers can build synchronization ecosystems that integrate any application, any data from any store using any protocol over any network. Sync Framework features technologies and tools that enable roaming, sharing, and taking data offline.
A key aspect of Sync Framework is the ability to create custom providers. Providers enable any data sources to participate in the Sync Framework synchronization process, allowing peer-to-peer synchronization to occur.
I have just built an application pretty much exactly as you described. I built it on top of the Microsoft Sync Framework that DjSol mentioned.
I use a C# front end application with a SqlCe database, and a SQL 2005 Server at the other end.
The following articles were extremely useful for me:
Tutorial: Synchronizing SQL Server and SQL Server Compact
Walkthrough: Creating a Sync service
Step by step N-tier configuration of Sync services for ADO.NET 2.0
How to Sync schema changed database using sync framework?
You don't say what your back-end database is, but if it's SQL Server you can use SqlCE (SQL Server Compact Edition) as the client DB and then use RDA merge replication to update the client DB as desired. This will handle all your requirements for sure; there is no need to reinvent the wheel for such a common requirement.
What I want is to be able to use Microsoft Sync Framework to sync two SQL 2008 databases that both use the SQL 2008 change tracking feature. One will be a server and the other will be a local SQLExpress instance, using a hub & spoke model.
There are some hints that others have accomplished this - namely Syncing SQL Server 2008 Databases over HTTP using WCF & Sync Framework. However, that solution requires "SqlExpressClientSyncProvider available from MSF's codeplex project site as Client Sync Provider". This same SqlExpressClientSyncProvider is hinted at elsewhere too. However, it either no longer exists or is hiding really well from my attempts to find it.
Most of the links and suggestions I've come across in my searching end up pointing to the MSDN samples site which has several examples (such as this one http://code.msdn.microsoft.com/Database-SyncSQL-Server-a9752fac) which all demonstrate using the sync framework via custom change auditing - which adds a lot of visible overhead to the schema such as tombstone tables, triggers, etc. This is what I want to avoid.
I'd be happy with a link to the bits for the SqlExpressClientSyncProvider, however, I suspect that it is built for the MSF 1.0 way of doing things. If there is a "MSF 2.1" way to do this via the SqlProvider classes or something along those lines, that would be great.
SQL Server Change tracking is only supported in the older offline provider (SqlCeClientSyncProvider/DbServerSyncProvider) which is the same providers used by the Local Database Cache project item in Visual Studio.
the newer SqlSyncProvider/SqlCeSyncProvider uses a custom tracking mechanism and it will take a great amount of customization to get it to work with Sql Change Tracking short of writing your own custom provider.
the SqlExpressClientSyncProvider is a sample provider that has since been taken out by MS from the download site.
this link still has some bits of it though: http://www.8bit.rs/blog/2009/05/debugging-sql-express-client-sync-provider/
I have the pleasure of using SyncFx v.2.1 on an application. The client side presently uses SQLCE and the Server side uses Server 2008 r2. I am using a SyncFx proxy and host the server SyncAdapterBuilder code in the WCF service. The client has the SyncAgent and SyncTables and it works fine. I am using the integrated SQL Change Tracking in lieu of the coupled (aka custom / scoped) change tracking because I am not permitted to modify existing schema.
So my issue is that the requirements for the system have changed and I am required to use SQL Express on the client in order to support stored procedures.
Why not merge replication? The requirements also prohibit modification of the schema or the use of triggers. In fact the original version of the app used merge replication with SQLCE before moving to SyncFx for SQLCE.
So how is this done? I've read a lot of conflicting information and I can only assume that this is in response to the ever evolving versions of SyncFx. There are no direct example of how SQL to SQL Express with Change Tracking on both is accomplished. Plus I am trying to transition from a functional SQLCE implementation to Express with as few changes as possible. The client is already capable of using either type of DB, it is just the current sync process that needs to change.
Here is what I've found, but have not had success. I've read every StackOverflow response on the matter and am still not finding a way to do this that actually works.
Database Sync:SQL Server and SQL Express N-Tier with WCF : This MS example works fine with the SyncOrchistrator but provisions side tracking tables and triggers. I was not able to modify this in such a way that change tracking could be used on the client and server.
Sync framework with SQL Server 2008 Change Tracking : StephaneT suggests here that simply by using the normal SQLCE approach with the SQL Express sample sync provider and SyncFx 2.0 techniques only client side table modification would be required. Unfortunately all links to this sample SQL Express provider seem to be removed and other posts from JuneT and even Liam Cavanagh on MSDN suggest moving forward with the new official SqlServerProvider instead of a customized version of the DbServerProvider. Problem is there are no sample implementations of this anywhere and I haven't been able to figure it out through trial and error.
Syncing SQL Server 2008 Databases over HTTP using WCF & Sync Framework : Raj gives the best example (simple and easily translated to SQLCE processes) unfortunately it also uses the SqlExpressClientSyncProvider that seems to have evaporated from the internet. It also requires an anchor table to track the clients, I think I can get away with that as I am not allowed to modify schema on "existing" tables.
So any examples out there that can help me. Essentially I want to port the existing functioning SQLCE SyncFx via proxy with integrated SQL Change Tracking using a SyncAgent to a version that works for SQL Express without changing existing scheama or using triggers. I should also mention that I use filter parameters heavily as there are 150+ tables in the replication and they would be extremely large without filters. I had read some references that said the SqlExpressClientSyncProvider didn't support filters, but this is impossible for me to verify since I can't find a reference to that code that is still good.
Maybe there is a refresh of Raj's example that uses SqlServerSyncProvider
Thanks in advance to anyone that can point me in the right direction!
check out this link and you might still find some of the download links in the comments area working: http://www.8bit.rs/blog/2009/05/debugging-sql-express-client-sync-provider/
take note that even the sample SqlExpressClientSyncProvider uses triggers to track the deletes in the tombstone tables. likewise, you need to have columns in your table to track when a row has been inserted or updated (datetime or timestamp columns).
with regards to filtering, you can easily modify the queries in the adapter to include a filter clause.
the newer SqlSyncProvider does not support Sql Change Tracking as it implements its own tracking mechanism. the newer providers works in a peer-to-peer scenario so its tracking as well from which replica a particular change has come from.
I have 1 application , in which conditions are like i have to use local databases only for each PC....Now if some enrollment is done from 1 pc then that data should be store in local database and also it should be send to another PC...in short i want to synchronise all the data..
I need to clear 1 thing that "Centralize database is not possible..I cant use 1 database and connect to it from all PC.."So i need synchronization only...
I am using SQL SERVER Express Edition...
and developing application in C# .NET
If have any doubt you can ask me i will describe more...
Since you're using SQL Server Express, replication is not an option. (Express versions can only subscribe in a replication scenario.) But you should take a look at the Sync Framework, formerly known as Sync Services for ADO.NET. It is an API for .NET that provides the kind of db sync capabilities which you may find helpful. From your description, the Collaboration Scenario seems the most applicable (peer-to-peer synchronization scenario.)
Just a thought, if you really want data redundancy, looking at something like cassandra might be a better alternative.
Im developing a windows forms application using C# 4.0 and that application is going to target different database engines like SQL, MySQL and Oracle i was wondering if there is a library that can talk to all the three engines instead of implementing my own layers for every one.
thanks in advance.
You could use an ORM tool; I like NHibernate But there are many more: see a list at wikipedia.
The problem is if you want to do anything remotely advanced (date arithmatic, generate primary keys, get the id of the last inserted record, pivot a table , use RANGE construct etc.) then both databases use completely different syntax.
The best solution (in the java world at least is either Ibatis or Hibernate) I know there is a .NET version of Hibernate I am not sure about Ibatis.
These libraries insulate your program from the various SQL dialects and provide a common API independent of the underlying database.
If you use the classes in System.Data.Common you can make your code database independent:
Writing Provider Independent Code in ADO.NET
I don't know C#, but I know it will have a library for ODBC.
It looks like MS has one here.
It's old, but actually it does the job just fine. Virtually every DB in existence provides an ODBC driver.
Checkout DbLinq.
DbLinq is THE LINQ provider that allows to use common databases with
an API close to Linq to SQL. It currently supports (by order of
appearance): MySQL, Oracle, PostgreSQL, SQLite, Ingres, Firebird...
And still SQL Server.