I am creating a simple project management system which uses NHibernate for object storage. The underlying database is SQL express (at least currently for development).
The client runs on either the desktop or laptop. I know I could use web-services and store the DB only on the desktop, but this would force the desktop to be available all the time.
I am currently thinking about duplicating the DB, having two instances with "different data". To clarify, we are not talking about a productive app here, its a prototype.
One way to achieve this very simple would be the following process:
Client: Check if desktop DB is available (through web service)
Client: If yes, use desktop storage, no problem here
Client: If not, use own DB as storage
Client: Poll desktop regulary, as soon as it comes on, sync
Client: Switch to desktop storage
...
Desktop: Do not attempt any DB operation before checking for required sync
Desktop: If sync needed, do it...
My question is now, how would you sync? Assume 4 or 5 types of objects, all have GUID as identifiers. Would you always manually "lazy load" all objects of a certain type and feed them to the DB. Would you always drop the whole desktop DB in case the client DB may be newer and out of sync?
Again, I want to stress out, I am not assuming any conflicts or stale data, I basically just want to "copy the whole DB from the client". Would you use NHibernate for this? Or would you separate the copy process?
When I think about it, my questions comes down to this:
Is there any function from NHibernate:
SyncDBs_SourceWins_(SourceDB, TargetDB)
Thanks for help,
Chris
Not NHibernate, but how about Sync Services for ADO.NET 2.0?
-sa
Related
I am working on a Windows 8.1 app which should work offline and online and it should connect to a database in both of the situation . So which kind of service or database I should implement to allow access to local and remote database and sync when it goes online.
There's not much tooling available to help you in your scenario. You can probably get the most out of Microsoft Sync framework, but you need to bear in mind that there's no official support for WinRT from Microsoft, nor is the framework further developed in its current form.
I suggest you take a look at SyncWinRT which is an open source library that can be used together with Microsoft Sync framework to make it work on Windows Phone and Windows Store apps. It's not all that easy to setup (neither is Sync framework itself) and there are some limitations, but depending on your requirements it might work for you.
Taking a broader look at your situation, there are 3 approaches which you can take:
Have your application work directly with the remote database when online and with local database when offline, but make it work the same in both scenarios otherwise: this is the most tricky to do, because you need to be able to switch between the two DALs (data access layers) based on the mode and also be able to sync your local database with the remote one when you get back online. For switching the DALs you might want to take a look at CSLA.NET which in general has such capabilities, but its learning curve is quite high. For synchronizing the two databases the above mentioned SyncWinRT should work for you.
Have your application always work with the local database copy which you synchronize with the remote one on user request: in this case you would have the same data structure locally as it is remotely, but your local business logic would always use the local database. This way you only need one DAL and the user can synchronize the data when he is online. Again for the synchronization you could use SyncWinRT, the rest will just be a standard app with local SQLite database which you can develop as you see fit.
Have custom local data structure and synchronization process: completely customize the mobile experience by giving control of data synchronization to the user. He should be aware what data he is synchronizing when preparring for the offline work and synchronize that data back when he is done. The local data structure can be tailored for this and the synchronization will have to be done completely manually, but you can make it more simple if you control the actual use cases.
In my experience the last option usually works best both for users and for the developers. Although it seems to be more work, you can avoid a lot of conflicting scenarios by having closer control over the user options. And even the users might like the simpler wizard like experience better than a copy of their original online one.
I am developing a windows application for agricultural purpose. This application will be used by multiple users to maintain the data. The main issue is there won't be network connectivity on the work location. But however by end of the day they can go and synchronize if there are any option.
I just want to know how can we import and store all the data locally and update the data to database when there is network.
The options that i thought is to have SQL on every machine that runs this application. Store the data to local database when there is no network.
Having a separate button to export the local data to the centralized database when there is network.
Looks like this is complicated. Is there any better and easier option.
I prefer using c#, Visual studio.
Thanks.
You can use SQLite for storing data locally. It's fast, lightweight, and public domain.
You can use whatever the database of choice for the centralized server.
Well, this a quite broad question, as it has many options and scenarios. The questions you should ask yourself are:
Does user handle new information only or any information from any other user from the previous syncing?
Do you have to handle update conflicts?
Do you handle text information only or you have complex types and binary files?
As for the solution, the easiest way, from my point of view, would be using SQL Lite on portable devices, is a lightweight SQL client that will allow you to handle information easily. On the server you can use whatever you want, SQL Server, MySQL or any other SQL flavor you may like. Just make sure there is a connector for your portable device OS.
If you keep thinking of using SQL server on the portable device, it's a battery hogger!!!, you might want to check Microsoft Sync framework, as it provides almost all possible scenarios for handling data syncing, manage conflicts, etc.
Thanks for the answers. Please find the below solution that we implemented.
1) Installed SQL express on all the local machines
2) Used Microsoft Sync framework to sync the data. The sync is configured on demand.
Issues faced:
1) We were using geometry datatype on few tables and this was not supported by sync framework.
2) Any change in the database schema will not reflect on the client machine. We will have to delete all the system generated procedures used to track the table change and regenerate it. I am sure there will be a much better way to do this.
Cheers,
Jebli
I have been working on an application for a couple of years that I updated using a back-end database. The whole key is that everything is cached on the client, so that it never requires an network connection to operate, but when it does have a connection it will always pickup the latest updates. Every application updated is shipped with the latest version of the database and I wanted it to download only the minimum amount of data when the database has been updated.
I currently use a table with a timestamp to check for updates. It looks something like this.
ID - Name - Description- Severity - LastUpdated
0 - test.exe - KnownVirus - Critical - 2009-09-11 13:38
1 - test2.exe - Firewall - None - 2009-09-12 14:38
This approach was fine for what I previously needed, but I am looking to expand more function of the application to use this type of dynamic approach. All the data is currently stored as XML, but I do not want to store complete XML files in the database and only transmit changed data.
So how would you go about allowing a fairly simple approach to storing dynamic content (text/xml/json/xaml) in a database, and have the client only download new updates? I was thinking of having logic that can handle XML inserted directly
ID - Data - Revision
15 - XXX - 15
XXX would be something like <Content><File>Test.dll<File/><Description>New DLL to load.</Description></Content> and would be inserted into the cache, but this would obviously be complicated as I would need to load them in sequence.
Another approach that has been mentioned was to base it on something similar to Source Control, storing the version in the root of the file and calculating the delta to figure out the minimal amount of data that need to be sent to the client.
Anyone got any suggestions on how to approach this with no risk for data corruption? I would also to expand with features that allows me to revert possibly bad revisions, and replace them with new working ones.
It really depends on the tools you are using and the architecture you already have. Is there already a server with some logic and a data access layer?
Dynamic approaches might get complicated, slow and limit the number of solutions. Why do you need a dynamic structure? Would it be feasible to just add data by using a name-value pair approach in a relational database? Static and uniform data structures are much easier to handle.
Before going into detail, you should consider the different scenarios.
Items can be added
Items can be changed
Items can be removed (I assume)
Adding is not a big problem. The client needs to remember the last revision number it got from the server and you write a query which get everything since there.
Changing is basically the same. You should care about identification of the items. You need an unchangeable surrogate key, as it seems to be the ID you already have. (Guids may be useful here.)
Removing is tricky. You need to either flag items as deleted instead of actually removing them, or have a list of removed IDs with the revision number when they had been removed.
Storing the data in the client: Consider using a relational database like SQLite in the client. (It doesn't need installation, it is just storing in a file. Firefox for instance stores quite a lot in SQLite databases.) When using the same in the server, you can probably reuse some code. It is also transaction based, which helps to keep it consistent (rollback in case of error during synchronization).
XML - if you really need it - can be stored just as a string in the database.
When using an abstraction layer or ORM that supports SQLite (eg. NHibernate), you may also reuse some code even when there is another database used by the server. Note that the learning curve for such an ORM might be rather steep. If you don't know anything like this, it could be too much.
You don't need to force reuse of code in the client and server.
Synchronization itself shouldn't be very complicated. You have a revision number in the client and a last revision in the server. You get all new / changed and deleted items since then in the client and apply it to the local store. Update the local revision number. Commit. Done.
I would never update only a part of a revision, because then you can't really know what changed since the last synchronization. Because you do differential updates, it is essential to have a well defined state of the client.
I would go with a solution using Sync Framework.
Quote from Microsoft:
Microsoft Sync Framework is a comprehensive synchronization platform enabling collaboration and offline for applications, services and devices. Developers can build synchronization ecosystems that integrate any application, any data from any store using any protocol over any network. Sync Framework features technologies and tools that enable roaming, sharing, and taking data offline.
A key aspect of Sync Framework is the ability to create custom providers. Providers enable any data sources to participate in the Sync Framework synchronization process, allowing peer-to-peer synchronization to occur.
I have just built an application pretty much exactly as you described. I built it on top of the Microsoft Sync Framework that DjSol mentioned.
I use a C# front end application with a SqlCe database, and a SQL 2005 Server at the other end.
The following articles were extremely useful for me:
Tutorial: Synchronizing SQL Server and SQL Server Compact
Walkthrough: Creating a Sync service
Step by step N-tier configuration of Sync services for ADO.NET 2.0
How to Sync schema changed database using sync framework?
You don't say what your back-end database is, but if it's SQL Server you can use SqlCE (SQL Server Compact Edition) as the client DB and then use RDA merge replication to update the client DB as desired. This will handle all your requirements for sure; there is no need to reinvent the wheel for such a common requirement.
I'm building a WPF, C#, .NET solution that needs to rapidly change data connections.
All the connections will eventually sync back to a parent Oracle database, but in the meantime, I might be pulling data from an access database, or a local sql server compact database, or an xml file, or even a web service, or sharepoint. Problem is, I might even need to add providers, and need to be able to keep the providers in sync with each other, and do it real time, with no loss of connection/seamless to the users. This is all dependent on what type of machine, which domain, and what kind of network connectivity we have, and is a client requirement, not something I can change.
Does anyone have a good recommendation for what the best way to accomplish this would be?
Has the client explained why they need this functionality? Often they ask for things to solve a problem that they forsee, without adequate knowledge of how best to solve it.
If you're coding something like an application for a travelling salesman to use, which will have intermittent connectivity to the Oracle database, then maybe you should look at using some means other than a direct database connection for synchronising the databases.
Say using a WCF/SOAP service to pass serialized data objects back and forth or you could look at using MSMQ to transfer changes back and forth between the intermittently connected mobile application and the Oracle database server. It would, of course mean that you'll need to run a server side application/service to handle this data and pass it into the Oracle database, but it would allow for intermittent connections to be handled more easily without having to handle database connection error logic.
In the meantime if your client code should look at layering the code to use a factory Repository type pattern. As business logic just calls an interface it is then possible to use database specific code within your data layer that was decided upon at run time (say through a config setting).
you can create one or two tables in any of your server whcih will include all connection strings and the condtions on which data you need to use which connection .
and your business layer will be independent of connections .
other options is to use sofware facotry pattern.
there you can repository independent of connections and on run time decided which data repository will connect to whcih DB.
I will be building an in-house, Occasionally Connected App (OCA). What technologies would you suggest I employ.
Here are my parameters:
.NET Shop(3.5sp1)
C# for code behind (winform,wpf,silverlight)
SQL Server Backend (2005 or possibly 2008 pending approval)
Solo Developer
Solo SQL Administrator
Low Tech end users
Low bandwidth to 5 Branch offices
This is a LOB app but not a POS.
Majority of users have laptops that they take to Member's Home
The Data for this App is stored in 5 separate Databases, though in one SQL instance.
I am looking for specific recommendations on which path to choose. Merge Replication or Sync Framework database synchronization providers? SQL Express or SQL CE at the Subscriber? Can I use LINQ to SQL for the DAL?
Is a Silverlight 'Offline/Out of Browser App' Example Here, feasible?
This is my first LARGE business application so any experienced comments are welcome.
As requested here is some additional info on the type of Data. My users are Nurses and Social Workers who go to Member's homes and create "Plans" or "Health Assessment Reviews" for them. These are things like a Medication List or a List of there current "Providers". Steps to achieve members' goals or a list of there current/past Diagnosis's. Things like that.
Also the typical Members Name, Address, Phone Number, etc. Mostly this is a Data Storage and Retrieval app that facilitates reporting. Very little "processing" takes place and Nurses and Social Workers work in teams that are assigned members so I usually have very little crossover or potential data conflicts. Nurses and SW's also are responsible for different area's of the MCP(Member Centered Plan)
Additional question; Is Sync Framework really only a viable option if I can use SQL 2008? Seems that way due to the Change Tracking etc....thoughts?
Once you solve the problem of change detection and data movement, everything else is trivial. In other words technologies like WPF, Silverlight, Forms and even WCF are orthogonal to your main problem and your choice should be based on your personal preferences and experience. The real hard nut to crack is working disconnected and synchronizing changes. Which leaves two out-of-the-box avenues: Synch Framework or Replication.
I would say, for your scenario, definetely Synch Framework. Merge replication, like all forms of replication, is designed for systems that are connected continously with intermitent disconnects. And most critically replication can work only over static names. Laptops connecting from various hot-spots and ISPs have a nasty habit of changing FQ names with each connection. Replication can overcome this only if a VPN of sort is used and VPN is usually a major support issue. Replication is just not designed for the high mobility of OCA systems.
Synch Framework will pretty much force you to SQL 2008 back end because of the need to Change Data Capture or Change Tracking, both being SQL 2008 only features.
You will still have plenty of hard problems to solve ahead (authentication, versioning and upgrade, data conflict resolution policies, securing data on the client for accidental media loss etc etc)
Personally, I would say:
.NET 3.5
WCF Data Services (for communication between the client app and your data)
SQL Server 2k5/2k8 (whichever you can use)
Silverlight w/ Out of Browser Functionality
VistaDB (to store data locally on the client until you can push to the server)
use unique-identifier for key if you are creating stuff while offline and not connected and when you do connect, updating the database.
this is going to be way easier than using auto-increment key
Having worked on an occasionally connected application, I'd encourage you to look in to SQL Server CE for the client machines, with Sync Services to handle the connections. Here is a good tutorial.
You could create this stuff from the ground up, it seems.
However, this seems an awful lot like a CRM application, and it wouldn't surprise me if you could find an enterprise software package to do this without starting from scratch and instead modify one of the configurations to meet your business rules.
In a previous life, I was a configuration developer for this thing called Siebel that might be close to what your'e looking for. They even have a built-in synchronization tool called Siebel Remote.
It might be a cheaper route to go than rolling your own from scratch.
I wrote an order taking program for wine sales reps. Here is the video. The client software is installed using click-once. That also installs SQL Server Express and loads the database. I used the Microsoft Sync Framework to sync the local database with the one on the server (see the last section of the video.)
With powerful clients now I don't see any reason to not use SQL Server Express, it is free with a limit of 4GB.
SQL CE had too many limitations - no stored procs being a major one.
You will need to use GUIDs everywhere as the primary key - see the new NewSequentialID().
I love click-once, it is a big time saver.
I'm looking forward to Silverlight, but just haven't had time to look into it. Not sure if I would have done it with Silverlight if doing it now or not.
Having said all this, this is not a project for anyone inexperienced. So I would also get some very experienced help.