Hy guys,
to monitor a specified table on db I'm polling this one every few milliseconds (10-20 ms).
Is there any chance to have a notification (avoiding SqlDependency, that in my scenario is too slow) instead polling?
Do you have any idea?
My scenario is .net + Sql Server 2008
Thanks!
It can be done, using a CLR stored procedure that will call a WCF/or a webservice. It is not something very difficult to do.
This needs practically 2 steps.
The modification of data.
After you modified the data you have to send the data to the clr stored procedure. The easiest way is to write it into one ore more temporary tables.
The clr stored procedure.
The clr store procedure will connect to the db with
"context connection=true"
so that you will have access to the stored procedures that you need. After loading the data you send it to a server (WCF/webservice). In the CLR you just need to add the service references that you need. Also on the server you will have to register some dlls for the server to use:
system.web
smdiagnostics
system.runtime.serialization
system.identitymodel
system.identitymodel.selectors
system.messagng
system.transactions.bridge
system.servicemodel
Everything else is plain .NET code to call a WCF/Web service.
This approach is very fast and very reliable.
To get onChange event notification in SQL, you can use the Query Notification feature which is built on Server Broker.
Alternatives are:
DB trigger
SignalR
Change Data Capture
Change Tracking
You could use the SqlChangeMonitor class, but that wraps SqlDependency in cached data scenarios. Your question is a little vague on why you want to do this, though.
I see you say you need notifications (and SqlDependency is available for that), but maybe you don't need instant notifications, and efficiently reading the changes periodically will do the task. If so, go google Change Data Capture, and Change Tracking.
Can the calling application be changed to say write to a queue instead?
If not I guess a trigger on the database which calls a CLR Stored Procedure? That could fire off any kind of event required.
Related
I am using SignalR to update client-side when data is changing on the backend. The data can be either from SQL Server or NoSQL DB.
What I am doing now is: I have a timer on the backend that constantly is trying to check whether there is a change in the DB and if it detects it then I allow SignalR to update the client-side.
I find this approach not good: the timer's interval value can't be really small e.g. even with 1 second I have a huge load on the DB. Besides, it's not real-time and even 1 second is too big. Additionally, that it's quite difficult to detect a DB change if it's not about adding a new record, but calculating some hash for all the records every 1 second or even less is surely not an option.
I think I used to read about another approach that is based on some event triggered either by the DB or by something else, but I can't remember this.
So I was wondering if somebody could advise me of a better solution.
I'm afraid we need to take different actions for different scenarios on this case.
As you mentioned that your data may come from SQL server and nosql db. So I think the scenario should like:
You write code to update the data in both of the database, if so, you can write the calling signalr code after the updating data code, so that you are no need to using trigger.
Based on the scenario 1 as well, if the database you used support using stored procedure, you may also try to find the possibility of writing stored procedure to call the signalr code.
You may also check if the databases you used have event like "data_onchange_event", this document is related to this scenario. I mean if the data stored in the database updated, your database will send an event, you can then write custom code to capture these event and then call signalr.
If you can't or don't write code to update your databases, then you may only using external trigger to monitor the database, but just like you mentioned in the question, it doesn't support high performance because of huge load on the DB. It's based on the database but not the external trigger.
I'm new to Azure, I want to know about how we can implement Data Export Service in C#.
Currently I am using windows azure for application development and sql azure for database. So there I want to read the data from my database and update that into some another client database based on a time interval.
I want to implement these function as a windows azure worker role. How can I do this?
Please help me...
I'm not sure what you're exactly trying to do but depending on your needs you could use one of the following options:
SQL DAC
This is more an import/export like approach (is this what you mean with "Data Export Service"?)
http://sqldacexamples.codeplex.com/wikipage?title=Import%20Export%20Service%20Client&referringTitle=Documentation
SQL Azure Data Sync
This is a synchronization solution, allowing you to use sync groups, filters, specific tables/columns, ... to synchronize data between multiple databases (both SQL Server and SQL Azure are supported).
http://msdn.microsoft.com/en-us/library/hh456371.aspx
As pointed out by Sandrino, you can use SQL Azure Data Sync. It supports the features you want to implement. Using an existing product is usually much easier than write your own.
If you want to write your own, please distinguish which data has been modified. You can take the same route as Data Sync does: Create triggers in the database. Whenever a data is inserted, updated, deleted, the trigger is invoked, and you insert some metadata in a tracking table. Then your worker role code queries the tracking table to figure out what data needs to be synched.
Best Regards,
Ming Xu.
I've written a blog post about this which uses some straightforward tooling to achieve a simple export and import, using task scheduler than worker-role which makes it pretty easy:
https://iainhunter.wordpress.com/2012/08/21/sql-azure-disaster-recovery/
We have a process that needs to fire when a change occurs to a specific database table in Oracle. At the moment a similar process has been developed using triggers and a bunch of subsequent database actions that occur when that trigger is fired.
However, in this solution we want to call a .NET component (most likely a service) when the change occurs to a row or bunch of rows in a database table. Now, you could implement a polling mechanism that will check the table at regular intervals for those modifications and then instantiate the service when it finds any. However, I would prefer a more event driven approach.
I assume this is something that has been done elsewhere so I was wondering what approaches other people have used for dealing with such requirements?
Thanks in advance
Edit: The process that fires when a change occurs to the underlying data is essentially a call to an external web service using some of the related data. I am beginning to think whether this call should occur as part of the same process that is submitting the data into the database, rather than being triggered by the data change itself.
You should look at Oracle Database Extensions for .NET.
From the linked article:
Oracle Database Extensions for .NET provides the following:
A Common Language Runtime (CLR) host for Oracle Database
Data access through Oracle Data Provider for .NET classes
Oracle Deployment Wizard for Visual Studio .NET
You would still use triggers to detect the db changes but instead of firing all the db-side logic you describe you would now be able to execute that logic from a .NET module.
If you are using Oracle's .NET driver, you can use Oracle Continuous Query Notification (CQN) to do that.
You just give it a normal SELECT query, and it will fire a callback in your app whenever the resultset for that query changes.
The one caveot I know of is that when it initially runs the query to subscribe for continuous notification, it momentarily requires an exclusive lock. Usually its not a big deal since you just evecute it once at startup, so any other DB queries on the same table will be blocked for a fraction of a second.
It sounds possible but will likely take some leg work. I think you want to look into the Oracle Access Manager
http://download.oracle.com/docs/cd/E12530_01/oam.1014/e10355/toc.htm
This is similar to Paul's; but does not assume that you have Oracle installed on a Windows machine.
You may use dbms_scheduler to create a job that will call your external process.
You may directly call an remote external job from Oracle (this requires Oracle Scheduler Agent to be installed but nothing else)
it requires a bit of leg work to get the authentication set up and such, but this works.
then you utilize an event to start your job(called from your trigger).
This way, you may actually be able to utilize a lot of what you already have coded and just have the oracle scheduler to handle the rest.
Oracle provides two mechanisms to deal with what describe in a rather nice way... on the DB side you implement triggers for detecting the changes or whatever should result in an action on the .NET side...
For the communication you use a publish/subsribe mechanism based on the Oracle built-in queueing technology (called Advanced Queueing Technology, AQ) - for a nice example see http://www.devart.com/dotconnect/oracle/docs/AQ.html (not affiliated, just a happy customer).
Another option is to use built-in DBMS_ALERTER package for communication which is transactional and asynchronous - see for an example http://www.devart.com/dotconnect/oracle/docs/Devart.Data.Oracle~Devart.Data.Oracle.OracleAlerter.html
Just to be clear:
the above technologies (DBMS_ALERTER and AQ) are Oracle built-in ones, not specific to any 3rd-party libraries... you just need an ADO.NET provider supporting them...
EDIT - after the EDIT from the OP:
If you have control over the code or the call to the code that triggers the data change (WebService?) then it is indeed the best way to deal with it is purely on the .NET side of things... this also helps to deal with situations where such a change runs into an error etc.
I want to listen for changes to data in a SQL Server database from C#. I was hoping that there would be some sort of listener which I could use to determine if data that I have is stale. Despite being a fairly common scenario I can't find any solutions which aren't to simply poll the database.
I use Linq-To-SQL to access the data and hence have a DataContext object, I was hoping I could listen for an on data changed event but I can't seem to find one.
I appreciate that it's a non-trivial barrier (From C# method to SQL Server DB), the reason I expected this to be a solved problem is that it's a common requirement for GUIs. If it's not possible to Listen for updates how to you keep the Data displayed in a GUI fresh (When it's backed by a SQL Server data source).
Although this isn't for GUI work I was expecting to adapt something from that realm.
Is there a way to subscribe to SQL Server database change events in C#?
I've never used them before, but have you tried SQL Server Events notifications?
See this article: Getting Started with SQL Server Event Notifications
You're looking for the SqlDependency class, which allows you to listen for changes to the resultset of a SQL query.
The DataContext won't offer you any type of listener functionality with SQL Server. Your best bet is to create a polling application, or even a separate thread which polls the database periodically for changes and expose an event which your main application can listen to.
If you are using SQL Server 2008, there is a built in Change Data Capture that's pretty handy.
http://msdn.microsoft.com/en-us/library/bb522489.aspx
You can read the CDC data.
I would use a table with a single row in the db to catalog last updated, inserted, or deleted events and then create triggers on each table of importance to update this table and then poll this table for changes.
I was wondering how to monitor a database for changes programmatically.
Suppose I want to have a .net application which would run after every 100th or(nth) row insertion or row deletion or updation . how can it be achieved?
I know little about triggers.they can be used to fire executable.
But I heard that it isn't a good practice.
Is there any other way?
2]Do database fire events on table updations? and can they be caught in a program?
3]Can SQL reporting services be used here?
(Also assuming that this application is independent from the actual program which does
database manipulation.)
SQL Server 2005 introduced query
notifications, new functionality that
allows an application to request a
notification from SQL Server when the
results of a query change. Query
notifications allow programmers to
design applications that query the
database only when there is a change
to information that the application
has previously retrieved.
Check out the MSDN link for more clarity
and sample immplementation
A trigger is really going to be your only way unless you aren't concerned about the accuracy of "100th" or "nth".
The answer to 2 and 3 are no.
You can write managed stored procedures (MSDN example) but that doesn't help you here really. In general triggers can be bad practice since they can block the initial caller but sometimes they are the only solution.
I think you need to question your requirement to place this low-level data monitoring in a separate application. Think about where your data changes could originate -
Do you have full understanding of every:
stored proc within your db (now and future) and which ones update this table?
application that may hit your database (now and future)
If not, then watching the changes right down at the data level (ie within the db) is probably the best option, and that probably means triggers...
Read about "Service Broker" at http://msdn.microsoft.com/en-us/library/ms166104(v=SQL.90).aspx