I am writing to seek advice to what approach and process to consider/or take, for creating a web api, which calls new data record automatically from the database table, when database table(db.data) is updated with a new data record.
Having done some research, I have only come up with creating sql trigger which can get data, from the db.data, database table, when the table is updated/after insert. After that, I am little unsure, how to connect that to web-api, if even possible to do so.
Any advice or suggestion would be very much appreciated, in what steps this task involve.
Many thanks.
I recommend .NET CLR Triggers. This is code you write in .NET (i.e. C#) which is then loaded into SQL Server and can be executed just as classic SQL trigger.
More information here: http://msdn.microsoft.com/en-us/library/ms131093(SQL.90).aspx
Related
I am using SignalR to update client-side when data is changing on the backend. The data can be either from SQL Server or NoSQL DB.
What I am doing now is: I have a timer on the backend that constantly is trying to check whether there is a change in the DB and if it detects it then I allow SignalR to update the client-side.
I find this approach not good: the timer's interval value can't be really small e.g. even with 1 second I have a huge load on the DB. Besides, it's not real-time and even 1 second is too big. Additionally, that it's quite difficult to detect a DB change if it's not about adding a new record, but calculating some hash for all the records every 1 second or even less is surely not an option.
I think I used to read about another approach that is based on some event triggered either by the DB or by something else, but I can't remember this.
So I was wondering if somebody could advise me of a better solution.
I'm afraid we need to take different actions for different scenarios on this case.
As you mentioned that your data may come from SQL server and nosql db. So I think the scenario should like:
You write code to update the data in both of the database, if so, you can write the calling signalr code after the updating data code, so that you are no need to using trigger.
Based on the scenario 1 as well, if the database you used support using stored procedure, you may also try to find the possibility of writing stored procedure to call the signalr code.
You may also check if the databases you used have event like "data_onchange_event", this document is related to this scenario. I mean if the data stored in the database updated, your database will send an event, you can then write custom code to capture these event and then call signalr.
If you can't or don't write code to update your databases, then you may only using external trigger to monitor the database, but just like you mentioned in the question, it doesn't support high performance because of huge load on the DB. It's based on the database but not the external trigger.
I've installed StreamInsight 2.1. I've been able to turn it on and Server.Create(), expose a WCF endpoint, and connect to it with the Event Flow Debugger. I've found plenty of examples of creating fake event sources.
I'd like to know how to create what I think to be possible: a standing query against a SQL 2012 database. I haven't found any sample code for connecting to an RDB - just statements that "This is just for show, for real you'll maybe connect to an RDB".
Can it only use LINQ-to-SQL? Entity Framework 4? plainer ADO.NET? Some MS SQL-specific protocol?
I guess the part that really confuses me is how these standing queries are supposed to work. It would seem odd to me, these days, for Microsoft to have an exposed capability in SQL Server that is ONLY accessible by a certain independent SKU (StreamInsight). Therefore, it must be one of these two:
Is SQL aware of the standing-ness of the query or does StreamInsight just modify a LINQ statement to poll efficiently (like by adding AND #LastExecution <= InsertedOn and proper indexes).
If SQL is aware of it as a special kind of query, what mechanism does StreamInsight use to create such a standing query? Is it something that a programmer can utilize without StreamInsight Services and so on?
The standing queries you mentioned are the ones written using LINQ that run inside the StreamInsight engine and not SQL Server itself. With StreamInsight, we the developers are on the hook for writing the adapter/source/sink code to get data in and out.
Can it only use LINQ-to-SQL? Entity Framework 4? plainer ADO.NET? Some MS SQL-specific protocol?
Yes. I have written this code before and I've had good luck with just plain ADO.NET. For what our needs were, it was way more flexible than LINQ-to-SQL and Entity Framework.
Do you have control over the schema of the table(s) you are querying? If you do, I would suggest adding some kind of LastUpdated datetime column that gets updated when the data in your table changes. That way you can write the query in such a way that only pulls back the changes without returning the whole table itself. Your adapter/source will need to have some kind of timer that executes your SQL statement periodically. You'll also need to maintain the last executed datetime so you can plug that into your SQL statement.
You could create a StreamInsight input adapter for SSSB to generate events based on messages from Server Broker queues.
I've not worked through this solution, but it seems to me that it would be possible to implement Service Broker or a SQL query notification in SQL Server to message the application on data change, then use Streaminsight to process and respond to this altered state.
I think this would give the effect of a "standing query" and avoid the (potentially wasteful) overhead of polling.
I'd like to know how to create what I think to be possible: a standing query against a SQL
2012 databas
And here we go. Not possible. Period. Why you think that is possible? Need to poll, the best is the ever waiting poll for the message queue, but no, standing query NOT POSSIBLE. Too bad.
Where did you get the funny idea it is possible?
I'm new to Azure, I want to know about how we can implement Data Export Service in C#.
Currently I am using windows azure for application development and sql azure for database. So there I want to read the data from my database and update that into some another client database based on a time interval.
I want to implement these function as a windows azure worker role. How can I do this?
Please help me...
I'm not sure what you're exactly trying to do but depending on your needs you could use one of the following options:
SQL DAC
This is more an import/export like approach (is this what you mean with "Data Export Service"?)
http://sqldacexamples.codeplex.com/wikipage?title=Import%20Export%20Service%20Client&referringTitle=Documentation
SQL Azure Data Sync
This is a synchronization solution, allowing you to use sync groups, filters, specific tables/columns, ... to synchronize data between multiple databases (both SQL Server and SQL Azure are supported).
http://msdn.microsoft.com/en-us/library/hh456371.aspx
As pointed out by Sandrino, you can use SQL Azure Data Sync. It supports the features you want to implement. Using an existing product is usually much easier than write your own.
If you want to write your own, please distinguish which data has been modified. You can take the same route as Data Sync does: Create triggers in the database. Whenever a data is inserted, updated, deleted, the trigger is invoked, and you insert some metadata in a tracking table. Then your worker role code queries the tracking table to figure out what data needs to be synched.
Best Regards,
Ming Xu.
I've written a blog post about this which uses some straightforward tooling to achieve a simple export and import, using task scheduler than worker-role which makes it pretty easy:
https://iainhunter.wordpress.com/2012/08/21/sql-azure-disaster-recovery/
I was wondering how to monitor a database for changes programmatically.
Suppose I want to have a .net application which would run after every 100th or(nth) row insertion or row deletion or updation . how can it be achieved?
I know little about triggers.they can be used to fire executable.
But I heard that it isn't a good practice.
Is there any other way?
2]Do database fire events on table updations? and can they be caught in a program?
3]Can SQL reporting services be used here?
(Also assuming that this application is independent from the actual program which does
database manipulation.)
SQL Server 2005 introduced query
notifications, new functionality that
allows an application to request a
notification from SQL Server when the
results of a query change. Query
notifications allow programmers to
design applications that query the
database only when there is a change
to information that the application
has previously retrieved.
Check out the MSDN link for more clarity
and sample immplementation
A trigger is really going to be your only way unless you aren't concerned about the accuracy of "100th" or "nth".
The answer to 2 and 3 are no.
You can write managed stored procedures (MSDN example) but that doesn't help you here really. In general triggers can be bad practice since they can block the initial caller but sometimes they are the only solution.
I think you need to question your requirement to place this low-level data monitoring in a separate application. Think about where your data changes could originate -
Do you have full understanding of every:
stored proc within your db (now and future) and which ones update this table?
application that may hit your database (now and future)
If not, then watching the changes right down at the data level (ie within the db) is probably the best option, and that probably means triggers...
Read about "Service Broker" at http://msdn.microsoft.com/en-us/library/ms166104(v=SQL.90).aspx
So I need to track changes that happen on a Mysql table. I was thinking of using triggers to log all the changes made to it and then save these changes in another table. Then I will have a cron script get all these changes and propagate the changes into the Mssql database.
I really dont expect a lot of information to be proporgated, but the data is very time sensitive. Ideally the MSSQL will see these changes within a minute, but I know that this requirement may be too high.
I was wondering if anyone had a better solution.
I have the bulk of the site written in .net but use vbulletin as the forums (sorry but there are no .net forums as powerful or feature rich like vbulletin)
The majority of the replicator tools use this technique. Fill another table on insert/update/delete triggers that containt the tablename and the PK or a unique key.
Then a reader reads this table, do the proper "select" if insert/update to get the data, then updates the other database.
HTH