I have a console application (in C#) to read data from SQL server (Microsoft SQL 2005) and write data to SQL server. What I need right now is to add a trigger to a table to exec the console application when any row of data is changed.
Not sure what SP available on Microsoft SQL server (2005) to launch a console application? and how can I pass the result from app back? Will it be run as sync or asych manor? Is there any permission issue I have to configure?
Don't launch external processes from a trigger, you'll bring the server onto its knees. Doesn't matter if is xp_cmdshell or CLR procedure. Instead use Service Broker and issue a SEND in your trigger to queue a message to a local service and rely on activation to do the external dependent processing, asynchronously and on a separate context.
The xp_cmdshell stored procedure can be used to launch an external process but this is often disabled for security reasons.
As the console application is already written in C#, perhaps it could be re-written as an SQLCLR stored procedure?
This seems a little shaky to me. Is there any chance of the trigger being called frequently, leading to many launches of the application? Also, I don't think the trigger will complete until the console application has finished or failed. Meantime, the operation that caused the trigger to fire will still be waiting.
Does the application need to run right away? If not, then maybe you could run this as a SQL Agent job periodically.
Trigger slow things down and in your case, this will lead to mass chaos. Triggers is not the route you want to take here.
You might want to consider having a console app that polls the database and every time it finds changes, it will display the changed rows for the users consumption.
You can track these changes using a field like [LastUpdatedDateTime] with a default of GetDate(), and don't send this value in your query. Therefore it will always have the latest timestamp of change. You can alternatively, have an audit table that gets filled by a trigger.
I agree with Ken, you might want to think about changing the architecture. Is the console application reading and writing data to the same SQL server that is invoking it? If so you are better off coding that logic into the trigger or stored procedure itself, and/or changing your database schema to make it so your logic doesn't have to be so complicated.
Related
I'm making a web application in aspnet, and I created a table in the database to insert notifications. I would like to know if there is any way to automatically delete data from this table that is older than 30 days. Is there any way this can be done?
Build a Stored Procedure with T-SQL that remove your older data than 30 days.
Assign procedure to SQL Server Agent.
Run server agent every day at specific hours.
With that way, your procedure will run every day and will delete your older data than 30 days according to the Procedure that you write.
I would like to know if there is any way to automatically
SQL Server build-in tool for schedule tasks (also called jobs) is the SQL Server Agent.
https://learn.microsoft.com/en-us/sql/ssms/agent/sql-server-agent
You can create a JOB which run a DELETE query
Note! I HIGHLY recommend NOT to delete data from the database in most cases! Disk are cheap and you will never know when the history information will be needed, but more important is simply that you should not have a reason for this in most cases. In big system you are using partitions and you can store the old data in separate file as well and there is no need to backup that file every time. In small system you can simply buy another cheap HDD disk if needed. Instead of DELETE rows you should design the system to mark the rows as logically deleted (for example add another column) if needed
Update: and what if we are using SQL Server Express?!?
SQL Server express does not support the use of the SQL Server Agent, which means that we need a different solution to schedule task, and the common solutions you can use are:
Solution 1: Using external scheduler
Disadvantage: This solution is in the Operating System level
If you are using SQL Server on windows, then you can Windows Task Scheduler: start menu -> search for "Task Scheduler" -> Right click "Task Scheduler" and select "Create Basic Task" -> fill the task information -> choose "Start a Program" as the action type and execute a ".bat" file which include sqlcmd command which execute your query
If you are using Linux then you can use Cron daemon - a system-managed executable that runs in memory in the background with which users may schedule tasks.
Solution 2: schedule tasks using SQL Server SERVICE (service broker); CONVERSATION TIMER; MESSAGE; CONTRACT; QUEUE
Note! This is my preferred option and I use it even when Agent is supported for many reasons! Service broker is super reliable, it's part of the SQL Engine, there is almost no overhead, it allows asynchronous and timed (repeated) execution at any time interval. With that said, it does not have access to the operating system like the SQL Agent does.
You can get a full sample including all the code to implement this solution in this post: https://www.sqlteam.com/articles/scheduling-jobs-in-sql-server-express-2
I need to call a restful web service with a single parameter from the insert event in a SQL Server database table. I know this is not ideal design but I don't have control of either end point. The usage of this functionality will not exceed 100 events/min at max load.
I'm open to other ideas but the two options I came up with were;
insert trigger - service broker - .NET windows service to call web service
C# CLR trigger to call restful service directly
I have been working on option 2 and created a SQL CLR trigger project in Visual Studio but after I add the references to System.Web the project will not build and there are no build errors, vs just says "build failed" in the output window.
Is there some restriction about which CLR libraries can be used in a CLR trigger?
Using the service broker scares me since I have only worked on one project with the service broker and found it very difficult to implement.
Any ideas on how to call the web service from the trigger event would be greatly appreciated.
A trigger is run in the context and scope of the statement that causes it to fire - this means, that statement isn't going to complete until the trigger is completely run.
A trigger should be very nimble - small and fast. You should never ever call external services from a trigger, you should not include cursors in triggers, you should not do any heavy lifting or lengthy calculations in a trigger.
If you really must do something like this, I'd recommend an approach that:
see the trigger just put a few values into a "Command" table - those values that the long-running process will need to complete its work
have a de-coupled, separate process (e.g. a stored procedure or something) that will check that "Command" table periodically for new tasks to complete - this can be done in SQL Server using a SQL Server Agent Job
the decoupled process then grabs the information from the "Command" table, does it's work, and updates the database (if necessary)
This way, your trigger is small and nimble and completes quickly, thus not slowing down your main process / main system. The lengthy process is decoupled, standalone, and can be implemented in whichever way makes most sense (stored procedure inside SQL Server, or a separate standalone e.g. command-line tool or whatever makes sense).
I'm working on a project which requires execution of C# method / function (coded on a web page) using a trigger or stored procedure in SQL Server.
I found this Execute-NET-Code-under-SQL-Server
But as stated on this link, I have to change the database property TRUSTWORTHY to ON.
Is there any way I can do this without changing database properties?
Thanks in advance
Well, if you don't want (or can) use SQLCLR - you can create some standalone console application containing all c# code you need and then execute this application with some parameters using xp_cmdshell.
But note: xp_cmdshell is disabled by default, you have to enable it using
EXEC sp_configure 'xp_cmdshell', 1
Since xp_cmdshell is server option - you don't need to change database properties, but you have to change server options to enable it in the case it is disabled. Choose yourself - what to change. ;)
Do you need to execute the external code synchronously? In other words, must the external code start and run to completion before your trigger ends?
If the answer is yes, then I suggest changing your design. You're using triggers incorrectly and making whatever DML operation fires the trigger tremendously slow.
Assuming the answer is no then, you have options to queue up whatever work needs to be done and have that work executed asynchronously. For example, your trigger could create a record in a work-queue table when it needs this external code to run. Some other process (e.g. a scheduled task, SQL Agent, something in your application, SQL Server Broker) can monitor this table for new records and fire off the appropriate calls as needed.
I've wrote a Restaurant Management application.
I have a Database based on SQL Server 2005 which has one table named OrdersItems. Every 5 minutes I want to read all rows of this table and based on a specific criteria Update some fields.
I don't want to do that in my main application and I prefer to have an Alternative engine to perform this.
Which method is the best method to perform such task ? Also note that this Table (OrdersItems) is under process every time because main application must be always running and get new Restaurant Orders.
You can create a SQL Server Agent job that does the update every five minutes.
If you are using SQL Server Express edition, you can't use SQL Server Agent because it's only included in the "bigger" versions of SQL Server.
In this case, you can create your jobs manually using batch files and Windows Task Scheduler.
I definitely agree with Christian and dougajmcdonald's points about using SQL Task/ Maintenance. However since you included c# in your tags an alternative is to create a Windows Service.
The benefits of this approach
Can run on a machine that doesn't have the SQL Server Agent installed (including express editions)
Can be run outside the context of a user login.
Has standard stop start pause continue mechanism that's well understood.
If the service itself fails will likely result in an event log
This answer contains a template for a windows service that periodically gets data and executes. You may simply want to change the DoStuff method to execute a stored procedure
Create a dialog timer and let it activate a stored procedure. This has the advantage of being fully contained inside the database (no external process), it does not require SQL Agent (runs on Express) and is completely crash resilient at the point it will survive database detach/attach and backup/restore operations (the scheduled job will run after recovery on the new restored database).
I would expect a SQL Task / Maintenance plan would be the best for this.
You can set them up for whatever interval you want, specifying a SQL statement, maintenance task etc you want to run.
You can also setup alerts etc if you want to know when it fails for example.
Deploy a cron job on a server with access to the database which is started every 5 minutes and processes your data, using transactions. I see one problem there: If the amount of data to be processed is large, it could quite work more than five minutes.
We have a reporting app thats needs to update it's charts as the data gets written to it's corresponding table. (the report is based off just one table). Currently we just keep the last read sessionid + rowid (unique combo) in memory and a polling timer just does a select where rowid > what we have in memory (to get the latest rows added). Timer runs every second or so and the fast sql reader does it's job well. So far so good. However I feel this is not optimal because sometimes there are pauses in the data writes due to the process by design. (user clicking the pause button on the system that writes data ..). Meanwhile our timer keeps hitting the db and does not get any new rows. No errors or anything. How is this situation normally handled. The app that writes the data is separate from the reporting app. The 2 apps run on different machines. Bottomline : How to get data into a c# app as and when it is written into a sql server table without polling unnecessarily. thank you
SQL Server has the capability to notify a waiting application for changes, see The Mysterious Notification. This is how SqlDependency works. But this will only work up to a certain threshold of data change rate. If your data changes too frequently then the cost of setting up a query notification just to be immediately invalidated by receiving the notification is too much. For really high end rates of changes the best place is to notify the application directly from the writer, usually achieved via some forms of a pub-sub infrastructure.
You could also attempt a mixed approach: pool for changes in your display application and only set up a query notification if there are no changes. This way you avoid the cost of constantly setting up Query Notifications when the rate of changes is high, but you also get the benefits of non-pooling once the writes settle down.
Unfortunately the only 'proper' way is to poll, however you can reduce the cost of this polling by having SQL wait in a loop (make sure you WAITFOR something like 30ms each loop pass) until data is available (or a set time period elapses, e.g. 10s). This is commonly used when writing SQL pseudoqueues.
You could use extended procs - but that is fragile, or, you could drop messages into MSMQ.
If your reporting application is running on a single server then you can have the application that is writing the data to SQL Server also send a message to the reporting app letting it know that new data is available.
However, having your application connect to the server to see if new records have been added is the most common way of doing it. As long as you do the polling on a background thread, it shouldn't effect the performance of your application at all.
you will need to push the event out of the database into the realm of your application.
The application will need to listen for the message. (you will need to decide what listening means - what port, what protocol, what format etc.)
The database will send the message based on the event through a trigger. (you need to look up how to use external application logic in triggers)