I need to call a restful web service with a single parameter from the insert event in a SQL Server database table. I know this is not ideal design but I don't have control of either end point. The usage of this functionality will not exceed 100 events/min at max load.
I'm open to other ideas but the two options I came up with were;
insert trigger - service broker - .NET windows service to call web service
C# CLR trigger to call restful service directly
I have been working on option 2 and created a SQL CLR trigger project in Visual Studio but after I add the references to System.Web the project will not build and there are no build errors, vs just says "build failed" in the output window.
Is there some restriction about which CLR libraries can be used in a CLR trigger?
Using the service broker scares me since I have only worked on one project with the service broker and found it very difficult to implement.
Any ideas on how to call the web service from the trigger event would be greatly appreciated.
A trigger is run in the context and scope of the statement that causes it to fire - this means, that statement isn't going to complete until the trigger is completely run.
A trigger should be very nimble - small and fast. You should never ever call external services from a trigger, you should not include cursors in triggers, you should not do any heavy lifting or lengthy calculations in a trigger.
If you really must do something like this, I'd recommend an approach that:
see the trigger just put a few values into a "Command" table - those values that the long-running process will need to complete its work
have a de-coupled, separate process (e.g. a stored procedure or something) that will check that "Command" table periodically for new tasks to complete - this can be done in SQL Server using a SQL Server Agent Job
the decoupled process then grabs the information from the "Command" table, does it's work, and updates the database (if necessary)
This way, your trigger is small and nimble and completes quickly, thus not slowing down your main process / main system. The lengthy process is decoupled, standalone, and can be implemented in whichever way makes most sense (stored procedure inside SQL Server, or a separate standalone e.g. command-line tool or whatever makes sense).
Related
I'm working on a project which requires execution of C# method / function (coded on a web page) using a trigger or stored procedure in SQL Server.
I found this Execute-NET-Code-under-SQL-Server
But as stated on this link, I have to change the database property TRUSTWORTHY to ON.
Is there any way I can do this without changing database properties?
Thanks in advance
Well, if you don't want (or can) use SQLCLR - you can create some standalone console application containing all c# code you need and then execute this application with some parameters using xp_cmdshell.
But note: xp_cmdshell is disabled by default, you have to enable it using
EXEC sp_configure 'xp_cmdshell', 1
Since xp_cmdshell is server option - you don't need to change database properties, but you have to change server options to enable it in the case it is disabled. Choose yourself - what to change. ;)
Do you need to execute the external code synchronously? In other words, must the external code start and run to completion before your trigger ends?
If the answer is yes, then I suggest changing your design. You're using triggers incorrectly and making whatever DML operation fires the trigger tremendously slow.
Assuming the answer is no then, you have options to queue up whatever work needs to be done and have that work executed asynchronously. For example, your trigger could create a record in a work-queue table when it needs this external code to run. Some other process (e.g. a scheduled task, SQL Agent, something in your application, SQL Server Broker) can monitor this table for new records and fire off the appropriate calls as needed.
I am developing a data optimization project. It has a client side web app to receive tasks from users. Let's say the tasks are some heavy calculations that cannot be done easily by normal systems. The program takes a long while for big amount of data to be calculated. So what I am trying to do is receive calculation orders from my web application and have a windows service on my server side to listen for new tasks to be done.
I would like my service to be listening to the data that is being inserted into my Tasks table and run the calculator based on the time of those dates. I will of course would have to deal with some multi-threading. And maybe if the program is busy, the other processes would have to wait.
I also don't mind having a small GUI for my application to see which orders are now being processed and whether my service is busy or idle.
I first thought about adding SQL Server jobs in my data base to query the table frequently and run the application based on the dates. But that does not look like a nice solution to me. What I want is a nimble ready-to-serve service who becomes aware when we have new data in the database and decides what to do.
I don't insist on Windows Services particularly here. So any good idea is welcome.
I've done something similar in Powershell.
What I have:
A "queue" table where requests gather
Requests come in by clients calling a procedure
A "queue history" table - request already "treated" go from "queue" to "queue_history" (delete trigger)
A powershell loop reading queue table
Requests from queue are started independently via start-job command, those are actually separate Powershell instances (my tasks are mostly some external exe calls, but I think could be usefull for stored procedure calls also).
The powershell loop also report his "heartbeat" to a table.
Also watches a table and file existance for STOP flag so I can stop it.
I have a need to call external web service on certain events in my application. I don't want to modify my application and make any dependencies of that external web service. So, I need to think of a way to do this with some sort of external component.
One possible approach is that I make database view which will get filled up when some events in my application occurs. Then I will set up trigger on that view which will call CLR function. In that CLR function I will make call to external web service. By doing this, I will get "real-time" integration which is good. But, this approach has downsides. Major one is that it seems that calling web service from CLR is not a good idea since it will block main SQL thread (?!) until CLR receives some answer.
Until now, I have only found that setting this property will help with performance issues:
System.Net.ServicePointManager.DefaultConnectionLimit = 9999
More about it you can find here.
Now, since you know my needs (that is real-time or at least close-to-real-time integration without any calls from my application to exteranl web service) is there some better way to do it?
One other approach I can think of is having some service which will periodically check for changes in my DB that needs to trigger calls to external web service. Once this service detects such change, it will call web service and transfer data. This is not true real-time integration of course. I must admit that, except for performance issues, I like having triggers and CLR much more since it guarantees real-time integration and has no affect on my application whatsoever.
I am not sure that I would agree with the design of moving the web-service call to a database. However, I am sure there are reasons as to why you wouldn't want to change the application.
Here are a couple of options that you can try -
1) Instead of a database, and CLR making web-service calls, use a message queue. NServiceBus is a good choice for passing event occurrences as message, which can trigger this call
2) If you are stuck with using SQL server to store the events, look at SQL server Service broker
I've wrote a Restaurant Management application.
I have a Database based on SQL Server 2005 which has one table named OrdersItems. Every 5 minutes I want to read all rows of this table and based on a specific criteria Update some fields.
I don't want to do that in my main application and I prefer to have an Alternative engine to perform this.
Which method is the best method to perform such task ? Also note that this Table (OrdersItems) is under process every time because main application must be always running and get new Restaurant Orders.
You can create a SQL Server Agent job that does the update every five minutes.
If you are using SQL Server Express edition, you can't use SQL Server Agent because it's only included in the "bigger" versions of SQL Server.
In this case, you can create your jobs manually using batch files and Windows Task Scheduler.
I definitely agree with Christian and dougajmcdonald's points about using SQL Task/ Maintenance. However since you included c# in your tags an alternative is to create a Windows Service.
The benefits of this approach
Can run on a machine that doesn't have the SQL Server Agent installed (including express editions)
Can be run outside the context of a user login.
Has standard stop start pause continue mechanism that's well understood.
If the service itself fails will likely result in an event log
This answer contains a template for a windows service that periodically gets data and executes. You may simply want to change the DoStuff method to execute a stored procedure
Create a dialog timer and let it activate a stored procedure. This has the advantage of being fully contained inside the database (no external process), it does not require SQL Agent (runs on Express) and is completely crash resilient at the point it will survive database detach/attach and backup/restore operations (the scheduled job will run after recovery on the new restored database).
I would expect a SQL Task / Maintenance plan would be the best for this.
You can set them up for whatever interval you want, specifying a SQL statement, maintenance task etc you want to run.
You can also setup alerts etc if you want to know when it fails for example.
Deploy a cron job on a server with access to the database which is started every 5 minutes and processes your data, using transactions. I see one problem there: If the amount of data to be processed is large, it could quite work more than five minutes.
I have a console application (in C#) to read data from SQL server (Microsoft SQL 2005) and write data to SQL server. What I need right now is to add a trigger to a table to exec the console application when any row of data is changed.
Not sure what SP available on Microsoft SQL server (2005) to launch a console application? and how can I pass the result from app back? Will it be run as sync or asych manor? Is there any permission issue I have to configure?
Don't launch external processes from a trigger, you'll bring the server onto its knees. Doesn't matter if is xp_cmdshell or CLR procedure. Instead use Service Broker and issue a SEND in your trigger to queue a message to a local service and rely on activation to do the external dependent processing, asynchronously and on a separate context.
The xp_cmdshell stored procedure can be used to launch an external process but this is often disabled for security reasons.
As the console application is already written in C#, perhaps it could be re-written as an SQLCLR stored procedure?
This seems a little shaky to me. Is there any chance of the trigger being called frequently, leading to many launches of the application? Also, I don't think the trigger will complete until the console application has finished or failed. Meantime, the operation that caused the trigger to fire will still be waiting.
Does the application need to run right away? If not, then maybe you could run this as a SQL Agent job periodically.
Trigger slow things down and in your case, this will lead to mass chaos. Triggers is not the route you want to take here.
You might want to consider having a console app that polls the database and every time it finds changes, it will display the changed rows for the users consumption.
You can track these changes using a field like [LastUpdatedDateTime] with a default of GetDate(), and don't send this value in your query. Therefore it will always have the latest timestamp of change. You can alternatively, have an audit table that gets filled by a trigger.
I agree with Ken, you might want to think about changing the architecture. Is the console application reading and writing data to the same SQL server that is invoking it? If so you are better off coding that logic into the trigger or stored procedure itself, and/or changing your database schema to make it so your logic doesn't have to be so complicated.