I created a REST service using ASP.net Web API 2. The service allows a client to POST records which are stored in a SQL Server database. At this point some other process needs to pull the records from the database, process them, and send the results to another, separate REST service.
Since the processing of the records will be time-consuming, I don't want to perform this step as part of the POST action, but I'm trying to figure out the best approach to designing this other process.
Is there a way to integrate this functionality into the ASP.net REST API project, or is something like a Windows Service the best solution?
You could write your web service asynchronously. So that,
public async Task<HttpResponseMessage> PostRecords(Records records)
{
await [POST data to db]
// process records asynchronously without awaiting
}
You can also perform the post operation synchronously and at the same time track the completion of the post in your database. And as you mentioned, have a job or windows service process the completed posts separately.
You can use Quartz.NET scheduling library within the asp.net mvc or web api. Define a job which executes your query and fire the job from your post action.
Related
We have a pretty memory heavy OData/Web API written in C# and running on an Azure App Service instance. Some specific endpoints which get used rarely can take a long time to execute (20s) or they can take an extremely long time (10min) depending on the query. We would like in that case to have some of sort of background task mechanism for executing these kind of tasks, ideally a header we can attach to any call to tell the API to run it in the background.
We have tried:
Returning the request immediately with a tracking id and then running the SendAsync task in a HostingEnvironment.QueueBackgroundWorkItem, however when the work is completed it throws an exception as its trying to do something with the initial web request which is already disposed
Returning the request immediately with a tracking id and then passing the SendAsync to HostingEnvironment.QueueBackgroundWorkItem as a function, however that also throws an error upon activating
Returning the request immediately with a tracking id and then enqueuing a job in the Hangfire which will make a direct call to the app service and can wait as long as it takes. However even using the internal dns, the azure app service has a timeout of 230 seconds, and we can not change that in any way
We've also tried running the business code itself in a different thread, but we use autofac DI and if we try and get a new scope it still closes when the original request returns.
Similar SO questions have gotten answers that I don't think apply to us:
Running the work itself in Hangfire, that would mean spinning up the whole API again, just for these rare calls, also we can't do that because our Hangfire is on AKS on Linux, and the API is .net framework.
Using a webjob - similar issue as the api is very complex and deploying it to a webjob seems very dangerous, also we will then have to pay for 24/7 webjob.
Using a queue, the problem is that I don't really know how we can take a message from a queue and then execute it against the odata controllers, which we will need to do to populate various contexts and then get data from the DB that we need for these request.
Using IHostedService - Our API is .net framework and not .net core
Basically any thoughts on how we can achieve this, running an odata web request for as long as needed after we return a tracking id to the client (we understand the risks), without having to do another deployment of our API?
I am looking to create a Web Api c# fire and forget service that I can post a payload to and immediately return a success or failure based on some initial checks, but then do the rest of the heavy processing asynchronously after the return 200 has been made.
What is the best approach for situations like this?
I am struggling to find a concrete example on the web to be honest.
Thanks
Neil
I wouldn't call the best approach but one possible approach is using message queues.
Your flow would be:
Call the RESTful API.
The RESTful API resource enqueues a message and returns 200/OK if it could be done successfully.
Some asynchronous worker (a different process than the Web API one) dequeues messages overtime and processes them. The whole process can be a Windows service (do you know Topshelf?), a Windows Task Scheduler task...
If you're already in Azure, take a look at Service Bus, otherwise, you might want to learn more about a message queue server like RabbitMQ.
I have some experience with WCF services development but with this requirement I want to get some help/suggestion from some of the experienced developers here. Here is my scenario,
I will have a service(REST) (let calls it Service 1) which will receive requests from a different service (lets calls it Service Main)with some parameters. I am planing to save these parameters in a database so that I can track the status of the progress in future steps. Then I have to a start a process on the server from Service 1 which will run for in determinant time (based on the parameters) and lets call this process A. When process A is done with its task and comes back with good results then I have to start a different process which is called Process B which will use files generated by process A. When process B is done with its business and sends an acknowledgement to service 1 then I have to send the information back to Service Main.
For database i am planing to use no sql database since there are no relationships involved and it is more like a cache. I am having hard time on how to architect this entire process so that all of these steps/tasks run asynchronous and able to scale and handle lot of requests.
Approach 1: My initial idea was to have a wcf or ASP.NET Web api service(REST) use TPL framework to launch process A and wait for it to complete and call async callback method of process A then launch Process B on a new Task. But I am not sure if that is a good solutions or even possible.
Approach 2: After lot of reading i thought may be having a windows service on the hosted server to launch process A and process B. WCF service will talk to window service to start the process.
Hopefully I explained the problem clearly and waiting to hear some advises.
I am trying to get some asynchronous work done with the System.Threading.Tasks.Task class. The scenario is simple. I have a web app and in one button click event I start a Task which must run to check some outside service for a couple of minutes. It is not a heavy task. All it's going to do is send a request every 5 seconds and get a response. But it must do it for at least a couple of minutes. So, I don't want user to wait until this task gets job done. After I have started the task, I immediately return to the user saying that the task started and he/she will be informed when it is done. I wonder if this task I created will cause any problems, since I returned and ended the HTTP response.
This type of "asynchronous work" isn't possible by using the Task type. As I mention on my blog, async does not change the HTTP protocol; you still get one response per request, that's it!
The ideal ASP.NET app does not do any work outside of a request/response pair. There are ways to make it work (also described on my blog), but it's almost never recommended.
The proper solution is to split up the processing. A web site (or service) should start the processing by placing a request into persistent storage (e.g., Azure queue), a separate worker service (e.g., Azure worker role / Win32 service) would do the polling and put the results into persistent storage (e.g., Azure table), and the web site/service could poll that.
You can consider using message based service bus, and a good tutorial on MSDN Building
Distributed Apps with NHibernate and Rhino Service Bus will be
very useful.
If you just return from a standard asp.net request then wouldn't you expect the HttpResponse to end? Starting up a task in itself won't hold the HttpResponse open, to that you'd need to stream your response and block on the server until your task is finished which is presumably not what you want to do?
Maybe you should look at some ajax on the client that periodically pings the server to see if the task has finished, or at HTML 5 push notifications if you know your browser is going to support it.
You can use this http://www.asp.net/web-forms/tutorials/aspnet-45/using-asynchronous-methods-in-aspnet-45 but imho ajax with web service much better
I'm looking to create a web service and accompanying web app that uses an async web service call. I've seen plenty of suggestions on how to do async calls but none seem to fit exactly what i'm trying to do or are using a really outdated tech. I'm trying to do this in ASP.net 3.5 (VS2008)
What i need to do is:
the webpage needs to submit a request to the service
the page then needs to poll the service every 5 seconds or so to see if the task has completed
once complete the request needs to be retrieved from the service.
Could someone give me some suggestions or point me in the right direction?
The way I have typically handled asynchronous server-side processing is by:
Have the webpage initiate a request against a webservice and have the service return an ID to the long-running transaction. In my case, I have used Ajax with jQuery on the client webpage and a webservice that returns data in JSON format. ASP.NET MVC is particularly well suited for this, but you can use ASP.NET to return JSON string in response to a GET, or not use JSON at all.
Have the server create a record in a database that also stores the associated data to be processed. The ID of this transaction is returned to the client webpage. The service then sends a message to a third service via a message queue. In my case, the service was a WCF service hosted in a Windows Service with MSMQ as the intermediary. It should be noted that it is better not to do the actual task processing in ASP.NET, as it is not meant for requests that are long-running. In a high demand system you could exhaust available threads.
A third service receives and responds to the queued message by reading and processing necessary data from the database. It eventually marks the database record "complete".
The client webpage polls the webservice passing the transaction record ID. The webservice queries the database based on this ID to determine if the record is marked complete or not. If it is complete, it queries for the result dataset and returns it. Otherwise it returns an empty set.
The client webpage processes the webservice response, which will either contain the resulting data or an empty set, in which it should continue polling.
This just serves as an example, you may find that you can take shortcuts and avoid doing processing in a third service and just use ASP.NET threads. But that presents it's own problems, namely how you would have another request (the polling request) know if the original request is complete. The hackish-solution to that is to use a thread-safe collection in a static variable which would hold a transaction ID/result pair. But for that effort, it really is better to use a database.
EDIT: I see now that it appears to be a demonstration rather than a production system. I still stand by my above outline for "real-world" situations, but for a demo the "hackish" solution would suffice.
Which part are going to need to do async ? As far as I can tell your actions are synchronous:
1) -> 2) -> 3)
A simple web service would do, IIS (as any web server) supports multiple request to be handled async so you have no problem.
Something which you may need to be aware of. And also the javascript engine executes code in a single thread.
Step 0: Create the web service.
Step 1: Create the web app project (assuming it's ASP.NET).
Step 2: Add a web reference to the webs service to your web app project.
Step 3: The reference would create a proxy for you, using which you can invoke both synchronous and asynchronous calls.