Writing a client application that sends images to a server via a webservice. As the amount of data can be large i have a need for a progressbar that shows the progress. Can someone point me in the right direction on how to hook into the webservice so i can show the progress in the client.
When you have to send a lot of data and you don't have threads, an easy way to get a progress bar is splitting the data in smaller chunks, and send them one by one, that way you know the progress, of course, the service must be able to join the pieces together afterwards.
One idea would be the following:
Call the web service as normal.
Web service returns immediately providing a unique identifier (e.g., id or GUID).
The web service continues to process the request (asynchronously). Occasionally it will update a status location (e.g., DB field) with progress update.
The client calls a second web service providing the identifier as a parameter. This second web service then provides the current status. One of the status messages is 'complete'.
The client continues to poll for status updates until the 'complete' message is returned.
Upon receiving the 'complete' message, the client is calling a third web service to retrieve the final result (providing the identifier).
One of the possible approaches involves splitting the file into smaller chunks, uploading them chunk-by-chunk with separate service calls which allows you to show the progress.
A small drawback is that you have to put all the chunks into the big file at the server side when the last one gets there.
Related
Our webserver generates a file on the fly for download. This takes about two minutes of heavy processing because it's taking 1000 word templates, mail merging them, converting them to pdf, then making a zip out of those 1000 files. The processing is freezing the web server from being able to do anything else in the mean time which is a problem since it's hosting 23 subdomains for clients and they noticed it freezing.
How can i force the UI thread to get some work done? I've been looking at Thread.Sleep and Thread.Yield but perhaps i'm not implementing them correctly. I'm very new to the concept of threading.
When starting the processing on the web server, generate a "job ID" and store it somewhere (such as a database). Add an endpoint so the client can query the status of the job. When the processing is complete, the user can use the job ID to get the resulting file(s). It works like this:
User wants to process files. They call the start endpoint, which returns a JobId.
The server begins processing that job in a non-request thread, or the job is picked up and processed by another server dedicated to that task. When the thread completes, it updates the job's status accordingly.
Later...
User wants to know the status of their process. They call the status endpoint using their JobId periodically. The server replies with some status information to show the user.
Later...
Once the job's status has changed to 'complete', the user can call the result endpoint with their JobId to get the final file(s).
The heavy processing should be done in a non-UI, non-request thread so other users are unaffected.
Using this approach, you can even do the processing on another server entirely. All the web server is doing is allowing the user to create and query processing jobs.
I am currently working at a personal project in c#.
The client is in WPF and connects with a WCF service that saves some data from web (the database is mapped with entity).
So, my problem is at my wcf service.
I am not very pro in c# or multitasking, but I need some guidance before going forward.
The scenario sounds like this:
Save from client to WCF.
WCF gets the request and initiates the function.
Returns a message to client that data will be processed and starts a task that:
Downloads data from internet
Saves it.
The problem is that the data that I save has an unique id and I check before saving so that data doesn't exists in database.
But, if 2 clients calls simultaneously save function with same data, there will be two tasks that saves info with same id. The id's are the imdbId which are unique.
My idea is that I make a list of tasks and I start a "watcher" that watches over that list and when a client makes a save call, I add the task for saving in the list.
Therefore, the task are ran iterative from the list.
How can I achieve this idea in c#? Data types? Tasks? ThreadPool?
Also, I think that the list must be singleton.
I am currently working on a project which I think using soap as part of it would be a good idea but I can't find how it will work in the way that I need.
I have a C# Console Application called ConsoleApp, ConsoleApp will also have a PHP web interface. What I'm thinking of doing, is the PHP web interface controls the ConsoleApp in some way, so I click a button on the web interface, then this does a sends a soap request to a soap service and then the soap service, sends the information on to the consoleApp, and the result is returned back to the SoapService and then returned back to PHP.
This seems like it would need to separate soap services, one for php to interface with and one within the ConsoleApp but this doesn't sound right, I think I might be misunderstanding the purpose of Soap.
How can this be achieved. Thanks for any help you can provide
UPDATE
As requested I thought I'd add a bit more information on what I am trying to achieve.
In the console app, it is acting as an email server sending out emails that are given to the program and then being sent on, and if it can't send it retries a couple of times until the email goes into a failed state.
The web interface will provide a status of what the email server is doing, i.e. how many emails are incoming, how many are yet to be processed, how many have sent and how many have failed.
From the web page you will be able to shutdown or restart the email server or put one of the failed emails back into the the queue to be processed.
The idea is, when the user adds a failed email back into the queue it sends a soap message that the console app will receive, add the information back into the queue, log the event in the console apps log file, increment a counter which is how it keep track of emails that need to be processed. Once this has been done it should then send a response back to the web interface to say whether or not the email was successfully added back into the queue or whether it failed for some reason.
I don't really want to keep on polling the database every so many seconds as there could be the potential for their to be a large number of emails that will be being processed so polling the database would put a large load on the MySQL server which I don't want, which is why I thought soap as the email server would only need to do something when it receives a soap request to do something.
Thanks for any help.
Every web service is going to need a client (in your case PHP) and a server (ConsoleApp). Even though there are two endpoints, it is still one web service. Your PHP will send a SOAP request which ConsoleApp will receive, process and respond to with a SOAP response.
So when someone clicks the button on the web page, you can use JavaScript to build and send the SOAP envelope in the browser. The alternative is to POST the values to a PHP page that will build and send the SOAP.
I have to admit though, your scenario sounds a unusual. I personally haven't heard of web pages talking directly with console apps. Web pages usually talk to web servers, and the servers are usually the ones issuing atypical requests, like your request to ConsoleApp. While it is technically possible, but I think it is going to be harder then you are expecting.
Personally, I would ditch SOAP in favor of a much more simple and scalable solution. Assuming you have access to a database, I would have the PHP create a record in the database when the user clicks the button. ConsoleApp would then poll the database every X seconds to look for new records. When it finds a new record, it processes it.
This has the benefit of being simple (database access is almost always easier than SOAP) and scalable (you could easily run an arbitrary number of ConsoleApps to process all of the incoming requests if you are expecting heavy loads). Also, neither the PHP page nor the ConsoleApp have a direct dependency on the other so each individual component is less likely to cause a failure in the whole system.
I'm looking to create a web service and accompanying web app that uses an async web service call. I've seen plenty of suggestions on how to do async calls but none seem to fit exactly what i'm trying to do or are using a really outdated tech. I'm trying to do this in ASP.net 3.5 (VS2008)
What i need to do is:
the webpage needs to submit a request to the service
the page then needs to poll the service every 5 seconds or so to see if the task has completed
once complete the request needs to be retrieved from the service.
Could someone give me some suggestions or point me in the right direction?
The way I have typically handled asynchronous server-side processing is by:
Have the webpage initiate a request against a webservice and have the service return an ID to the long-running transaction. In my case, I have used Ajax with jQuery on the client webpage and a webservice that returns data in JSON format. ASP.NET MVC is particularly well suited for this, but you can use ASP.NET to return JSON string in response to a GET, or not use JSON at all.
Have the server create a record in a database that also stores the associated data to be processed. The ID of this transaction is returned to the client webpage. The service then sends a message to a third service via a message queue. In my case, the service was a WCF service hosted in a Windows Service with MSMQ as the intermediary. It should be noted that it is better not to do the actual task processing in ASP.NET, as it is not meant for requests that are long-running. In a high demand system you could exhaust available threads.
A third service receives and responds to the queued message by reading and processing necessary data from the database. It eventually marks the database record "complete".
The client webpage polls the webservice passing the transaction record ID. The webservice queries the database based on this ID to determine if the record is marked complete or not. If it is complete, it queries for the result dataset and returns it. Otherwise it returns an empty set.
The client webpage processes the webservice response, which will either contain the resulting data or an empty set, in which it should continue polling.
This just serves as an example, you may find that you can take shortcuts and avoid doing processing in a third service and just use ASP.NET threads. But that presents it's own problems, namely how you would have another request (the polling request) know if the original request is complete. The hackish-solution to that is to use a thread-safe collection in a static variable which would hold a transaction ID/result pair. But for that effort, it really is better to use a database.
EDIT: I see now that it appears to be a demonstration rather than a production system. I still stand by my above outline for "real-world" situations, but for a demo the "hackish" solution would suffice.
Which part are going to need to do async ? As far as I can tell your actions are synchronous:
1) -> 2) -> 3)
A simple web service would do, IIS (as any web server) supports multiple request to be handled async so you have no problem.
Something which you may need to be aware of. And also the javascript engine executes code in a single thread.
Step 0: Create the web service.
Step 1: Create the web app project (assuming it's ASP.NET).
Step 2: Add a web reference to the webs service to your web app project.
Step 3: The reference would create a proxy for you, using which you can invoke both synchronous and asynchronous calls.
I am developing an Application where I am submitting POST Requests to a .NET Web Service.
Current implementation is to process the request instantly and give response. In actual deployment, there will be huge amount of data that needs to be processed and thus the request must be processed offline.
What are the strategies that can have the task accomplished
Should I implement a Windows Service, or a scheduled task that invokes an application to perform the desired task.
This might be a good case for MSMQ. Your webservice can fill the queue with incoming data, and another process can read those messages and perform the necessary processing.
Here's a good overview of MSMQ:
http://www.primaryobjects.com/CMS/Article77.aspx
If you have so much data it cannot be processed in real-time, I would probably setup the service to do the following:
ProcessRecordViaPost
Create new record in "Queue" database with UniqueID, and all other info to be processed
Return UniqueID to client immediatly
ReadRecordViaGet
Check queue, if processed return data if not return status code (number of items in queue before it?)
I would also have a windows service that continually grabs the oldest item from the Queue, and processes it and moves on to the next oldest.