We Keep Coding

How to correctly implement a bi-directional streaming http proxy?

I'm trying to write a .NET Core API Controller action that would basically act like a HTTP proxy:
Data comes in POSTed via Request.Body stream
Then POSTed to another backend service which processes the input data stream live and starts sending result back while still reading and processing input
Resulting POST response is read and streamed back to the caller of this 'proxy'
I tried a couple of things, playing with both HttpClient and old HttpWebRequest but still can't make this work properly, the process deadlocks somewhere.
I would also like to better control if possible the HTTP request output buffers as I observed that data starts to pile up in-memory if backend service can't accept it fast enough (stream.Write() calls don't block immediately and process memory spikes).
Any ideas or examples?

Best practices: Synchronous response from asynchronous processing

A Web API receives a customer's credit card request data at an endpoint.
The endpoint sends a message with this data to the Kafka.
Several pods/containers will be connected to Kafka in this topic for processing each request in parallel. The application requires high asynchronous processing power.
After sending the request, the frontend will display a progress bar and will wait. It needs a response as soon as the process is finished.
The question
How to return in the same call to this endpoint the result of a
processing that will be done in another web API project?
(asynchronous)
What I thought
Creating a topic in Kafka to be notified of the completion of processing and subscribe to it in the endpoint after sending the CreditCardRequest message to process on Kafka.
Using a query on the mongo on every 3~4 seconds (pooling) and check if the record has been included by the Worker / Pod / Processing Container. (URRGGH)
Creating another endpoint for the frontend to query the operation status, this endpoint will also do a query in the mongo to check the current status of the process.
I wonder deeply if there is no framework/standard already used for these situations.

yes, there are frameworks that handle this.
From a .NET perspective, I have used nServiceBus todo something similar in the past (coordinate long-running processes).
They have a concept called Sagas https://docs.particular.net/nservicebus/sagas/
A saga will wait for all messages that are required to finish processing before notifying the next step of the process to continue.
If the framework is not useful, hopefully, the processes are and you can discover how to implement in a Kafka/Mongo environment.

Thread management when populating a cache in a .NET web app

I have a .NET MVC web app for data reporting. When the app first loads, it requests a lot of data from remote servers then caches it locally in web server memory. Before the cache loads, there's no way to respond to incoming requests. Every request that comes in while the cache is loading must wait for the thread loading the cache.
If I use a critical section (lock) on the caching code, all the requests will block. That's a huge waste of resources and I could even exhaust my IIS worker thread pool.
If I use async requests, the web requests will all return before the cache is loaded. I can't "callback" a web request which has already returned its contents to the client!
How can I manage the threads properly? Maybe there a way to move all requests to a single thread which asynchronously waits for the cache to load and then move them back out to individual threads once the cache is loaded?

If I use a critical section (lock) on the caching code, all the requests will block.
That is true. You can use SemaphoreSlim.WaitAsync to asynchronously wait. This behaves like a lock. You probably want a solution for the case that many requests queue up (like 1000s). You can use a second semaphore for that with a max count of 1000 and a wait timeout of zero. If the wait fails you know that >= 1000 requests are running and you can fail the request.
Big downside is that now all your requests have some async component to it. Maybe you can do this in some central place such as an async MVC action filter. Otherwise you will be forced to make all MVC actions async which is a headache.
Be sure to correctly configure all ASP.NET and IIS queues.
If I use async requests
Not sure you understand what async means in the context of ASP.NET. Async request processing is an implementation detail of the server. The client can't detect it. The request is not prematurely completed. Async IO (and any other form of async blocking) does not cause the request to end prematurely.

Don't put the Loading of the cache in a new thread: If you want that no requests are served you should load the cache in app_start. If the application loaded reuqests are served again.

Does the app require restarting often? If not, you can have some type of a fetch routine that requests updated data from remote web servers, and swap out all relevant caches in one critical section.
In essence, fetch the data in the background and commit all at once (you can lock during the commit to avoid contention over proper values).

Using Task with ASP.NET

I am trying to get some asynchronous work done with the System.Threading.Tasks.Task class. The scenario is simple. I have a web app and in one button click event I start a Task which must run to check some outside service for a couple of minutes. It is not a heavy task. All it's going to do is send a request every 5 seconds and get a response. But it must do it for at least a couple of minutes. So, I don't want user to wait until this task gets job done. After I have started the task, I immediately return to the user saying that the task started and he/she will be informed when it is done. I wonder if this task I created will cause any problems, since I returned and ended the HTTP response.

This type of "asynchronous work" isn't possible by using the Task type. As I mention on my blog, async does not change the HTTP protocol; you still get one response per request, that's it!
The ideal ASP.NET app does not do any work outside of a request/response pair. There are ways to make it work (also described on my blog), but it's almost never recommended.
The proper solution is to split up the processing. A web site (or service) should start the processing by placing a request into persistent storage (e.g., Azure queue), a separate worker service (e.g., Azure worker role / Win32 service) would do the polling and put the results into persistent storage (e.g., Azure table), and the web site/service could poll that.

You can consider using message based service bus, and a good tutorial on MSDN Building
Distributed Apps with NHibernate and Rhino Service Bus will be
very useful.

If you just return from a standard asp.net request then wouldn't you expect the HttpResponse to end? Starting up a task in itself won't hold the HttpResponse open, to that you'd need to stream your response and block on the server until your task is finished which is presumably not what you want to do?
Maybe you should look at some ajax on the client that periodically pings the server to see if the task has finished, or at HTML 5 push notifications if you know your browser is going to support it.

You can use this http://www.asp.net/web-forms/tutorials/aspnet-45/using-asynchronous-methods-in-aspnet-45 but imho ajax with web service much better

Async Web Service Calls

I'm looking to create a web service and accompanying web app that uses an async web service call. I've seen plenty of suggestions on how to do async calls but none seem to fit exactly what i'm trying to do or are using a really outdated tech. I'm trying to do this in ASP.net 3.5 (VS2008)
What i need to do is:
the webpage needs to submit a request to the service
the page then needs to poll the service every 5 seconds or so to see if the task has completed
once complete the request needs to be retrieved from the service.
Could someone give me some suggestions or point me in the right direction?

The way I have typically handled asynchronous server-side processing is by:
Have the webpage initiate a request against a webservice and have the service return an ID to the long-running transaction. In my case, I have used Ajax with jQuery on the client webpage and a webservice that returns data in JSON format. ASP.NET MVC is particularly well suited for this, but you can use ASP.NET to return JSON string in response to a GET, or not use JSON at all.
Have the server create a record in a database that also stores the associated data to be processed. The ID of this transaction is returned to the client webpage. The service then sends a message to a third service via a message queue. In my case, the service was a WCF service hosted in a Windows Service with MSMQ as the intermediary. It should be noted that it is better not to do the actual task processing in ASP.NET, as it is not meant for requests that are long-running. In a high demand system you could exhaust available threads.
A third service receives and responds to the queued message by reading and processing necessary data from the database. It eventually marks the database record "complete".
The client webpage polls the webservice passing the transaction record ID. The webservice queries the database based on this ID to determine if the record is marked complete or not. If it is complete, it queries for the result dataset and returns it. Otherwise it returns an empty set.
The client webpage processes the webservice response, which will either contain the resulting data or an empty set, in which it should continue polling.
This just serves as an example, you may find that you can take shortcuts and avoid doing processing in a third service and just use ASP.NET threads. But that presents it's own problems, namely how you would have another request (the polling request) know if the original request is complete. The hackish-solution to that is to use a thread-safe collection in a static variable which would hold a transaction ID/result pair. But for that effort, it really is better to use a database.
EDIT: I see now that it appears to be a demonstration rather than a production system. I still stand by my above outline for "real-world" situations, but for a demo the "hackish" solution would suffice.

Which part are going to need to do async ? As far as I can tell your actions are synchronous:
1) -> 2) -> 3)
A simple web service would do, IIS (as any web server) supports multiple request to be handled async so you have no problem.
Something which you may need to be aware of. And also the javascript engine executes code in a single thread.

Step 0: Create the web service.
Step 1: Create the web app project (assuming it's ASP.NET).
Step 2: Add a web reference to the webs service to your web app project.
Step 3: The reference would create a proxy for you, using which you can invoke both synchronous and asynchronous calls.