I'm wondering to use SignalR in my web-application (c#, mvc).
Scenario: my users giving me input and I processing it and show them back the results.
This processing can be very long. How long? let's say 3 minutes (depending on netwok traffic, usage, ... - cannot be predicated).
Currently, I doing this process in AJAX request for long polling. While AJAX is running, I display on screen: "Please wait".
Now I added to this scenatio a new server limitation: Since I'm using CloudFlare, they are limiting me that each request must takes less then 100 seconds. Otherwise, they abort the request.
So, I'm though about this and decide to check the option to move into SignalR. Why? because SignalR can manage this long polling for me. And mostly use another approach (like sockets or other technics), which may avoid this 100 seconds server limitation.
I'm reading in SignalR website that they checking the client capabilities and decide with which technology to use.
My concern: Since CloudFlare is limiting request\response to 100 seconds, this can do problems to SignalR. Let's say the my client is client without any new web feature (like WebSocket or other). This could lead the SignalR to do long polling - which may failed.
Is possible to define SignalR to avoid long polling?
Or
How do you recommend to avoid this problematic case.
SignalR allows you to configure the maximum amount of time the server will keep a long polling request open.
By default, the SignalR server will close poll requests that have been open for 110 seconds without receiving messages. Of course, if a message is sent to a client before the 110 seconds is up, the poll request will be closed when the message is sent. In both scenarios, the SignalR client will repoll when the server closes the previous poll (otherwise it wouldn't be long polling I guess).
You can lower the default timeout of 110 seconds during your application startup using IConfigurationManager.ConnectionTimeout:
// Make long polling connections wait a maximum of 60 seconds for a
// response. When that time expires, trigger a timeout command and
// make the client reconnect.
GlobalHost.Configuration.ConnectionTimeout = TimeSpan.FromSeconds(60);
http://www.asp.net/signalr/overview/signalr-20/hubs-api/handling-connection-lifetime-events#connectiontimeout
Related
I've built a bot using botframework V4 for .Net that replies to user for both email and directline channels.
However, some of the request takes more than 15 seconds to complete, therefore I'd receive a GatewayTimeout error:
These requests are heavy (fetch some data from the database, fetch other data from another server via API calls, process the data, generate HTML and send them back to the user...) therefore nothing can be done to shorten the process.
I am aware that the gateway timeout delay is by design (the 15 seconds), but the problem is that the channel automatically retries the request after a small period of time and I end up receiving multiple emails for the same query (approx. 1 minute apart each).
I noticed as well that the directline replies are much faster than email ones (websocket vs SMTP), therefore this is mainly occurring with the email channel only. Noting that the emails are kept under 300KB as per this comment but can easily have a size close to this limit.
Therefore, is there a way to:
Increase the timeout delay?
Disable the automatic retries?
Or perhaps a certain workaround to prevent this issue?
Remember that your bot is a web app that exposes an HTTP endpoint, and every activity sent to your bot is an API call. Long-running API calls should be designed to return a response immediately and do their processing asynchronously. For example, consider the Recognize Text Computer Vision API. It just returns an Operation-Location where the actual result will become available later.
For bot Framework bots, all you have to do to send a message to the channel after the turn already ended is to send a proactive message. It's often also a good idea to design your bot to give the user an indication that the result is coming, such as by sending a preliminary "processing" message or a typing indicator, but that's probably unwanted in the case of the email channel. Eric Dahlvang explained this in the issue you linked to:
If the developer knows the response will take longer than 15 seconds, it is possible, depending on the channel, to start a separate thread to handle the long running process, return a valid status code on the receiving thread, and when the process finishes, send a proactive message from the background thread.
I'm currently developing website in asp core 2.2. This site use external API. But I have one big problem and don't know how to solve this. This external API has limit 10 reguest per IP/s. If 11 user click button on my site and call API at the same time, the API can cut me off for a couple hours. The API owner tells clients to take care of not exceeding the limit. Can you have any idea how doing this?
ps. Of course, a million users are a joke, but I want the site to be publicly available :)
That 10 request/s is a hard limit and it seems like theres no way around it. So you have to solve it on your end.
There are couple options:
Calls that API directly using Javascript. This way each user will be able to do 10 request/s instead of 10 request/s for all users (recommended)
Queue the requests and only send out at most 10/s (highly not recommended, kills your thread pool and can block everyone from accessing your site when the speed of input coming is > output)
Drop the request on server side when you are reaching that 10/s limit and have the client retry at a later time. (wait time will be infinite when speed of input coming is > output)
And depending on the content returned by the API you might be able to cache it on server side to avoid having to request it from the 3rd party again.
In this scenario you would need to account for the possibility that you can't process requests in real time. You wouldn't want to have thousands of requests waiting on access to a resource that you don't control.
I second the answer about calling the API from the client, if that's an option.
Another option is to keep a counter of current requests, limit it to ten, and return a 503 error if a request comes in that exceeds that capacity. That's practical if you really don't expect to exceed ten concurrent requests often or ever but want to be sure that in the odd chance that it happens it doesn't shut down this feature of your site.
If you actually expect large volumes where you would exceed ten concurrent requests then you would need to queue the requests, but do it in a process separate from your web application. As mentioned, if you have tons of requests waiting for the same resource your application will become overloaded. You could enqueue the request with an entirely different process, and then the client would have to poll your application with occasional requests to see if there's a response.
The big flaw in this last scenario is that it means your users could end up waiting a long time because your application depends on a finite resource that you cannot scale. You can manage it in a way that keeps your application from failing, but not in a way that makes it respond quickly.
I've got a simple project that use M2mqtt client library to connect to HiveMQ broker. When a message arrives an event will fire, the fact is I can receive up to 100 messages per second and the program is able to processing just 20 messages per second.
HiveMQClient.MqttMsgPublishReceived += HiveMQClient_MqttMsgPublishReceived;
So, I have all the HiveMQ logs and telemetry and i can clearly see that the messages arrives in my application with the right rate (100 per second) but the strange thing is that the CPU of the PC where the client program is hosted runs at 10% of its capacity.
I was wondering if I need to "multi thread the event" or there is something that I'm missing.
Thank you all
EDIT
Inside the MqttMsgPublishReceived event i've got a ThreadPool that stores the messages that i receive inside an azure blob account. After some reviews I understood that this is the problem (thanks #Hans Kilian ).
Now i've got an azure blob storage account, standard configuration) that accepts only 30 calls per second. I tried to update into the premium tier but it is only for virtual machine VHD images.
Anybody knows how to improve these numbers?
The MqttMsgPublishReceived call back runs on the clients network thread, if you are interested in performance then you should not be doing any real work in this callback.
For high performance applications the model is normally to use the MqttMsgPublishReceived method to place the incoming message in a local queue in the client and have a thread pool consuming messages from that queue.
This becomes even more important when using QOS 1 or 2 messages as the broker will not send the next message until the MqttMsgPublishReceived has returned and the QOS handshake completes.
As #HandKilian says in the comments things like databases can also be a bottle neck, but using a thread pool combined with a database connection pool can help as it makes sure you are not building and tearing down a connection to the database for each message.
I have ASP.NET MVC application and I need to send email in "X" minutes(for each user time is different) to user after he leaves the page.
How can I do it?
Http is stateless and the time response is sent execution of page is finished. You need an application that will be sending mail even when website is not accessed by some body for a significant time interval. You can put the mails that need to be send after an interval of time in the database. Another application could be a Windows service that will pool the database after fixed interval of time let's say 30 seconds and send the mails which have reached the send time.
The solution I would choose depends on the needed scale and reliability of the system you're building.
If it's a low scale (i.e. 1 server with not too many users at the same time), non mission-critical system (i.e. it's OK if from time to time some emails are not actually sent, for example if your server crashes), then the solution can be as simple as managing a queue in memory with a thread that would wake periodically to send emails to the users that recently left the page.
If you need to build something that would be very reliable and potentially have to send a very large number of emails in a short time, and if your system has to scale to a lot of machines, then you would want to build a solution based on a queue in some storage, where as many machines as needed would pick items and handle them. An API such as Windows Azure Queue Service can be a good fit for this if you need a really high scale and reliability.
I'm attempting to create a WCF service where several thousand (~10,000) clients can connect via a duplex NetTcpBinding for extended periods of time (weeks, maybe months).
After a bit of reading, it looks like it's better to host in IIS than a custom application or Windows service.
Is using WCF for such a service acceptable, or even possible? If so, where can I expect to run into throttling or performance issues, such as increasing the WCF ListenBacklog & MaxConcurrentConnections?
Thanks!
Why do you need to maintain opened connection for weeks / months? That will introduce a lot of complexity, timeouts handling, error handling, recreating connection, etc. I even doubt that this will work.
Net.tcp connections use transport session which leads to PerSession instancing of WCF service - the single service instance servers all requests and lives for the whole duration of the session (weeks or months in your case) = instance and whole its content is still in the memory. Any interruption or unhandled exception will break the channel and close the session = all session's local data are lost and client must crate new proxy to start a new session again. Also any timeout (default is 20 minutes of inactivity) will close the session. For the last - depending of business logic complexity you can find that if even few hundreds clients needs processing in the same time single server is not able to serve all of them and some clients will timeout (again breaks the session). Allowing load balancing with net.tcp demands load balancing algorithm with sticky sessions (session affinity) and whole architecture becomes even more complicated and fragile. Scalability in net.tcp means that service can be deployed on multiple servers but the whole client session must be handled by single server (if server dies all sessions served by the server die as well).
Hosting in IIS/WAS/AppFabric has several advantages where two of them is health monitoring and process recycling. Health monitoring continuously check that worker process is still alive and can process request - if it doesn't it silently starts new worker process and routes new incoming requests to that process. Process recycling regularly recycles (default setting is after 29 hours) application domain which makes process healthy and reducing memory leaks. The side effect is that both recreating process or application domain will kill all sessions. Once you self host the service you lose all of them so you have to deal with health of your service yourselves.
Edit:
IMHO health status information doesn't have to be send over TCP. That is information that doesn't require all the fancy stuff. If you lose some information it will not affect anything = you can use UDP for health status transfers.
When using TCP you don't need to maintain proxy / session opened just to keep opened the connection. TCP connection is not closed immediately when you close the proxy. It remains opened in a pool for short duration of time and if any other proxy needs connection to the same server it is reused (the default idle timeout in pool should be 2 minutes) - I discussed Net.Tcp transport in WCF in another answer.
I'm not a fan of callbacks - this whole concept in WCF is overused and abused. Keeping 10.000 TCP connection opened for months just in case to be able to send sometimes data back to few PCs sounds ridiculous. If you need to communicate with PC expose the service on the PC and call it when you need to send some commands. Just add functionality which will call the server when the PC starts and when the PC is about to shut down + add transfering monitoring informations.
Anyway 10.000 PCs sending information every minute - this can cause that you will receive 10.000 requests in the same time - it can have the same effect as Denial of service attack. Depending on the processing time your server(s) may not be able to process them and many requests will timeout. You can also think about some message queuing or publish-subscribe protocols. Messages will be passed to a queue or topic and server(s) will process them continuously.