We Keep Coding

What are the downsides to Request throttling using delay (C# .Net 4 Web Server)

We are running a Http Api and want to be able to set a limit to the number of requests a user can do per time unit. When this limit has been reached, we don't want the users to receive errors, such as Http 429. Instead we want to increase the response times. This has the result that the users can continue to work, but slower, and can then choose to upgrade or not upgrade its paying plan. This solution can quite easily be implemented using Thread.sleep (or something similar) for x number of seconds, on all requests of a user that has passed its limit.
We think that in worst case there might be a problem with the number of possible connections for a single server, since as long as we keep delaying the response, we keep a connection open, and therefore limiting the number of possible other connections.
All requests to the Api is running asynchronously. The Server itself is built to be scalable and is running behind a load balancer. We can start up additional servers if necessary.
When searching for this type of throttling, we find very few examples of this way of limiting the users, and the examples we found seemed not concerned at all about connections running out. So we wonder is this not a problem?
Are there any downsides to this that we are missing, or is this a feasible solution? How many connections can we have open simultaneously without starting to get problems? Can our vision be solved in another way, that is without giving errors to the user?

Thread.Sleep() is pretty much the worst possible thing you can do on a web server. It doesn't matter that you are running things asynchronously because that only applies to I/O bound operations and then frees the thread to do more work.
By using a Sleep() command, you will effectively be taking that thread out of commission for the time it sleeps.
ASP.Net App Pools have a limited number of threads available to them, and therefore in the worst case scenario, you will max out the total number of connections to your server at 40-50 (whatever the default is), if all of them are sleeping at once.
Secondly
This opens up a major attack vector in terms of DOS. If I am an attacker, I could easily take out your entire server by spinning up 100 or 1000 connections, all using the same API key. Using this approach, the server will dutifully start putting all the threads to sleep and then it's game over.
UPDATE
So you could use Task.Delay() in order to insert an arbitrary amount of latency in the response. Under the hood it uses a Timer which is much lighter weight than using a thread.
await Task.Delay(numberOfMilliseconds);
However...
This only takes care of one side of the equation. You still have an open connection to your server for the duration of the delay. Because this is a limited resource it still leaves you vulnerable to a DOS attack that wouldn't have normally existed.
This may be an acceptable risk for you, but you should at least be aware of the possibility.

Why not simply add a "Please Wait..." on the client to artificially look like it's processing? Adding artificial delays on server costs you, it leaves connections as well as threads tied up unnecessarily.

Prevent overload on remote system when using Parallel.ForEach()

We've built this app that needs to have some calculations done on a remote machine (actually a MatLab server). We're using web services to connect to the MatLab server and perform the calculations.
In order to speed things up, we've used Parallel.ForEach() in order to have multiple service calls going at the same time. If we're very conservative in setting ParallelOptions.MaxDegreeOfParallelism (DOP) to 4 or something, everything works fine and well.
However, if we let the framework decide on the DOP it will spawn so many threads that it forces the remote machine on its knees and timeouts start occurring ( > 10 minutes ).
How can we solve this issue? What I would LOVE to be able to do is use the response time to throttle the calls. If response time is less than 30 sec, keep adding threads, as soon as it's over 30 sec, use less. Any suggestions?
N.B. Related to the response in this question: https://stackoverflow.com/a/20192692/896697

Simplest way would be to tune for the best number of concurrent requests and hardcode that as you have done so far, however there are some nicer options if you are willing to put in some effort.
You could move from a Parallel.ForEach to using a thread pool. That way as things come back from the remote server you can either manually or programatically tune the number of available threads. reducing/increasing the number of available threads as things slow down/speed up, or even kill them if needed.
You could also do a variant of the above using Tasks which are the newer way of doing parallel/async stuff in .net.
Another option would be to use a timers and/or jobs model to schedule jobs every x milliseconds, which could then be throttled/relaxed as results returned from the server. The easiest way to get started would be using Quartz.Net.

Poorly Ordered Data Over Websockets

I've been working on learning to deliver and display data in "real-time" over websockets. Real-time, in this context, just means that the data is being read from a sensor (camera with image processing) every .03 to 1 seconds. Each data point consists of a time and a value (t,v) which are encoded as doubles (although the time is always an integer in this case, I'm not assuming it will be).
The server side uses Alchemy Websockets implementation (C#) as I found it very easy to understand/modify for my purposes.
I'm leveraging the websockets examples found here and here as well as the examples included with Alchemy.
I'm using HighCharts to display the data in "real-time", but I also have it printing to a div for debug purposes (independent example so they don't interfere with each other).
So much of it already works pretty well, but there's an obvious problem that happens when I send data too quickly (to be clear, sending the data about a point every second or two results in a nice graph which doesn't appear to have any problems - the problems become more pronounced the faster I call the alchemy server "send" function).
The data appears to be coming in in the wrong order resulting in an interesting "mangled" effect.
I'm going to start delving into the packet order contained on the server side buffer (the server is instructed to send a certain number of "historical" points when a new user connects and it is already running - this results in a pronounced problem like the one shown above) as well as the client side receive order by looking at the timestamps.
The error is inconsistent in that each time I reload the page it results in a different "mangled" data set. This makes me suspect the communication over websockets to be responsible or something involving the alchemy server.
I will attach the full code if necessary, but right now it's rather messy so I am more looking for troubleshooting ideas.
I've gathered this is not expected behavior for a web socket as it is built on TCP.
Any suggestions/ideas for things to look at?
Thanks!
Edit: I ran another test to check how many data points were out of order each time I refreshed the page. The numbers are as follows:
1 2 3 25 6 5 10 11 96 2 8
Very inconsistent (never 0). Certainly interesting!
This result was taken by excluding the charting component and only using websockets and an array to store the data.
Update:
I decided I'd start analyzing the order things come in in and it does appear to be randomly receiving out of order points using an identical data set. I implemented an "insert" function which will take into account out of order packets. The result (plus a little theme change) looks pretty good!
Open question remains: Is it expected that a websocket can deliver information out of order? Or is there something wrong with my implementation on the server side (or Alchemy). I will investigate further when I have time.
SOLUTION!
I figured it out! After a lot of testing, I realized that my Connection object (which is responsible for watching a data set for new data and sending it as is appropriate given how the connection is configured) was implemented using a Timer object. This was something I took from an example (normally I just use the Thread object for most things asynchronous).
As the Timer object speeds up, it starts executing asynchronously with it's previous calls to it's Tick function. This means that very occasionally, one call to it's Tick function will happen a little faster than another (due to the latency in the Alchemy Send function). This causes minor out of order problems.
I switched the implementation of the communication loop from a Timer object to a Thread object, thus enforcing synchronization, and the out of order packets went away!

Websockets use TCP, and TCP guarantees that data is delivered in order.
I would hope that the browser fires websocket message events in order as well. In my tests, this seemed to be the case.
Using this simple Node app to test:
var sio = require('socket.io')
, http = require('http')
, express = require('express')
, app = express()
, srv = http.createServer(app)
, io = sio.listen(srv)
, fs = require('fs')
, idx = fs.readFileSync('index.html').toString()
;
app.get('/', function(req, res) {
res.send(idx);
});
var i = 0;
setInterval(function() {
io.sockets.emit('count', i++);
}, 1);
srv.listen(888);
This simply sends websocket messages as fast as it can with a number that's incremented each message. The client:
<script src="/socket.io/socket.io.js"></script>
<script>
var last = 0;
var socket = io.connect('/');
socket.on('count', function(d) {
if (d-1 != last) console.warn('out of order!', last, d);
last = d;
});
</script>
Throws a console warning if it receives a message that contains a number that is not one more than the previous message.
In Chrome and Firefox, I saw zero out-of-order messages.
I also tried blocking for a while in the message received event (for (var i = 0; i < 1000000; i++) { }) to simulate work that would cause messages to queue up. Message events still fired in order.
In other words, it's something else in your setup. Most likely, the Alchemy server is actually sending the messages in a different order.

Do not use an object like Timer which has asynchronous callbacks when the task is synchronous. Use a Thread and run the communication loop that way.

I don't know when the issue was posted. I also have similar problem. I use Alchemy Client send small data, then there is no problem. There are a lot example for chat Service. But when I send some file more than 4 KB (not precisely), the problem take place. I try to find what happened. I wrote a program sent numbers from 0-7000 by Alchemy client, and received from UserContext.DataFrame(onreceive), there will happen that DataFrame.ToString will get extra "\0\0\0\0" on about position 508. Then, after this position, the data ordering will be wrong. I used the 2.2.1 from nuget. And I read version 2.0 on GiHub. The source code is not workable. So, to old and no reference value.

locking call to webservice in ASP.NET to avoid Oracle CRM time per web service call limit

I have a web application using ASP.NET, that is connecting to Oracle CRM as a back end. The ASP.Net uses some business objects to call into the Oracle CRM webservices, and this works fine.
Except, however, Oracle CRM has a limitation where they only allow you to make 20 web service calls per second (or one call per 50mS), and if you exceed this rate a SOAPException is returned "The maximum rate of requests was exceeded. Please try again in X ms."
The traffic to the site has increased recently, so we are now getting a lot of these SOAPExceptions, but as the code that calls the webservice is wrapped up in a business object, I thought I would modify it to ensure that the 50ms limit is never breached.
I use the following code
private static object lock_obj = new object();
lock (lock_obj)
{
call webservice;
System.Threading.Thread.Sleep(50);
}
However, I am still getting some SOAP Exceptions. I did try writing the code using mutexes instead of lock(), but the performance impact proved to be a problem.
Can anyone explain to me why my solution isn't workinf, and perhaps suggest an alternative?

Edit: Moved to answer. Possible due to > 1 IIS worker process. I don't think object locking spans worker processes so subsequent simultaneous threads could be started but I could be wrong
http://hectorcorrea.com/Blog/Log4net-Thread-Safe-but-not-Process-Safe
My suggestion would be an application variable which stores the tick of the last request, then from that you can work out when it's safe to fire the next.

As long as your application is running with only one ASP.NET worker process you should be ok with what you have, but there are a few things to potentially consider.
Are you using a Web Garden? If so this creates multiple worker processes and therefore a lock is only obtained per/process
Are you in a load balanced environment? If so you will need to go to a different method.

OK, it turns out that a compounding issue was that we have a windows service running on the same server that was also calling into some of the same objects every 4 minutes (running on a different process of course). When I turn it off (and having bumped the sleep up to 100 as per Mitchel's suggestion) the problem seems to have gone away almost entirely.
I say almost, because every so often I still get the odd mysterious soapexception, but I think by and large the problem is sorted. I'm still a bit mystified as to how we can get any of these Exceptions, but we will live with it for now.
I think Oracle should publicise this feature of Oracle CRM on Demand a little more widely.

How to prevent my HTTP server from "sleeping"?

I've written a very basic HTTP-based server program that runs in the background on my computer to allow me to automate various tasks from my Android (via HTTP requests in Tasker). This all works fine (barring this problem), except that after more than about 30 minutes of inactivity, the application ends up falling asleep, as though it's being shunted out of memory, and takes a good minute or so to wake up when it receives an HTTP request or when I try to restore the window and interact with the UI.
I'm using System.Net.HttpListener to implement the server (using asynchronous calls to BeginListen). How should I go about keeping it on its toes?

Is it possible that your process memory is being swapped out to disk? Perhaps write a little monitor task that just pings it every 5 minutes? Note that this could also be useful for making sure it's still running - it could email you if it's down, for example :)

What is your OS (Windows XP / Seven / Server 2008 ...) and distribution ? (Home / Pro / SBS...) ?
Try to give hight priority to your process (like realtime).
You can also try to perform some task with your process every 10 minutes...

Maybe you could do another thread and check if your HttpListerner instance IsListening periodically?

Looks like the problem description is missing mentioning of the actual problem itself. On windows applications do not go into sleep mode. Inactive apps might be swapped, but it should not take very long time when it gets a request.
In .Net 2 I have noticed similar behavior due to garbage collection of the Listener Object itself or associated objects due to long inactivity. Try to keep a static reference of the listener object.This might help.