I have to add N (independent) items frequently in redis cache using stackexhange.redis C# each with different expiration time so that there is minimum time at client side and min blocking & cost at server side. Redis server will receive hundreds of get requests per second so I don't want to mess with the get time at all.
I have read the documentation here and answer here. I could not find a single method that does this operation. Considering different options:
Using transaction- This will block any operation at server side. So this should not be right solution
Using batching- This will block any operation at client side till all the batch operation is complete. This should not be right solution.
Using pipelining - This will not block any operation at client side and server side. But it can send multiple requests (packets less than N) may consume more network but may increase memory consumption at client side which may induce latency.
Using fire and forget - This also will not block any operation at client side and server side. But it will send multiple requests (more packets than pipelining) which may consume more network bandwidth but no memory consumption at client side.
Which should be the best approach?
I assumed competing operations means 2 inserts and one get and insert cannot go together though they may be accessing different keys. Am I correct in this otherwise what does it mean?
Redis is single-threaded when it comes to either read or write on a database.
What's the best solution in your case? Who knows, it might depend on a lot of variables and each use case should be analyzed separately to implement the right solution.
Redis MULTI can't be avoided unless you want to corrupt your data if something goes wrong in your application layer. Actually, if you want to avoid many requests to Redis, you should use Lua scripts instead.
In the other hand, the point of Redis is trying to make many operations but be sure that those are as small as possible because of the Redis single-threaded nature. Right, it's blazing fast, unless you execute an operation that takes too much time.
In summary, I wouldn't be too concerned about sending many requests as its an in-memory database and works at the lightspeed. Also, consider the wonders of Redis Cluster (i.e. sharding) to being able to optimize your scenario.
Finally, I would take a look at this Redis tutorial: Redis latency problems troubleshooting
You should add to your list of options, Lua scripting. See EVAL.
Also, consider the data structure that you will use. For example you can use MSET to send multiple values in Redis with one hop.
Related
I have an endpoint which returns the response containing hotels and a flag which shows more results are available, the client needs to call this endpoint recursively till the time the server returns more results flag as false. What is the better way to implement this? Could anyone help me on this?
First Option: Avoid It If Possible
Please try to avoid calls on HTTP APIs so as to avoid network latency.
This is very important if you want to make multiple calls from a client which is supposed to be responsive.
e.g. if you are developing a web application / WPF application and you want user to click on something which triggers 10-20 calls to API, the operation may not complete quickly may result in poor user experience.
If it is a background job, then probably it multiple calls would make more sense.
Second Option: Optimize HTTP Calls From Client
If you still want to make multiple calls over HTTP, then you will have to somehow optimize the code in such a way that at least you avoid the network latency.
For avoiding network latency, you can bring all the data or major chunk of the data in one call on the client side. Then client can iterate over this set of data.
Even if you reduce half of the calls you buy much more time for client processing.
Another Option
You can also try to think if this can be a disconnected operation - client sending just one notification to server and then server performing all iterations.
Client can read status somewhere from database to know if this operation is complete.
That way your client UI would still say responsive and you will be able to offload all heavy processing to Server.
You will have to think and which of these options suits High Level Design of your product/project.
Hope I have given enough food for thoughts (although this may not be solving your issue directly).
We are running a Http Api and want to be able to set a limit to the number of requests a user can do per time unit. When this limit has been reached, we don't want the users to receive errors, such as Http 429. Instead we want to increase the response times. This has the result that the users can continue to work, but slower, and can then choose to upgrade or not upgrade its paying plan. This solution can quite easily be implemented using Thread.sleep (or something similar) for x number of seconds, on all requests of a user that has passed its limit.
We think that in worst case there might be a problem with the number of possible connections for a single server, since as long as we keep delaying the response, we keep a connection open, and therefore limiting the number of possible other connections.
All requests to the Api is running asynchronously. The Server itself is built to be scalable and is running behind a load balancer. We can start up additional servers if necessary.
When searching for this type of throttling, we find very few examples of this way of limiting the users, and the examples we found seemed not concerned at all about connections running out. So we wonder is this not a problem?
Are there any downsides to this that we are missing, or is this a feasible solution? How many connections can we have open simultaneously without starting to get problems? Can our vision be solved in another way, that is without giving errors to the user?
Thread.Sleep() is pretty much the worst possible thing you can do on a web server. It doesn't matter that you are running things asynchronously because that only applies to I/O bound operations and then frees the thread to do more work.
By using a Sleep() command, you will effectively be taking that thread out of commission for the time it sleeps.
ASP.Net App Pools have a limited number of threads available to them, and therefore in the worst case scenario, you will max out the total number of connections to your server at 40-50 (whatever the default is), if all of them are sleeping at once.
Secondly
This opens up a major attack vector in terms of DOS. If I am an attacker, I could easily take out your entire server by spinning up 100 or 1000 connections, all using the same API key. Using this approach, the server will dutifully start putting all the threads to sleep and then it's game over.
UPDATE
So you could use Task.Delay() in order to insert an arbitrary amount of latency in the response. Under the hood it uses a Timer which is much lighter weight than using a thread.
await Task.Delay(numberOfMilliseconds);
However...
This only takes care of one side of the equation. You still have an open connection to your server for the duration of the delay. Because this is a limited resource it still leaves you vulnerable to a DOS attack that wouldn't have normally existed.
This may be an acceptable risk for you, but you should at least be aware of the possibility.
Why not simply add a "Please Wait..." on the client to artificially look like it's processing? Adding artificial delays on server costs you, it leaves connections as well as threads tied up unnecessarily.
In my client-server architecture I have few API functions which usage need to be limited.
Server is written in .net C# and it is running on IIS.
Until now I didn't need to perform any synchronization. Code was written in a way that even if client would send same request multiple times (e.g. create sth request) one call will end with success and all others with error (because of server code + db structure).
What is the best way to perform such limitations? For example I want no more that 1 call of API method: foo() per user per minute.
I thought about some SynchronizationTable which would have just one column unique_text and before computing foo() call I'll write something like foo{userId}{date}{HH:mm} to this table. If call end with success I know that there wasn't foo call from that user in current minute.
I think there is much better way, probably in server code, without using db for that. Of course, there could be thousands of users calling foo.
To clarify what I need: I think it could be some light DictionaryMutex.
For example:
private static DictionaryMutex FooLock = new DictionaryMutex();
FooLock.lock(User.GUID);
try
{
...
}
finally
{
FooLock.unlock(User.GUID);
}
EDIT:
Solution in which one user cannot call foo twice at the same time is also sufficient for me. By "at the same time" I mean that server started to handle second call before returning result for first call.
Note, that keeping this state in memory in an IIS worker process opens the possibility to lose all this data at any instant in time. Worker processes can restart for any number of reasons.
Also, you probably want to have two web servers for high availability. Keeping the state inside of worker processes makes the application no longer clustering-ready. This is often a no-go.
Web apps really should be stateless. Many reasons for that. If you can help it, don't manage your own data structures like suggested in the question and comments.
Depending on how big the call volume is, I'd consider these options:
SQL Server. Your queries are extremely simple and easy to optimize for. Expect 1000s of such queries per seconds per CPU core. This can bear a lot of load. You can use a SQL Express for free.
A specialized store like Redis. Stack Overflow is using Redis as a persistent, clustering-enabled cache. A good idea.
A distributed cache, like Microsoft Velocity. Or others.
This storage problem is rather easy because it fits a key/value store model well. And the data is near worthless so you don't even need to backup.
I think you're overestimating how costly this rate limitation will be. Your web-service is probably doing a lot more costly things than a single UPDATE by primary key to a simple table.
After experimenting with many IMAP API's, I have decided to write my own (Most are memory hogs, some just do not work, out of memory exception etc etc etc).
Anyway I have wrote some code that works (using TCPClient object) and its good so far. However, I want my one to be able to handle multiple requests to the mail server. For example:
Say I get a list of all the UIDs, then I cycle through this list getting what I want (Body, Header, etc) for each message.
The question is, how would I handle, multiple requests at the same time? So instead of looping through the UIDs one at a time I can instead process 10 at a time.
What would be the best approach here? An array of TCP Clients, each with its own thread?
Thanks.
In general it's recommended that IMAP clients only have at most one connection to the server at all times. Not only does additional connections require valuable resources on the server but more important the IMAP specification does not guarantee that two connections can select the same mailbox at the same time. Relying on this capability being present on the server may render your client incompatible with those servers.
Instead you should use the protocol as efficient as possible. Note that many commands can operate on a set or range of UIDs. This allows you to make one singe request where you specify every UID instead of making one request for each UID separately.
Another good practice is to not request more data than what is currently needed. For example say that you have a list of messages. Then don't request detailed information for all of them, only request information for the messages that are currently visible.
I highly recommend that you read RFC 2683, IMAP4 Implementation Recommendations. It covers this among other things.
If you decide to use multiple connections anyway then a good approach is usually to use asynchronous operations and not use individual threads explicitly. Combination with some kind of run loop integration is often useful as well, that way your code is called when there is data to read instead of you code having to poll or explicitly check for it. This is often a good approach even if you're only using a single connection. Keep in mind that according to the IMAP protocol the server may send you responses even when you have not explicitly asked for them.
I have a SL client and a WCF service. The client polls the WCF every 4 seconds and I have almost 100 clients at a time.
The web server is an entry level server with 512 MB RAM.
I want to know, if polling is dependent on the server configuration, if I increase the server configuration will the polling for clients work better?
And second, would pushing (duplex) be better than polling? I have got some mixed response from the blogs I have been reading.
Moreover, what are the best practices in optimizing polling for quicker response at the client? My application needs real-time data
Thanks
My guess would be that you have some kind of race condition that is showing up only with a larger number of clients. What concurrency and instancing modes are you using for your WCF service? (See MSDN: WCF Sessions, Instancing, and Concurrency at http://msdn.microsoft.com/en-us/library/ms731193.aspx)
If you're "losing" responses the first thing I would do is start logging or tracing what's happening at the server. For instance, when a client "doesn't see" a response, is the server ever getting a request? (If so, what happens to it, etc etc.)
I would also keep an eye on memory usage -- you don't say what OS you're using, but 512 MB is awfully skinny these days. If you ever get into a swap-to-disk situation, it's clearly not going to be a good thing.
Lastly, assuming that your service is CPU-bound (i.e. no heavy database & filesystem calls), the best way to raise your throughput is probably to reduce the message payload (wire size), use the most performant bindings (i.e. if client is .NET and you control it, NetTcp binding is much faster than HTTP), and, of course, multithread your service. IMHO, with the info you've provided -- and all other things equal -- polling is probably fine and pushing might just make things more complex. If it's important, you really want to bring a true engineering approach to the problem and identify/measure your bottlenecks.
Hope this helps!
"Push" notifications generally have a lower network overhead, since no traffic is sent when there's nothing to communicate. But "pull" notifications often have a lower application overhead, since you don't have to maintain state when the client is just idling waiting for a notification.
Push notifications also tend to be "faster", since clients are notified immediately when the event happens rather than waiting for the next polling interval. But pull notifications are more flexible -- you can use just about any server or protocol you want, and you can double your client capacity just by doubling your polling wait interval.