We are using HttpRuntime.Cache API in an ASP.NET to cache data retrieved from a database.
For this particular application, our database queries feature a LOT of parameters, so our cache keys look something like this:
table=table1;param1=somevalue1;param2=somevalue2;param3=somevalue3;param4=somevalue4;param5=somevalue5;param6=somevalue6... etc...
For some queries, we have so many parameters that the cache key is several hundred characters long.
My question: is there a limit to the length of these cache keys? Internally, it is using a dictionary, so theoretically the lookup time should be constant. However, I wonder if we have potential to run into some performance/memory problem.
Internally, Dictionary uses the hash code of the key you give it. Effectively every key is stored as an integer.
You have nothing to worry about.
Related
Currently I am working on writing a caching system using Redis. Initially, I was storing a dictionary of as the value, and a combination of dates as the key. This worked great, but I was concerned about eventually hitting the size limit for the value. I tried using HashEntries, but it was incredibly inefficient. Now, I am trying storing each MyClass object separately, using a the original key with a the ID appended to it. However, when I am doing the retrieval from the cache, I need to be able to retrieve values where the key contains the date substring. I read that using Keys() is very slow which defeats the purpose of the cache.
I read here about using scan and cursors, but couldn't get the keys from the RedisResult.
I was hoping that somebody either could help me with the scan, show me a way to get the keys that doesn't hurt performance, or another idea for caching large lists of data.
Well, scanning caches is not something that would increase perfomance of application.
What would be suggested is to re-structure your cache to be able to address a single record by date.
Could Redis lists or sets be an option for you?
I need to cache information about user roles in ASP.NET Web API. I have decided to use System.Web.Helpers.WebCache class. Role is plain string, which is about 40 character long. Each user may have between 1-10 roles.
I am thinking of two ways to do this:
Use WebCache.Set(UserID, List<String>). Use user id as key and store List of roles (string) as value. Its easy to retrieve.
Use dictionary, where I will use userId as key and list of roles as value & then cache the dictionary. This way I am caching with only one key. When I retrieve this information, I first retrieve dictionary and then use user id to get the role information.
Questions:
Which approach is better? I like approach one as its easy to use. Does it have any downside?
The way I calculated memory use for keeping these keys into cache is by adding same amount of data (stored 10 roles of type string into) into a notepad and then calculated the size of the notepad (used UTF-8 encoding). The size was about 500 bytes and size of disk was 4 KB . Then if I have 200 users, I will multiply 200 * 500 bytes to calculate the memory usage. Is this right (I am ok if approximately closed) way to calculate?
I prefer the approach of saving individual keys instead of saving the roles of all users as a single cache object.
Following are the reasons:
1) Creation is simple, when user logs in or at an appropriate moment in time, the cache is checked for and 'if empty' created for that user, no need of iterating through the dictionary object (or LINQ) to get to that key item.
2) When user logs off or at an appropriate moment, the cache object is destroyed completely instead of only removing that particular key from cache.
3) Also no need of locking the object when multiple users are trying to access the object at the same time and this scenario will happen. Since object is created per user, there is no risk of locking that object or need to use synchronization or mutex.
Thanks, Praveen
1. Solution one is preferrable. It is straightforward and appears to only offer advantages.
2. Your calculation makes sense for option 1 but not for option 2. A C# dictionary using hashing takes up more memory, for primitive and short data like this, the data taken by hashes may be a significant increase.
The memory storage in individual bytes for this type of application would typically be a secondary concern compared to maintainability and functionality, this is because user roles are often a core functionality with fairly large security concerns and as the project grows it will become very important that the code is maintainable and secure.
Caching should be used exclusively as an optimization and because this is related to small amounts of data for a relatively small user base(~200 people) it would be much better to make your caching of these roles granular and easy to refetch.
According to the official documentation on this library
Microsoft system.web.helpers.webcache
In general, you should never count on an item that you have cached to be in the cache
And because I'll assume that user roles defines some fairly important functionality, it would be better to add queries for these roles to your web API requests instead of storing them locally.
However if you are dead set on using this cache and refetching should it ever disappear then according to your question, option one would be a preferrable choice.
This is because a list takes less memory and in this case appears to be more straight forward and i see no benefits from using a dictionary.
Dictionaries shine when you have large datasets and need speed, but for this scenario where all data is already being stored in memory and the data set is relatively small, a dictionary introduces complexity and higher memory requirements and not much else. Though the memory usage sounds negligible in either scenario on most modern devices and servers.
While a dictionary may sound compelling given your need to lookup roles by users, the WebCache class appears to already offer that funcitonality and thus an additional dictionary loses its appeal
Q1: Without knowing the actual usage of Cache items, it is difficult to draw the conclusion. Nonetheless, I think it all comes down to the design of the life spam for those items. If you want to retire them all in once for certain period and then query a new set of data, storing a ConcurrentDictionary which houses users and roles to WebCache is a easier managing solution to do so.
Otherwise, if you want to retire each entry according to certain event individually, approach one seems a rather straight forward answer. Just be mindful, if you choose approach two, use ConcurrentDictionary instead of Dictionary because the latter is not thread safe.
Q2: WebCache is fundamentally a IEnumerable>, thus it stores the key strings and the memory locations of each value, apart from the meta data of the objects. On the other hand, ConcurrentDictionary/Dictionary stores the hash codes of key strings and the memory locations of each value. While each key's byte[] length is very small, its hashcode could be slightly bigger than the size of the string. Otherwise, the sizes of HashCodes are very predictable and reasonably slim(around 10 bytes in my test). Every time when you add an entry, the size of the whole collection increment about 30 bytes. Of course this figure does not include the actual size of the value as it is irrelevant to the collection.
You can calculate the size of the string by using:
System.Text.Encoding.UTF8.GetByteCount(key);
You might also find it useful to write code to achieve the size of an object:
static long GetSizeOfObject(object obj)
{
using (var stream = new MemoryStream())
{
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, obj);
return stream.Length;
}
}
First of all make sure there will be abstract layer and you can easyly change implementation if future.
I cant see any significant difference between this two approaches both of them use hashtable for search.
But second use search two times I suppose, when it serch dictionary in cache and when it search user in dictionary.
I whold recommend in addition
if users are huge amount , store not roles as strings but roles
Ids. if there are 1000-10000 of no sence to do it
List item
Do not forget
to clear cache record when user roles are updated
You don't need option 2, option 1 should suffice as all you need is key,list<string>.
Few points to consider in general before using caching:-
What is amount of data being cached.
What mode of caching are you using In Memory/Distributed.
How are you going to manage the cache.
If data being cached grows beyond threshold what is the fall over mechanism.
Cache has its pros and cons, In your scenario you have already done the payload analysis so I don't see any issue with option 1.
I am working on an existing system that using NCache. it is a distributed system with large caching requirements, so there is no question that caching is the correct answer, but...
For some reason, in the existing code, all cache keys are hashed before storing in the cache.
My argument is that we should NOT hash the key, as the caching library may have some super optimized way of storing it's dictionary and hashing everything means we may actually be slowing down lookups if we do this.
The guy who originally wrote the code has left, and the knowledge of why the keys are cached has been lost.
Can anyone suggest if hashing is the correct thing to do, or should it be removed.
Okay so your question is
Should we hash the keys before storing?
If you yourself do hashing, will it slow down anything
Well, the cache API works on strings as keys. In the background NCache automatically generates hashes against these keys which help it to identify where the object should be stored. And by where I mean in which node.
When you say that your application Hashes keys before handing it over to NCahe, then it is simple an unnecessary step. NCache API was meant to take this headache from you.
BUT if those hashes were generated because of some internal Logic within your application then that's another case. Please check carefully.
Needless to say, if you're doing something again and again then it will definitely have a performance degradation. The Hash strings that you provide will be used again to generate another hash value (int).
Whether you should or shouldn't hash keys depends on your system requirements.
NCache identifies object by it's key, and considers objects with equal keys to be equal. Below is a definition of a hash function from Wikipedia:
A hash function is any function that can be used to map data of
arbitrary size to data of fixed size.
If you stop hash keys, then cache may behave differently. For example, some objects that NCache considered equal, now NCache may consider not equal. And instead of one cache entry you will get two.
NCache doesn't require you to hash keys. NCache key is just a string that is unique for each object. Relevant excerpt from NCache 4.6 Programmer’s Guide:
NCache uses a “key” and “value” structure for objects. Every object
must have a unique string key associated with it. Every key has an
atomic occurrence in the cache whether it is local or clustered.
Cached keys are case sensitive in nature, and if you try to add
another key with same value, an OperationFailedException is thrown by
the cache.
I have a scenario to optimise how my web app is storing data in the session and retrieving it. I should point out that I'm using SQL Server as my session store.
My scenario is I need to store a list of unique IDs mapped to string values in the user's session for later use. The current code I've inherited is using a List<T> with a custom object but I can already see some kind of dictionary is far better for performance.
I've tested two ideas for alternatives:
Storing a Dictionary<int, string> in the session. When I need to get the strings back, I get the dictionary from the session once and can test each ID on the dictionary object.
Since the session is basically like a dictionary itself, store the string directly in the session using a unique session key e.g. Session["MyString_<id>"] = stringValue". Getting the value out of the session would basically be the inverse operation.
My test results show the following based on the operation I need to do and using 100 strings:
Dictionary - 4552 bytes, 0.1071 seconds to do operation
Session Direct - 4441 bytes, 0.0845 seconds to do operation
From these results I see that I save some space in the session (probably because I've not got the overhead of serialising a dictionary object) and it seems to be faster when getting the values back from the session, maybe because strings are faster to deserialise than objects.
So my question is, is it better for performance to store lots of smaller objects in session rather than one big one? Is there some disadvantage for storing lots of smaller objects vs. one bigger object that I haven't seen?
There are penalties for serializing and searching large objects (they take up more space and processor time due to the need to represent a more complex structure).
And why do 2 searches when you can do only one.
Also, all documentation that deal with caching/storing solutions mention that it is much more efficient to serialize a single value from a list based on a computed key, rather than store all the dictionary and retrieve that and search in it.
I think you have almost answered your own question in showing that that yes, there is an overhead with deserializing objects but I think the real reason should be one of manageability and maintainability.
The size of storage difference is going to be minimal when you are talking about 100 objects but as you scale this up to 1000's of objects the differences will increase too, especially if you are using complex custom objects. If you have an application that has many users all using 1000's of sessions then you can imagine how this is just not scalable.
Also, by having many session objects you are undoubtedly going to have to write more code to handle each varying object. This may not be a vast amount more, but certainly more. This would also potentially make it more difficult for a developer picking up your code to understand you reasoning etc and therefore extend your code.
If you can handle the session in a single barebones format like a IEnumerable or IDictionary then this in my opinion is preferable even if there is a slight overhead involved.
Say I have a collection of Users. Each user has a User_ID, Username and Tenant_ID.
Sometimes I need all Users for a specific Tenant_ID.
Sometimes I need a specific User based on User_ID".
Sometimes I need a User based on Username.
I will always have the Tenant_ID available to use as part of the lookup.
What is the ideal way to implement a cache layer for this data?
What considerations should I make since I'm dealing with a multi-tenant system?
What is the best way to manage all the possible cache keys involved?
Option #1: Store all Users together under a single key of "Tenant_1_Users"
The problem with this is that I will be transferring a lot of unwanted data over the wire. If I just need to find a specific user then I need to do the lookup in code using LINQ or something after retrieving the whole collections.
Option #2: Duplicate the same User object under different keys such as "TenantID_1_UserID_5" and "TenantID_1_Username_Jason".
The problem here is managing all the various Keys and location of User objects. Especially if I need to flush a specific User because it has been updated in the database. I now need to know all the possible places it could be stored. Also uses more memory since the same User can be under different keys.
Related: AppFabric Cache 'Design' - Caching Individual Items or Collections?
The problem is that Azure AppFabric Caching does not support Regions, Tags or Notifications.
For what its worth, I would tend to not want to cache the same object under multiple keys, because if you need to update or delete that record later, you'll have to remember to update/delete all the possible keys that it could have used.
In the past I have added items to the cache individually, keying the cache off of the primary key of the record. Then, where needed, I maintained a dictionary object that gave me a fast lookup to find the primary key(s) based on some other value. When I needed this I would create a custom collection class that wrapped up all of the logic for maintaining the dictionary, looking up records that had expired from the cache, searching for records that were never added to the cache, providing accessors for retrieving items by their various values, etc. This structure made it so that each data item could expire individually without taking the rest of the collection with it, but my dictionaries would still maintain the list of which keys belonged with which secondary values so I could restore an expired item easily no matter which accessor was used to request it.
Can’t say this is necessarily the best way, but it seemed to work well for reducing trips to the data store without bogging down the app server because it had too much crammed in memory. I was working with standard ASP.NET and a SQL backend when I developed this technique but I can't see any reason why it could not be converted over to a cloud environment.
In this case I'd be tempted to do something like:
List<User> users = [list of users]
Dictionary<string,int> userIDLookup = ...
Dictionary<string,int> tennantIDLookup = ...
Dictionary<string,int> usernameLookup = ...
This has the benefits of a single object being cached with multiple lookup dictionaries that provide the index into the List. You could encapsulate this in some nice methods like GetUserFromName(string username)... etc... to make it easier to use in your code. Maybe all put together in a UserCache class so you can instantiated the cache and call the lookup methods on it.