I have two web services that use MemcacheD to store the results of common database queries. Both services run on three load-balanced web servers, as does MemcacheD, and the performance is good. However, every fortnight one of the web service's CPU usage increases to 20% from its more normal 5%. I've found that if I restart Memcache then this sorts the problem out. This particular web service uses MemCacheD more and stores bigger objects
I don't think the issue is with my web service (famous last words I know) since after re-starting Memcache the CPU and memory usage stabilise and remain so for several days.
I think there might be a problem with Memcache either with my usage of it or the way I set it up. If I had to guess I'd say the problem is 'thrashing' in that my web service is having to wait to get an object from Memcache.
Could it be my version of Memcache is too old (I got it from here) or that I'm not giving it enough memory. Each instance is currently allocated 128 MB. Is that too low? Are there any tools I could use to measure Memcache's performance to help?
Edit
I have increased the memory allocated to 500MB and found that only one of the instances of Memcache needs to be re-started. This points the problem not being with Memcache but with the way I'm using one particular object in there. I'm hoping to push a fix to this soon. If that makes a difference I'll post an answer to this question.
I am storing two types of data in MemcacheD: one is pure information similar in nature to 'what is the capital of Switzerland', the other are lists used as in-memory stores. It is the latter that caused the problem. One of the lists was not being cleared and as a result was constantly growing. This is why the re-start appears to work because the list is now empty.
MemcacheD does not automatically remove items that have passed their 'expiry' date. What it will do is remove expired items when it needs to. Therefore this list was never being removed. When MemcacheD stores items it stores them on 'pages'. These pages group together items of a similar size. Therefore there will be one for 16KB items, another for 32KB items and so on. Obviously, with a list that was constantly growing this would also be making MemcacheD work harder than it needed to.
Related
I have a Web API that provides complex statistical/forecast data. I have one endpoint that can take even 20s to complete, so I started looking at caching to boots the performance. My situation is very different from these described in many examples, so I need help.
Long story short, the method returns a batch of forecasts and statistics for item. For single item, it's as quick as 50ms, that's good. But there is also a method (very complex) that needs 2000-3000 items AT ONCE, to calculate different statistics. And this is a problem.
There are probably around 250,000 items in the database, around 200M rows in one table. The good part is: Table only updates ONCE per day and I would need around 1GB of data (around 80M "optimized" rows).
So my idea was, once per day (I know exactly when) the API would query, transform, optimize and put into memory 1GB of data from that table, and during the day, it will be lighting fast.
My question is, is it a good idea? Should I use some external provider (like Memcached or Redis) or just a singleton list with proper locking using semaphores etc?
If Memcache, how can I do it? I don't want to cache this table "as is". It's too big. I need to do some transformation first.
Thanks!
This is a good solution if you are not limited by server RAM, imo. Since it's .Net Core you can try System.Runtime.Caching.MemoryCache
In my application I use a dictionary (supporting adding, removing, updating and lookup) where both keys and values are or can be made serializable (values can possibly be quite large object graphs). I came to a point when the dictionary became so large that holding it completely in memory started to occasionally trigger OutOfMemoryException (sometimes in the dictionary methods, and sometimes in other parts of code).
After an attempt to completely replace the dictionary with a database, performance dropped down to an unacceptable level.
Analysis of the dictionary usage patterns showed that usually a smaller part of values are "hot" (are accessed quite often), and the rest (a larger part) are "cold" (accessed rarely or never). It is difficult to say when a new value is added if it will be hot or cold, moreover, some values may migrate back and forth between hot and cold parts over time.
I think that I need an implementation of a dictionary that is able to flush its cold values to a disk on a low memory event, and then reload some of them on demand and keep them in memory until the next low memory event occurs when their hot/cold status will be re-assessed. Ideally, the implementation should neatly adjust the sizes of its hot and cold parts and the flush interval depending on the memory usage profile in the application to maximize overall performance. Because several instances of a dictionary exist in the application (with different key/value types), I think, they might need to coordinate their workflows.
Could you please suggest how to implement such a dictionary?
Compile for 64 bit, deploy on 64 bit, add memory. Keep it in memory.
Before you grown your own you may alternatively look at WeakReference http://msdn.microsoft.com/en-us/library/ms404247.aspx. It would of course require you to rebuild those objects that were reclaimed but one should hope that those which are reclaimed are not used much. It comes with the caveat that its own guidleines state to avoid using weak references as an automatic solution to memory management problems. Instead, develop an effective caching policy for handling your application's objects.
Of course you can ignore that guideline and effectively work your code to account for it.
You can implement the caching policy and upon expiry save to database, on fetch get and cache. Use a sliding expiry of course since you are concerned with keeping those most used.
Do remember however that most used vs heaviest is a trade off. Losing an object 10 times a day that takes 5 minutes to restore would annoy users much more than losing an object 10000 times which tool just 5ms to restore.
And someone above mentioned the web cache. It does automatic memory management with callbacks as noted, depends if you want to lug that one around in your apps.
And...last but not least, look at a distributed cache. With sharding you can split that big dictionary across a few machines.
Just an idea - never did that and never used System.Runtime.Caching:
Implement a wrapper around MemoryCache which will:
Add items with an eviction callback specified. The callback will place evicted items to the database.
Fetch item from database and put back into MemoryCache if the item is absent in MemoryCache during retrieval.
If you expect a lot of request for items missing both in database and memory, you'll probably need to implement either bloom filter or cache keys for present/missing items also.
I have a similar problem in the past.
The concept you are looking for is a read through cache with a LRU (Least Recently Used or Most Recently Used) queue.
Is it there any LRU implementation of IDictionary?
As you add things to your dictionary keep track of which ones where used least recently, remove them from memory and persist those to disk.
I have an application which query the database for records. The records can be thousands in numbers and this can shoot up the memory of the process and eventually leads up to a crash or slow in response.
The paginated query is a solution for this but the information in the record always keep changing. Hence to give a unique experience, we are forced to show the information available at the time which user make the query.
Employing paging could dynamically update the content on moving from pages to page. I believe a client-side caching could solve this problem.
One way I am finding is to store the results in to disk in XML format and query using LINQ to XML. Are there any proven client side caching mechanism which can work with desktop application (not web)
See some pattern like http://msdn.microsoft.com/en-us/library/ff664753
It talks about the use of the Enterprise Library Caching Application Block that lets developers incorporate a local cache in their applications.
Read also http://www.codeproject.com/Articles/8977/Using-Cache-in-Your-WinForms-Applications
Enterprise Library 5.0 can be found here http://msdn.microsoft.com/en-us/library/ff632023
Memory usage shouldn't really be an issue unless you are letting your cache grow indefinitely. There is little benefit to pre-fetching too many pages the user may never see, or in holding on to pages that the user has not viewed for a long time. Dynamically fetching the next/previous page would keep performance high, but you should clear from the cache pages that have been edited or are older than a certain timespan. Clearing from the cache simply requires discarding all references to the page (e.g. removing it from any lists or dictionaries) and allowing the garbage collector to do its work.
You can also potentially store a WeakReference to your objects and let the garbage collector collect your objects if it needs to, but this gives you less control over what is an isn't cached.
Alternatively there are some very good third party solutions for this, especially if its a grid control. The DevExpress grid controls have an excellent server mode that can handle very large data sets with good performance.
We have an ASP.NET 4.0 application that draws from a database a complex data structure that takes over 12 hours to push into an in memory data structure (that is later stored in HttpRuntime.Cache). The size of the data structure is quickly increasing and we can't continue waiting 12+ hours to get it into memory if the application restarts. This is a major issue if you want to change the web.config or any code in the web application that causes a restart - it means a long wait before the application can be used, and hinders development or updating the deployment.
The data structure MUST be in memory to work at a speed that makes the website usable. In memory databases such as memcache or Redis are slow in comparison to HttpRuntime.Cache, and would not work in our situation (in memory db's have to serialize put/get, plus they can't reference each other, they use keys which are lookups - degrading performance, plus with a large amount of keys the performance goes down quickly). Performance is a must here.
What we would like to do is quickly dump the HttpRuntime.Cache to disk before the application ends (on a restart), and be able to load it back immediately when the application starts again (hopefully within minutes instead of 12+ hours or days).
The in-memory structure is around 50GB.
Is there a solution to this?
In memory databases such as memcache or Redis are slow in comparison to HttpRuntime.Cache
Yes, but they are very fast compared to a 12+ hour spin-up. Personally, I think you're taking the wrong approach here in forcing load of a 50 GB structure. Just a suggestion, but we use HttpRuntime.Cache as part of a multi-tier caching strategy:
local cache is checked etc first
otherwise redis is used as the next tier of cache (which is faster than the underlying data, persistent, and supports a number of app servers) (then local cache is updated)
otherwise, the underlying database is hit (and then both redis and local cache are updated)
The point being, at load we don't require anything in memory - it is filled as it is needed, and from then on it is fast. We also use pub/sub (again courtesy of redis) to ensure cache invalidation is prompt. The net result: it is fast enough when cold, and very fast when warm.
Basically, I would look at anything that avoids needing the 50GB data before you can do anything.
If this data isn't really cache, but is your data, I would look at serialization on a proper object model. I would suggest protobuf-net (I'm biased as the author) as a strong candidate here - very fast and very small output.
Pardon if this is more serverfault vs. stackoverflow. It seems to be on the border.
We have an application that caches a large amount of product data for an e-commerce application using ASP.NET caching. This is a dictionary object with 65K elements, and our calculations put the object's size at ~10GB.
Problem:
The amount of memory the object consumes seems to be far in excess of our 10GB calculation.
BIGGEST CONCERN: We can't seem to use over 60% of the 32GB in the server.
What we've tried so far:
In machine.config/system.web (sf doesn't allow the tags, pardon the formatting):
processModel autoConfig="true" memoryLimit="80"
In web.config/system.web/caching/cache (sf doesn't allow the tags, pardon the formatting):
privateBytesLimit = "20000000000" (and 0, the default of course)
percentagePhysicalMemoryUsedLimit = "90"
Environment:
Windows 2008R2 x64
32GB RAM
IIS7
Nothing seems to allow us to exceed the 60% value.
See screenshot of taskman.
http://www.freeimagehosting.net/image.php?7a42144e03.jpg
A little late but I'm having nearly the same issue. The problem with the memoryLimit setting on processModel is that it just seems to have no effect despite being documented as such.
percentagePhysicalMemoryUsedLimit similarly looks like it should do something but has no effect.
privateBytesLimit="20000000000" does work though. I went and debugged the process and found the CacheMemorySizePressure object and it successfully picked up the value and set it to _memoryLimit. I would double check that.
Another option is setting a Private Memory Usage recycle threshold on the IIS app pool. That should also get picked up and override the default 60% limit.
A third option is using the new MemoryCache class and setting the PhysicalMemoryLimit on it.
Have you considered using a different caching strategy? The in built caching is not all that feature rich and you will struggle to get it to do much more (unless some IIS guru has some clever work about).
We spent a lot of time working on this and gave up. We actually use slimmer objects to store in the cache and get the fuller objects as needed.
When we have needed to contemplate this we investigated Memcached and Velocity but retreated from deploying them just yet. They are more feature rich though.
Also how are you storing the items in the cache through code? Are you putting them in there at application start or after the first request for each? The reason I ask is whether your cache keys are effective and that actually you are populating the cache over and over and not retrieving anything (this may be the case of just one object type). We managed to do this once by appending the time to a date specific cache key for example.