We Keep Coding

Dot Net MemoryCache Eviction

When does dot net's MemoryCache eviction occur? How can I simulate the eviction in console application? Whenever I try to add objects to the memory cache until eviction occurs I get OutofMemoryException instead.

See MemoryCacheElement, that is what controls the default behavior if you don't pass in values to the NameValueCollection config in the constructor or you use the default instance.
Looking through the defaults of the MemoryCacheElement, it checks every two minutes (however it does go faster the closer you are to the high pressure limit). Once inside the timer's callback it will caculate the percentage to trim off of the MemoryCache and then will call MemoryCache.Trim(Int32) using the calculated percentage.
One thing to note in the percentage calculation, if no Gen 2 garbage collections have happened the cache does not attempt to shrink itself.
It is very possible that the way your test console program was running it used up all the memory before a Gen 2 collection could occur or was still in the initial two minute slow mode to check on the memory pressure before it could clear items out.
If you would like to simulate a eviction just call
MemoryCache.Default.Trim(50);
And that will evict half of the cache from the default cache.

MemcacheD causing Web Services to slow down

I have two web services that use MemcacheD to store the results of common database queries. Both services run on three load-balanced web servers, as does MemcacheD, and the performance is good. However, every fortnight one of the web service's CPU usage increases to 20% from its more normal 5%. I've found that if I restart Memcache then this sorts the problem out. This particular web service uses MemCacheD more and stores bigger objects
I don't think the issue is with my web service (famous last words I know) since after re-starting Memcache the CPU and memory usage stabilise and remain so for several days.
I think there might be a problem with Memcache either with my usage of it or the way I set it up. If I had to guess I'd say the problem is 'thrashing' in that my web service is having to wait to get an object from Memcache.
Could it be my version of Memcache is too old (I got it from here) or that I'm not giving it enough memory. Each instance is currently allocated 128 MB. Is that too low? Are there any tools I could use to measure Memcache's performance to help?
Edit
I have increased the memory allocated to 500MB and found that only one of the instances of Memcache needs to be re-started. This points the problem not being with Memcache but with the way I'm using one particular object in there. I'm hoping to push a fix to this soon. If that makes a difference I'll post an answer to this question.

I am storing two types of data in MemcacheD: one is pure information similar in nature to 'what is the capital of Switzerland', the other are lists used as in-memory stores. It is the latter that caused the problem. One of the lists was not being cleared and as a result was constantly growing. This is why the re-start appears to work because the list is now empty.
MemcacheD does not automatically remove items that have passed their 'expiry' date. What it will do is remove expired items when it needs to. Therefore this list was never being removed. When MemcacheD stores items it stores them on 'pages'. These pages group together items of a similar size. Therefore there will be one for 16KB items, another for 32KB items and so on. Obviously, with a list that was constantly growing this would also be making MemcacheD work harder than it needed to.

A dictionary that can save its elements accessed less often to a disk

In my application I use a dictionary (supporting adding, removing, updating and lookup) where both keys and values are or can be made serializable (values can possibly be quite large object graphs). I came to a point when the dictionary became so large that holding it completely in memory started to occasionally trigger OutOfMemoryException (sometimes in the dictionary methods, and sometimes in other parts of code).
After an attempt to completely replace the dictionary with a database, performance dropped down to an unacceptable level.
Analysis of the dictionary usage patterns showed that usually a smaller part of values are "hot" (are accessed quite often), and the rest (a larger part) are "cold" (accessed rarely or never). It is difficult to say when a new value is added if it will be hot or cold, moreover, some values may migrate back and forth between hot and cold parts over time.
I think that I need an implementation of a dictionary that is able to flush its cold values to a disk on a low memory event, and then reload some of them on demand and keep them in memory until the next low memory event occurs when their hot/cold status will be re-assessed. Ideally, the implementation should neatly adjust the sizes of its hot and cold parts and the flush interval depending on the memory usage profile in the application to maximize overall performance. Because several instances of a dictionary exist in the application (with different key/value types), I think, they might need to coordinate their workflows.
Could you please suggest how to implement such a dictionary?

Compile for 64 bit, deploy on 64 bit, add memory. Keep it in memory.
Before you grown your own you may alternatively look at WeakReference http://msdn.microsoft.com/en-us/library/ms404247.aspx. It would of course require you to rebuild those objects that were reclaimed but one should hope that those which are reclaimed are not used much. It comes with the caveat that its own guidleines state to avoid using weak references as an automatic solution to memory management problems. Instead, develop an effective caching policy for handling your application's objects.
Of course you can ignore that guideline and effectively work your code to account for it.
You can implement the caching policy and upon expiry save to database, on fetch get and cache. Use a sliding expiry of course since you are concerned with keeping those most used.
Do remember however that most used vs heaviest is a trade off. Losing an object 10 times a day that takes 5 minutes to restore would annoy users much more than losing an object 10000 times which tool just 5ms to restore.
And someone above mentioned the web cache. It does automatic memory management with callbacks as noted, depends if you want to lug that one around in your apps.
And...last but not least, look at a distributed cache. With sharding you can split that big dictionary across a few machines.

Just an idea - never did that and never used System.Runtime.Caching:
Implement a wrapper around MemoryCache which will:
Add items with an eviction callback specified. The callback will place evicted items to the database.
Fetch item from database and put back into MemoryCache if the item is absent in MemoryCache during retrieval.
If you expect a lot of request for items missing both in database and memory, you'll probably need to implement either bloom filter or cache keys for present/missing items also.

I have a similar problem in the past.
The concept you are looking for is a read through cache with a LRU (Least Recently Used or Most Recently Used) queue.
Is it there any LRU implementation of IDictionary?
As you add things to your dictionary keep track of which ones where used least recently, remove them from memory and persist those to disk.

50GB HttpRuntime.Cache Persistence Possible?

We have an ASP.NET 4.0 application that draws from a database a complex data structure that takes over 12 hours to push into an in memory data structure (that is later stored in HttpRuntime.Cache). The size of the data structure is quickly increasing and we can't continue waiting 12+ hours to get it into memory if the application restarts. This is a major issue if you want to change the web.config or any code in the web application that causes a restart - it means a long wait before the application can be used, and hinders development or updating the deployment.
The data structure MUST be in memory to work at a speed that makes the website usable. In memory databases such as memcache or Redis are slow in comparison to HttpRuntime.Cache, and would not work in our situation (in memory db's have to serialize put/get, plus they can't reference each other, they use keys which are lookups - degrading performance, plus with a large amount of keys the performance goes down quickly). Performance is a must here.
What we would like to do is quickly dump the HttpRuntime.Cache to disk before the application ends (on a restart), and be able to load it back immediately when the application starts again (hopefully within minutes instead of 12+ hours or days).
The in-memory structure is around 50GB.
Is there a solution to this?

In memory databases such as memcache or Redis are slow in comparison to HttpRuntime.Cache
Yes, but they are very fast compared to a 12+ hour spin-up. Personally, I think you're taking the wrong approach here in forcing load of a 50 GB structure. Just a suggestion, but we use HttpRuntime.Cache as part of a multi-tier caching strategy:
local cache is checked etc first
otherwise redis is used as the next tier of cache (which is faster than the underlying data, persistent, and supports a number of app servers) (then local cache is updated)
otherwise, the underlying database is hit (and then both redis and local cache are updated)
The point being, at load we don't require anything in memory - it is filled as it is needed, and from then on it is fast. We also use pub/sub (again courtesy of redis) to ensure cache invalidation is prompt. The net result: it is fast enough when cold, and very fast when warm.
Basically, I would look at anything that avoids needing the 50GB data before you can do anything.
If this data isn't really cache, but is your data, I would look at serialization on a proper object model. I would suggest protobuf-net (I'm biased as the author) as a strong candidate here - very fast and very small output.

IIS6 ASP.NET 2.0 Application Cache - data storage options and performance for large amounts of data

In an ASP.NET 2.0 site on IIS6 I would like to store Key / Value pairs in the Application Cache. Each Key will always be a string with a 5 character length and each Value a string of 15 - 250 characters length.
The usage scenario is that the Cache will be queried once per webpage request, if the Key exists use the Value otherwise query a database and either add a new Key / Value to the Cache or replace an existing entry based upon some application logic.
In this scenario I envisage / require the Cache size to reach circa 1000 entries at which size it will become stable and will rarely (if at all) be changed as described above.
Before I just "performance test it myself" does anyone have any experience of large amounts of Cached data as to whether it is preferable for Performance to:
(1) Use 1 Cache object containing a SortedDictionary<string, string> or
(2) allow the creation of 1,000 Cache objects and the use the Cache itself as a dictionary or
(3) It just doesn't matter for the amount of data in question. In which case would your answer change if the number of entries increased to 10,000 or 100,000?
Many Thanks.

1000 is not a large amount of data; that will work fine, but you will need to think about synchronization if this data is shared between requests. In reality a lock to make access to a Dictionary<string,string> is probably fine, although you can be more fine-grained if you need.
However, the inbuilt web cache (HttpContext.Cache) will also approach this same problem, and has all the thread-safety built in.
Don't use SortedDictionary<,> unless you have a care that the data is sorted. I don't think you do.
As numbers get larger, I'd be more inclined to think about stores such as redis / memcached, with local memory as a local shortcut.