We currently are facing problems due to high amount of cached objects. We cache Data from an ERP system (for an Online Shop) and IIS will refresh the webpage as it reaches the maximum amount of memory and we loose all cashed objects. As this makes the idea of caching a little bit problematic we are searching for a solution to cache the objects with a different solution.
I have found AppFabric from Microsoft as it is already included into our Windows Server licenses to be a pretty neat solution.
How ever I still fear that we will have enormous performance problems when using AppFabric Velocity instead of the MemoryCache Class (our current solution for Caching).
So my question is now, is this a solution for our problem or am I over-thinking here and is the performance of AppFabric fast enough?
Grid Dynamics did a great report on using AppFabric here. While I don't know the numbers for your specific cache operations, the report showed great numbers performance wise for AppFabric. In one test, they wanted to see how the size of the cache impacted the cache operations performance. When just reading the data, it had little to no impact on the cache operations performance. When updating, there was impact on cache operations performance, but not a ridiculous amount. When testing object size and performance, obviously, larger objects lowered the performance (throughput performance here). Overall, the report has solid tests and statistics that show that the performance of AppFabric Cache is excellent.
No, Grid Dynamics does not compare the results to other products, but they do show you what the performance of AppFabric Cache is like in different tests. They have a particularly useful Appendix section that can provide details to help people in different usage scenarios.
As always, using a solution that is not on the same machine as the IIS instance will add a little bit of time to the fetching of session data from the cache, but we are talking a small amount of time.
If I am understanding your situation than there are object caching solutions available that let you cache objects in memory and expire them according to your application logic or when the cache starts filling up.
Appfabric is not a very mature product in this regard especially when talking about an "inproc" cache. You'd need a client cache, which really is a subset of the distributed cache (meaning all the cached objects) that resides "in proc" and is kept synchronized with the distributed cache.
One solution that I'd recommend is to use NCache as a distributed cache and use its clinet caching feature for your ERP objects.
Related
I have an ASP.NET MVC application that runs on IIS 7. It is setup as a web garden and the number of worker processes matches the number of my processors. I tend to experience some heavy load at times and this setup as worked best.
I have implemented some caching using System.Web.Cache. I will occasionally need to invalidate some of items in my cache however I cannot clear the cache across all processes.
Does the .NET 4 System.Runtime.Caching features make this any easier? Here is a similar question but I hoping there is better advice with .NET 4.
Flush HttpRuntime.Cache objects across all worker processes on IIS webserver
System.Web.Cache and System.Runtime.Caching provide almost the same features, it is just a simple memory cache where items in the cache can have an expiration time, dependencies etc...
If you want to run your site on multiple physical machines or in your case you run it as web garden, caching data in any in process cache doesn't make a lot of sense because it would cache it for each process again. This will let the memory consumption grow pretty quickly I guess...
In those scenarios a distributed cache system is the best choice, because all processes can leverage the already cached data...
I worked with 2 pretty popular distributed in memory cache systems, one is memcached which was also mentioned in the your link.
The other one is the app fabric cache, here is a good example of how to use it
Memchached is a "simple" cache, it doesn't care about security and all this stuff in the first place. But it is very easy to implement and there are good .Net clients which are really simple to use, almost exactly as the .Net build in crap.
If you want to encrypt the data transfers of your cache and have all this high secured, you might want to go with app fabric...
We have an ASP.NET 4.0 application that draws from a database a complex data structure that takes over 12 hours to push into an in memory data structure (that is later stored in HttpRuntime.Cache). The size of the data structure is quickly increasing and we can't continue waiting 12+ hours to get it into memory if the application restarts. This is a major issue if you want to change the web.config or any code in the web application that causes a restart - it means a long wait before the application can be used, and hinders development or updating the deployment.
The data structure MUST be in memory to work at a speed that makes the website usable. In memory databases such as memcache or Redis are slow in comparison to HttpRuntime.Cache, and would not work in our situation (in memory db's have to serialize put/get, plus they can't reference each other, they use keys which are lookups - degrading performance, plus with a large amount of keys the performance goes down quickly). Performance is a must here.
What we would like to do is quickly dump the HttpRuntime.Cache to disk before the application ends (on a restart), and be able to load it back immediately when the application starts again (hopefully within minutes instead of 12+ hours or days).
The in-memory structure is around 50GB.
Is there a solution to this?
In memory databases such as memcache or Redis are slow in comparison to HttpRuntime.Cache
Yes, but they are very fast compared to a 12+ hour spin-up. Personally, I think you're taking the wrong approach here in forcing load of a 50 GB structure. Just a suggestion, but we use HttpRuntime.Cache as part of a multi-tier caching strategy:
local cache is checked etc first
otherwise redis is used as the next tier of cache (which is faster than the underlying data, persistent, and supports a number of app servers) (then local cache is updated)
otherwise, the underlying database is hit (and then both redis and local cache are updated)
The point being, at load we don't require anything in memory - it is filled as it is needed, and from then on it is fast. We also use pub/sub (again courtesy of redis) to ensure cache invalidation is prompt. The net result: it is fast enough when cold, and very fast when warm.
Basically, I would look at anything that avoids needing the 50GB data before you can do anything.
If this data isn't really cache, but is your data, I would look at serialization on a proper object model. I would suggest protobuf-net (I'm biased as the author) as a strong candidate here - very fast and very small output.
As I said above, I want to know what are all the disadvantage of using cache? Is that good to use cache in a website?
I don't see any disadvantages to using the cache.
The only disadvantages if you can call them that is incorrect usage.
There are several potential problems when using the cache though:
You will experience increased memory usage if you store objects in memory instead of a database
You may end up storing objects in cache that you don't want there (old objects or dynamic data for instance)
You may cache too much - causing you applications performance to degrade since your cache eats all the servers resources
You may cache too little ending up with a system with increased complexity and no performance gain
You may cache wrong data
And so on. Caching is hard, but used correctly it is a Good Thing.
Cache is a good thing. It will help your site run faster and avoid downloading the same content over and over again. Of course you should avoid caching for dynamically generated pages.
Another problem is with the caching of images or similar resources. If you do use caching for them, it will be tricky to update them when the need arises. You should always choose the caching times properly, making a compromise between faster loading and update efficiency.
What is a good design for caching the results of an expensive search in an ASP.NET system?
Any ideas would be welcomed ... particularly those that don't require inventing a complex infrastructure of our own.
Here are some general requirements related to the problem:
Each search result can produce include from zero to several hundred result records
Each search is relatively expensive and timeconsuming to execute (5-15 seconds at the database)
Results must be paginated before being displayed at the client to avoid information overload for the user
Users expect to be able to sort, filter, and search within the results returned
Users expect to be able to quickly switch between pages in the search results
Users expect to be able to select multiple items (via checkbox) on any number of pages
Users expect relatively snappy performance once a search has finished
I see some possible options for where and how to implement caching:
1. Cache on the server (in session or App cache), use postbacks or Ajax panels to facilitate efficient pagination, sorting, filtering, and searching.
PROS: Easy to implement, decent support from ASP.NET infrastructure
CONS: Very chatty, memory intensive on server, data may be cached longer than necessary; prohibits load balancing practices
2. Cache at the server (as above) but using serializeable structures that are moved out of memory after some period of time to reduce memory pressure on the server
PROS: Efficient use of server memory; ability to scale out using load balancing;
CONS: Limited support from .NET infrastructure; potentially fragile when data structures change; places additional load on the database; significantly more complicated
3. Cache on the client (using JSON or XML serialization), use client-side Javascript to paginate, sort, filter, and select results.
PROS: User experience can approach "rich client" levels; most browsers can handle JSON/XML natively - decent libraries exist for manipulation (e.g. jQuery)
CONS: Initial request may take a long time to download; significant memory footprint on client machines; will require hand-crafted Javascript at some level to implement
4. Cache on the client using a compressed/encoded representation of the data - call back into server to decode when switching pages, sorting, filtering, and searching.
PROS: Minimized memory impact on server; allows state to live as long as client needs it; slightly improved memory usage on client over JSON/XML
CONS: Large data sets moving back and forth between client/server; slower performance (due to network I/O) as compared with pure client-side caching using JSON/XML; much more complicated to implement - limited support from .NET/browser
5. Some alternative caching scheme I haven't considered...
For #1, have you considered using a state server (even SQL server) or a shared cache mechanism? There are plenty of good ones to choose from, and Velocity is getting very mature - will probably RTM soon. A cache invalidation scheme that is based on whether the user creates a new search, hits any other page besides search pagination, and finally a standard timeout (20 minutes) should be pretty successful at weeding your cache down to a minimal size.
References:
SharedCache (FOSS)
NCache ($995/CPU)
StateServer (~$1200/server)
StateMirror ("Enterprise pricing")
Velocity (Free?)
If you are able to wait until March 2010, .NET 4.0 comes with a new System.Caching.CacheProvider, which promises lots of implementations (disk, memory, SQL Server/Velocity as mentioned).
There's a good slideshow of the technology here. However it is a little bit of "roll your own" or a lot of it infact. But there will probably be a lot of closed and open source providers being written for the Provider model when the framework is released.
For the six points you state, a few questions crops up
What is contained in the search results? Just string data or masses of metadata associated with each result?
How big is the set you're searching?
How much memory would you use storing the entire set in RAM? Or atleast having a cache of the most popular 10 to 100 search terms. Also being smart and caching related searches after the first search might be another idea.
5-15 seconds for a result is a long time to wait for a search so I'm assuming it's something akin to an expedia.com search where multiple sources are being queried and lots of information returned.
From my limited experience, the biggest problem with the client-side only caching approach is Internet Explorer 6 or 7. Server only and HTML is my preference with the entire result set in the cache for paging, expiring it after some sensible time period. But you might've tried this already and seen the server's memory getting eaten.
Raising an idea under the "alternative" caching scheme. This doesn't answer your question with a given cache architecture, but rather goes back to your original requirements of your search application.
Even if/when you implement your own cache, it's effectiveness can be less than optimal -- especially as your search index grows in size. Cache hit rates will decrease as your index grows. At a certain inflection point, your search may actually slow down due to resources dedicated to both searching and caching.
Most search sub-systems implement their own internal caching architecture as a means of efficiency in operation. Solr, an open-source search system built on Lucene, maintains its own internal cache to provide for speedy operation. There are other search systems that would work for you, and they take similar strategies to results caching.
I would recommend you consider a separate search architecture if your search index warrants it, as caching in a free-text keyword search basis is a complex operation to effectively implement.
Since you say any ideas are welcome:
We have been using the enterprise library caching fairly successfully for caching result sets from a LINQ result.
http://msdn.microsoft.com/en-us/library/cc467894.aspx
It supports custom cache expiration, so should support most of your needs (with a little bit of custom code) there. It also has quite a few backing stores including encrypted backing stores if privacy of searches is important.
It's pretty fully featured.
My recommendation is a combination of #1 and #3:
Cache the query results on the server.
Make the results available as both a full page and as a JSON view.
Cache each page retrieved dynamically at the client, but send a REQUEST each time the page changes.
Use ETAGs to do client cache invalidation.
Have a look at SharedCache- it makes 1/2 pretty easy and works fine in a load balanced system. Free, open source, and we've been using it for about a year with no issues.
While pondering your options, consider that no user wants to page through data. We force that on them as an artifact of trying to build applications on top of browsers in HTML, which inherently do not scale well. We have invented all sorts of hackery to fake application state on top of this, but it is essentially a broken model.
So, please consider implementing this as an actual rich client in Silverlight or Flash. You will not beat that user experience, and it is simple to cache data much larger than is practical in a regular web page. Depending on the expected user behavior, your overall bandwidth could be optimized because the round trips to the server will get only a tight data set instead of any ASP.NET overhead.
i am really struggling finding the best solution, what is really confusing me is as long as .net framework purges low priority cached items why should i worry about memory ( i know it is a dump question) ?
can i really constrain an ASP.net Caching to use certain amount of RAM, not from IIS cause my site is on shared hosting, also what that affect on caching ?
would static classes make better use for some situations ?
Sadly, the memoryLimit attribute on the processModel configuration element is only allowed to be set at the machine level (in machine.config) - and the way config files work, there's no way to say "this is the maximum memory any one app can have, but they can ask for less if they want".
IF you're finding that things are constantly spooling out of your cache, should they really be in there? You're either not reading them often enough to bother caching them in the first place, or your trying to put too much stuff into cache at once.
Cache spooling can be seen with the built in Performance Monitors. Under ASP.NET Applications and ASP.NET Apps v2.0.50727 you can find things like:
Cache API Trims
Cache API Turnover Rate
Cache Total Trims
Cache Total Turnover Rate
More details can be found on TechNet.