This question already has an answer here:
Closed 10 years ago.
Possible Duplicate:
ASP.NET cache maximum size
I'm caching quite a lot of datatables using asp.net caching (the floowing code):
HttpContext.Current.Cache.Insert(GlobalVars.Current.applicationID + "_" + cacheName, itemToCache, null, System.Web.Caching.Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(240));
However I think that the cache on the server is getting full and having to re-obtain the datatable data from the database. Is there any limit to the amount of data that can be cached on the server or any IIS settings that can be tweaked?
There is a way to upgrade the limit but I would strongly recommend that you use other kind of Caching System (more about this below).
.NET Cache
To know more about the .NET Caching limitation, please read this great answer from a Microsoft .NET Team member.
If you want to see the current limits of .NET Cache, you can try:
var r = new Dictionary<string, string>();
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache % Machine Memory Limit Used", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_MachineMemoryUsed", String.Concat(pc.NextValue().ToString("N1"), "%"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache % Process Memory Limit Used", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_ProcessMemoryUsed", String.Concat(pc.NextValue().ToString("N1"), "%"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Entries", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_Entries", pc.NextValue().ToString("N0"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Misses", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_Misses", pc.NextValue().ToString("N0"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Hit Ratio", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_HitRatio", String.Concat(pc.NextValue().ToString("N1"), "%"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Trims", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_Trims", pc.NextValue().ToString());
}
MemCached
I'm currently using Memcached, and if you're hosting your site somewhere, you can use a paid service like:
http://www.memcachier.com/
Or, if you're using your own server, you can download Couchbase Community Edition and hosted our own.
You will find more questions here about the use of MemCache, such as:
Which .NET Memcached client do you use, EnyimMemcached vs. BeITMemcached?
how to start with memcached
Make room for any Cache system
To use other cache system without changing your code, you could adopt to create an interface, like
public interface ICacheService
{
T Get<T>(string cacheID, Func<T> getItemCallback) where T : class;
void Clear();
}
then is you're using .NET Cache, your implementation would be something like
public class InMemoryCache : ICacheService
{
private int minutes = 15;
public T Get<T>(string cacheID, Func<T> getItemCallback) where T : class
{
T item = HttpRuntime.Cache.Get(cacheID) as T;
if (item == null)
{
item = getItemCallback();
HttpRuntime.Cache.Insert(
cacheID,
item,
null,
DateTime.Now.AddMinutes(minutes),
System.Web.Caching.Cache.NoSlidingExpiration);
}
return item;
}
public void Clear()
{
IDictionaryEnumerator enumerator = HttpRuntime.Cache.GetEnumerator();
while (enumerator.MoveNext())
HttpRuntime.Cache.Remove(enumerator.Key.ToString());
}
}
and you would use it as:
string cacheId = string.Concat("myinfo-", customer_id);
MyInfo model = cacheProvider.Get<MyInfo>(cacheId, () =>
{
MyInfo info = db.GetMyStuff(customer_id);
return info;
});
if you're using Memcached, all you need to do is create a new class that implement ICacheService and select the class you want, either by using IoC or direct call as:
private ICacheService cacheProvider;
protected override void Initialize(System.Web.Routing.RequestContext requestContext)
{
if (cacheProvider == null) cacheProvider = new InMemoryCache();
base.Initialize(requestContext);
}
The cache uses the memory allocation for the worker process. By default the worker process is allowed to get 60 percent of the machine memory in order to do its work .
As per the link, this can be changed to allow more of the machine memory to be used by the worker process by editing the machine.config file. Presumably you have the cache built to already update when it detects that data is out of date, so this should allow you to put more objects into cache.
When inserting an item into the cache add an CacheItemRemovedCallback method.
In the callback log the reason why the item was removed. This way you see if it memory pressure or something else.
public static void OnRemove(string key,
object cacheItem,
System.Web.Caching.CacheItemRemovedReason reason)
{
AppendLog("The cached value with key '" + key +
"' was removed from the cache. Reason: " +
reason.ToString());
}
http://msdn.microsoft.com/en-us/library/aa478965.aspx
Related
I'm trying to configure the write-through and read-through properties of apache Ignite with an Oracle database. I searched in many places like the Ignite oficial documentation, also in the ignite examples on GitHub,
but there isn't much information or examples coded in C# that is the lenguaje in which I'm developing my app.
What I want is to retrieve from a persistent store (in this case an Oracle database), an specific data in the cache (Ignite) that is not already loaded. In a similar way, I need all my changes on the cache to be reflected on the database.
I tied to create and spring.xml with the configuration of the database (ip, port, username, pass, database), but I can't make it work if that is the way it should be done.
Thanks in advance and sorry for my english.
1) Implement ICacheStore interface (or inherit CacheStoreAdapter helper class)
public class OracleStore : CacheStoreAdapter
{
public override object Load(object key)
{
using (var con = new OracleConnection
{
ConnectionString = "User Id=<username>;Password=<password>;Data Source=<datasource>"
})
{
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "SELECT * FROM MyTable WHERE ID=#id";
cmd.Parameters.Add("#id", OracleType.Int32);
cmd.Parameters["#id"].Value = key;
using (var reader = cmd.ExecuteReader())
{
// Read data, return as object
}
}
}
public override void Write(object key, object val)
{
oracleDb.UpdateRow(key, val);
}
public override void Delete(object key)
{
oracleDb.DeleteRow(key);
}
}
2) Implement store factory:
public class OracleStoreFactory : IFactory<OracleStore>
{
public OracleStore CreateInstance()
{
return new OracleStore();
}
}
3) Configure cache to use store:
using (var ignite = Ignition.Start())
{
var cacheCfg = new CacheConfiguration
{
ReadThrough = true,
WriteThrough = true,
KeepBinaryInStore = false, // Depends on your case
CacheStoreFactory = new OracleStoreFactory()
};
var cache = ignite.CreateCache<int, MyClass>(cacheCfg);
cache.Get(1); // OracleStore.Load is called.
}
Documentation for Ignite.NET (in C#): https://apacheignite-net.readme.io/docs/persistent-store
C# examples are available in a full download package: https://ignite.apache.org/download.cgi#binaries (click apache-ignite-fabric-1.9.0-bin.zip, download, unzip, open platforms\dotnet\examples\Apache.Ignite.Examples.sln)
Blog post explaining cache store implementation in C#:
https://ptupitsyn.github.io/Entity-Framework-Cache-Store/
Working with Oracle DB in .NET: Connecting to Oracle Database through C#?
In my Azure ASP.NET MVC website I want to display how many clients are connected to a Redis sessions state provider and how long they are active. I use the aspnet-redis-providers lib on the Azure Github.
In Redis, it creates a {[app_name]_[sessionkey}_Internal key with a SessionTimeout key with the value of the configured session timeout. The EXPIRE for that key is set to that time and when you check to TTL for the key you see the session access.
How can I use the session state provider library to access this information? If that is not possible, is there any other library I can use to query this info safely, without interfering with the session state provider?
Here is what I was able to do. I created my own session object collection and grabbed all keys (which I am putting in DB 1) then I loop through all keys and grab the TTL.
using StackExchange.Redis;
using StackExchange.Redis.Extensions.Newtonsoft;
using StackExchange.Redis.Extensions.Core;
using System.Linq;
private static Lazy<ConnectionMultiplexer> conn = new Lazy<ConnectionMultiplexer>(
() => ConnectionMultiplexer.Connect(ConfigurationManager.AppSettings["RedisServerMaster"]
+ "," + ConfigurationManager.AppSettings["RedisServerSlave"]
+ "," + ConfigurationManager.AppSettings["RedisOptions"])
public class SessionObjects
{
public string SessionId { get; set; }
public TimeSpan? TTL { get; set; }
}
List<SessionObjects> lso = new List<SessionObjects>();
var serializer = new NewtonsoftSerializer();
StackExchangeRedisCacheClient cacheClient;
cacheClient = new StackExchangeRedisCacheClient(rConn, serializer, 1);
IEnumerable<string> keys = cacheClient.SearchKeys("*");
var db = rConn.GetDatabase(1);
foreach (var s in keys)
{
SessionObjects so = new SessionObjects();
so.SessionId = s;
so.TTL = db.KeyTimeToLive(s);
lso.Add(so);
}
I am trying to use the AzureDirectory library to store a Lucene.NET index on a Azure Cloud Storage account.
I am using the following versions:
Microsoft.Windows.Azure.Storage 4.3.0.0
Lucene.Net 3.0.3.0
Lucene.Net.Store.Azure 3.0.5553.21100
And calling the following method:
public void UpdateDocument(Term keyTerm, Document document, string indexName)
{
using (var analyser = new StandardAnalyzer(LuceneVersion))
{
using (var directory = new AzureDirectory(cloudStorage.GetStorageAccount(), indexName, new RAMDirectory()))
{
using (var indexWriter = new IndexWriter(directory, analyser, true, IndexWriter.MaxFieldLength.UNLIMITED))
{
indexWriter.UpdateDocument(keyTerm, document);
}
}
}
}
When I call the method even as little as 10 times (from a unit test) the overall time is around 30 seconds.
I have tried various changes with the index writer to see if any performance gains can be made but so far nothing. I have tried changing the code to reuse the index writer and directory classes but I end up with file locks. I also wanted to keep the index code abstracted away from the caller to keep Lucene isolated. If I comment out indexWriter.UpdateDocument(keyTerm, document); then its responsive which tells me this is where the slowness is.
I would like to know if I am doing something wrong or missing something here?
The method above just needed to be adjusted to work better with the resources as opening the directory and index writer for each document was too costly. My adjusted method works fine:
public void UpdateDocumentBatch(Term keyTerm, IEnumerable<Document> documents, string indexName)
{
using (var analyser = new StandardAnalyzer(LuceneVersion))
{
using (var directory = new AzureDirectory(cloudStorage.GetStorageAccount(), indexName, new RAMDirectory()))
{
var createIndex = !IndexReader.IndexExists(directory);
using (var indexWriter = new IndexWriter(directory, analyser, createIndex, IndexWriter.MaxFieldLength.UNLIMITED))
{
indexWriter.SetRAMBufferSizeMB(100);
foreach (var document in documents)
{
keyTerm.Text = document.GetField(keyTerm.Field).StringValue;
indexWriter.UpdateDocument(keyTerm, document);
}
indexWriter.Commit();
}
}
}
}
I'm using C# to work with AD (Win 2012R2).
We are syncing AD users,groups and their relationship to SQL database.
Full sync works well.
But when using synchronization cookie, the relationship changes does not detected.
What may be the reason?
Thanks.
Here is my code:
public void DirSyncChanges(DirectoryEntry de, byte[] cookie)
{
DirectorySynchronization syncData = new DirectorySynchronization(cookie);
srch = new DirectorySearcher(de)
{
Filter = "(&(objectClass=user)(objectCategory=person))",
SizeLimit = Int32.MaxValue,
Tombstone = true
};
srch.DirectorySynchronization = syncData;
syncData.Option = DirectorySynchronizationOptions.None;
using(SearchResultCollection results = srch.FindAll())
foreach (SearchResult res in results)
{
//results is empty. no loop
}
}
Please specify the DirectorySearcher.PropertiesToLoad. Only if any of the attributes in PropertiesToLoad are updated, you will get them in delta sync.
As i remember the search root of DirSync must be naming context root object.
Better use paged search. No matter how large the value you set to SizeLimit. It will only return at most 1000 or 1500 (forgot exact number) results.
My answer is based on .NET 3.5.
I have a bunch of key/value pairs I'd like to cache for my WPF application. In Silverlight this is deliciously easy - I can just do:
IsolatedStorageSettings userSettings = IsolatedStorageSettings.ApplicationSettings;
wombat = (string)userSettings["marsupial"];
Is there anything like this in WPF? A wombat may not be a marsupial, now I think about it. Some work needed there.
Edit: I would like if I can to avoid serialising these to/from en masse, as there are going to be a very large number of them with large amounts of data in them (I'm caching web pages).
The IsolatedStorageSettings doesn't exist in the desktop version of the .NET Framework, it's only available in Silverlight. However you can use IsolatedStorage in any .NET application; just serialize a Dictionary<string, object> to a file in isolated storage.
var settings = new Dictionary<string, object>();
settings.Add("marsupial", wombat);
BinaryFormatter formatter = new BinaryFormatter();
var store = IsolatedStorageFile.GetUserStoreForAssembly();
// Save
using (var stream = store.OpenFile("settings.cfg", FileMode.OpenOrCreate, FileAccess.Write))
{
formatter.Serialize(stream, settings);
}
// Load
using (var stream = store.OpenFile("settings.cfg", FileMode.OpenOrCreate, FileAccess.Read))
{
settings = (Dictionary<string, object>)formatter.Deserialize(stream);
}
wombat = (string)settings["marsupial"];
If by WPF, you mean the full .Net runtime, then yes. There's a default Settings class created with the WPF project template.
Settings class
See this discussion
It doesn't exist in WPF but can easily be ported from Mono's moonlight implementation (http://vega.frugalware.org/tmpgit/moon/class/System.Windows/System.IO.IsolatedStorage/IsolatedStorageSettings.cs)
//Modifications at MoonLight's IsolatedStorageSettings.cs to make it work with WPF (whether deployed via ClickOnce or not):
// per application, per-computer, per-user
public static IsolatedStorageSettings ApplicationSettings {
get {
if (application_settings == null) {
application_settings = new IsolatedStorageSettings (
(System.Threading.Thread.GetDomain().ActivationContext!=null)?
IsolatedStorageFile.GetUserStoreForApplication() : //for WPF, apps deployed via ClickOnce will have a non-null ActivationContext
IsolatedStorageFile.GetUserStoreForAssembly());
}
return application_settings;
}
}
// per domain, per-computer, per-user
public static IsolatedStorageSettings SiteSettings {
get {
if (site_settings == null) {
site_settings = new IsolatedStorageSettings (
(System.Threading.Thread.GetDomain().ActivationContext!=null)?
IsolatedStorageFile.GetUserStoreForApplication() : //for WPF, apps deployed via ClickOnce will have a non-null ActivationContext
IsolatedStorageFile.GetUserStoreForAssembly());
//IsolatedStorageFile.GetUserStoreForSite() works only for Silverlight applications
}
return site_settings;
}
}
Note that you should also change the #if block at the top of that code to write
if !SILVERLIGHT
Also take a look at this for custom settings storage