We Keep Coding

Is it safe to replace immutable data structure with Interlocked.Exchange(ref oldValue, newValue) in ASP.NET Core Web-Api

I got an api that's an end point for geographic coordinate requests. That means users can search for specific locations in their area. At the same time new locations can be added. To make the query as fast as possible, I thought I would make the R-tree unchangeable. That is, there are no locks within the R-Tree, since several threads can read at the same time, without race condition. The updates are collected and if e.g. 100 updates are collected, I want to create a new R-Tree and replace the old one. And now my question is how to do this best?
I have a SearchService, which is stored as a single tone and has an R-Tree as private instance.
In my Startup.cs
services.AddSingleton<ISearchService, SearchService>();
ISearchService.cs
public interface ISearchService
{
IEnumerable<GeoLocation> Get(RTreeQuery query);
void Update(IEnumerable<GeoLocation> data);
}
SearchService.cs
public class SearchService : ISearchService
{
private RTree rTree;
public IEnumerable<GeoLocation> Get(RTreeQuery query)
{
return rTree.Get(query);
}
public void Update(IEnumerable<GeoLocation> data)
{
var newTree = new RTree(data);
Interlocked.Exchange<RTree>(ref rTree, newTree);
}
}
My question is, if I exchange the reference with Interlock.Exchange() the operation is atomic and there should be no race condition. But what happens if threads still use the old instance to process their request. Could it be that the garbage collector deletes the old instance when threads still access it? After all, there is no longer a reference to the old instance.
I am relatively new to this topic, so any help is welcome. Thanks for your support!

Read and writes to references are atomic, which means there will be no alignment issues. However, they could be stale.
Section 12.6.6 of the CLI specs
Unless explicit layout control (see Partition II (Controlling Instance
Layout)) is used to alter the default behavior, data elements no
larger than the natural word size (the size of a native int) shall be
properly aligned. Object references shall be treated as though they
are stored in the native word size.
In regards to the GC, your trees are safe from garbage collection while they are running Get.
So in summary, your methods are thread safe as far as reference atomicity go, you can also use the Update method and safely overwrite the reference, there is no need for Interlocked.Exchange. The worst that can happen with your current implementation is you just get a stale tree which you have mentioned is not an issue.

Understanding class level variable usage in Multi-Threading C#

I want to know the consequence of using a class level variable across different functions in a multi-threaded app.
I am creating a class variable and sharing it across get and set function.
The variable is bound to have a value like :
"testuser-2.3" {username-projectversion}
Code:
class Test()
{
private string key;
public Get(string something)
{
key = setToSOmething();
}
public Set(string something)
{
key = setToSOmething();
}
}
Is this code prone to fail in multithreaded environments? Like if two users are accessing diffrent versions of the project, will the "key" value will be diffrent at any random given point?
Thanks in advance.

I want to know the consequence of using a class level variable across different functions in a multi-threaded app.
What you're doing in your code will work, sort of, but it doesn't demonstrate the consequence of allowing multiple threads to modify a variable. It doesn't answer your question. It just means that you'll be okay with the particular thing you're doing with this particular variable.
In this case you're just assigning a different reference to your string variable. That's safe, in a way. You won't get any mangled strings, but it means that you don't know which string a given function will get when it reads the variable. In some scenarios that's not so bad, but it's a little chaotic.
The problem occurs when multiple threads interact with your variable in a way that isn't thread safe.
Here's a really simple test method I wrote without actually knowing what was going to happen:
public class MultithreadedStringTest
{
private string _sharedString;
[TestMethod]
public void DoesntMessUpStrings()
{
var inputStrings = "The quick fox jumped over the lazy brown dog".Split(' ');
var random= new Random();
Parallel.ForEach(Enumerable.Range(0, 1000), x =>
{
_sharedString += " " + inputStrings[random.Next(0, 9)];
});
var outputStrings = _sharedString.Trim().Split(' ');
var nonMangledStrings = outputStrings.Count(s => inputStrings.Contains(s));
Assert.AreEqual(1000, outputStrings.Length,
$"We lost {1000-outputStrings.Length} strings!");
Assert.AreEqual(nonMangledStrings, outputStrings.Length,
$"{outputStrings.Length-nonMangledStrings} strings got mangled.");
}
}
I'm starting with 10 words, and then, in a Parallel.Each loop appending 1000 words selected from those 10 to a single string from concurrent threads.
I expected the string to get mangled. It didn't. Instead what happened is that out of my 1000 words that I added, typically a few hundred just got lost.
We lost 495 strings!
Obviously that's not the particular operation that you're performing. But what it shows is that when we perform concurrent operations, we need to know that we're either calling a thread safe method or we're using some mechanism to prevent conflicts. We want to know how our code will behave and not cross our fingers and hope for the best.
If we're not careful with it the results will be unpredictable and inconsistent. It might work when you test it and fail later, and when it does it will be difficult to diagnose because you won't be able to see it happen.

Leaving aside the fact that Get and Set are both setting key in your code...
This code won't be prone to failure because of the nature of string. It's an immutable class, and the data is basically constructed elsewhere, and then your key assignment happens, as a single, atomic operation. (It's a reference or basically a pointer assignment).
So...even if you were to have two setters, key will always reference a valid string. Depending on the use of this code, though, the order in which the assignments actually happen could be counterintuitive. Say your get actually returns the string...set(get() + "X") could eventually lose Xes if called multiple times from multiple threads. Because all the get calls could get the same old string and perform the string addition on that. But this is you assuming set AND get together are an atomic operation. The accepted answer here:
reference assignment is atomic so why is Interlocked.Exchange(ref Object, Object) needed?
explains this better than I'm doing.
The contrary example would be if you were to use StringBuilder and actually modify the data inside the class...that would not be thread-safe and would certainly require a lock.
Updating answer to explain my reasoning behind my argument that the OP's code is fundamentally thread-safe considered on its own. Consider the following code which attempts to add thread-safety:
public partial class MainWindow : Window
{
private object emergencyLock = new object();
private string _status;
public string status
{
get
{
//make sure if an emergency is happening we get the best possible string.
lock (emergencyLock)
{
return _status;
}
}
set
{
//we could also lock here instead of in GetEmergencyString()..which would fix the get+set=atomic issue
_status = value;
}
}
private string GetEmergencyString()
{
//this function understands an emergency is happening
lock (emergencyLock)
{
//Maybe I'm fetching this string from a database, or building it by hand
//It takes a while...We'll simulate this here
Thread.Sleep(1000);
return "Oh crap, emergency!";
}
}
private void Normal_Button_Click(object sender, RoutedEventArgs e)
{
status = "Nothing much going on.";
}
private void Emergency_Button_Click(object sender, RoutedEventArgs e)
{
//GetEmergencyString() is evaluated first..finally returns a string,
//and THEN the assignment occurs as a single operation
status = GetEmergencyString();
}
}
I'll make the following points about this code:
It does prevent a status seeker from getting a "boring" status during an emergency. It also potentially forces the status seeker to wait a full second before getting that status...Effectively solving nothing, most likely.
Also consider that even single-threaded, there's a fundamental issue here. The fundamental issue is NOT thread safety (in my opinion). The fundamental issue is delay. Better solutions? Fixing the delay. Active notification of the new state..Events, pubsub, etc. A state machine. Even a volatile bool IsEmergency is much better than the "thread-safety" I've added. Any active, intelligent logic in the code. Maybe you don't want the emergency state to be overwritten by the normal state? Again...not a threading issue.

Are Private properties of a class called within a Parallel.Foreach body Thread Safe?

I am tasked with writing a system to process result files created by a different process(which I have no control over) and and trying to modify my code to make use of Parallel.Foreach. The code works fine when just calling a foreach but I have some concerns about thread safety when using the parallel version. The base question I need answered here is "Is the way I am doing this going to guarantee thread safety?" or is this going to cause everything to go sideways on me.
I have tried to make sure all calls are to instances and have removed every static anything except the initial static void Main. It is my current understanding that this will do alot towards assuring thread safety.
I have basically the following, edited for brevity
static void Main(string[] args)
{
MyProcess process = new MyProcess();
process.DoThings();
}
And then in the actual process to do stuff I have
public class MyProcess
{
public void DoThings()
{
//Get some list of things
List<Thing> things = getThings();
Parallel.Foreach(things, item => {
//based on some criteria, take actions from MyActionClass
MyActionClass myAct = new MyActionClass(item);
string tempstring = myAct.DoOneThing();
if(somecondition)
{
MyAct.DoOtherThing();
}
...other similar calls to myAct below here
};
}
}
And over in the MyActionClass I have something like the following:
public class MyActionClass
{
private Thing _thing;
public MyActionClass(Thing item)
{
_thing = item;
}
public string DoOneThing()
{
return _thing.GetSubThings().FirstOrDefault();
}
public void DoOtherThing()
{
_thing.property1 = "Somenewvalue";
}
}
If I can explain this any better I'll try, but I think that's the basics of my needs
EDIT:
Something else I just noticed. If I change the value of a property of the item I'm working with while inside the Parallel.Foreach (in this case, a string value that gets written to a database inside the loop), will that have any affect on the rest of the loop iterations or just the one I'm on? Would it be better to create a new instance of Thing inside the loop to store the item i'm working with in this case?

There is no shared mutable state between actions in the Parallel.ForEach that I can see, so it should be thread-safe, because at most one thread can touch one object at a time.
But as it has been mentioned there is nothing shared that can be seen. It doesn't mean that in the actual code you use everything is as good as it seems here.
Or that nothing will be changed by you or your coworker that will make some state both shared and mutable (in the Thing, for example), and now you start getting difficult to reproduce crashes at best or just plain wrong behaviour at worst that can be left undetected for a long time.
So, perhaps you should try to go fully immutable near threading code?
Perhaps.
Immutability is good, but it is not a silver bullet, and it is not always easy to use and implement, or that every task can be reasonably expressed through immutable objects. And even that accidental "make shared and mutable" change may happen to it as well, though much less likely.
It should at least be considered as a possible option/alternative.
About the EDIT
If I change the value of a property of the item I'm working with while
inside the Parallel.Foreach (in this case, a string value that gets
written to a database inside the loop), will that have any affect on
the rest of the loop iterations or just the one I'm on?
If you change a property and that object is not used anywhere else, and it doesn't rely on some global mutable state (for example, sort of a public static Int32 ChangesCount that increments with each state change), then you should be safe.
a string value that gets written to a database inside the loop - depending on the used data access technology and how you use it, you may be in trouble, because most of them are not designed for multithreaded environment, like EF DbContext, for example. And obviously do not forget that dealing with concurrent access in database is not always easy, though that is a bit away from our original theme.
Would it be better to create a new instance of Thing inside the loop to store the item i'm working with in this case - if there is no risk of external concurrent changes, then it is just an unnecessary work. And if there is a chance of another threads(not Parallel.For) making changes to those objects that are being persisted, then you already have bigger problems than Parallel.For.
Objects should always have observable consistent state (unlike when half of properties set by one thread, and half by another, while you try to persist that who-knows-what), and if they are used by many threads, then they should be already thread-safe - there should be no way to put them into inconsistent state.
And if they want to be persisted by external code, such objects should probably provide:
Either SyncRoot property to synchronize property reading code.
Or some current state snapshot DTO that is created internally by some thread-safe method like ThingSnapshot Thing.GetCurrentData() { lock() {} }.
Or something more exotic.

Memory management / caching for costly objects in C#

Assume that I have the following object
public class MyClass
{
public ReadOnlyDictionary<T, V> Dict
{
get
{
return createDictionary();
}
}
}
Assume that ReadOnlyDictionary is a read-only wrapper around Dictionary<T, V>.
The createDictionary method takes significant time to complete and returned dictionary is relatively large.
Obviously, I want to implement some sort of caching so I could reuse result of createDictionary but also I do not want to abuse garbage collector and use to much memory.
I thought of using WeakReference for the dictionary but not sure if this is best approach.
What would you recommend? How to properly handle result of a costly method that might be called multiple times?
UPDATE:
I am interested in an advice for a C# 2.0 library (single DLL, non-visual). The library might be used in a desktop of a web application.
UPDATE 2:
The question is relevant for read-only objects as well. I changed value of the property from Dictionary to ReadOnlyDictionary.
UPDATE 3:
The T is relatively simple type (string, for example). The V is a custom class. You might assume that an instance of V is costly to create. The dictionary might contain from 0 to couple of thousands elements.
The code assumed to be accessed from a single thread or from multiple threads with an external synchronization mechanism.
I am fine if the dictionary is GC-ed when no one uses it. I am trying to find a balance between time (I want to somehow cache the result of createDictionary) and memory expenses (I do not want to keep memory occupied longer than necessary).

WeakReference is not a good solution for a cache since you object won´t survive the next GC if nobody else is referencing your dictionary. You can make a simple cache by storing the created value in a member variable and reuse it if it is not null.
This is not thread safe and you would end up in some situations creating the dictionary several times if you have heavy concurent access to it. You can use the double checked lock pattern to guard against this with minimal perf impact.
To help you further you would need to specify if concurrent access is an issue for you and how much memory your dictionary does consume and how it is created. If e.g. the dictionary is the result of an expensive query it might help to simply serialize the dictionary to disc and reuse it until you need to recreate it (this depends on your specific needs).
Caching is another word for memory leak if you have no clear policy when your object should be removed from the cache. Since you are trying WeakReference I assume you do not know when exactly a good time would be to clear the cache.
Another option is to compress the dictionary into a less memory hungry structure. How many keys does your dictionary has and what are the values?

There are four major mechanisms available for you (Lazy comes in 4.0, so it is no option)
lazy initialization
virtual proxy
ghost
value holder
each has it own advantages.
i suggest a value holder, which populates the dictionary on the first call of the GetValue
method of the holder. then you can use that value as long as you want to AND it is only
done once AND it is only done when in need.
for more information, see martin fowlers page

Are you sure you need to cache the entire dictionary?
From what you say, it might be better to keep a Most-Recently-Used list of key-value pairs.
If the key is found in the list, just return the value.
If it is not, create the one value (which is supposedly faster than creating all of them, and using less memory too) and store it in the list, thereby removing the key-value pair that hasn't been used the longest.
Here's a very simple MRU list implementation, it might serve as inspiration:
using System.Collections.Generic;
using System.Linq;
internal sealed class MostRecentlyUsedList<T> : IEnumerable<T>
{
private readonly List<T> items;
private readonly int maxCount;
public MostRecentlyUsedList(int maxCount, IEnumerable<T> initialData)
: this(maxCount)
{
this.items.AddRange(initialData.Take(maxCount));
}
public MostRecentlyUsedList(int maxCount)
{
this.maxCount = maxCount;
this.items = new List<T>(maxCount);
}
/// <summary>
/// Adds an item to the top of the most recently used list.
/// </summary>
/// <param name="item">The item to add.</param>
/// <returns><c>true</c> if the list was updated, <c>false</c> otherwise.</returns>
public bool Add(T item)
{
int index = this.items.IndexOf(item);
if (index != 0)
{
// item is not already the first in the list
if (index > 0)
{
// item is in the list, but not in the first position
this.items.RemoveAt(index);
}
else if (this.items.Count >= this.maxCount)
{
// item is not in the list, and the list is full already
this.items.RemoveAt(this.items.Count - 1);
}
this.items.Insert(0, item);
return true;
}
else
{
return false;
}
}
public IEnumerator<T> GetEnumerator()
{
return this.items.GetEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
}
In your case, T is a key-value pair. Keep maxcount small enough, so that searching stays fast, and to avoid excessive memory usage. Call Add each time you use an item.

An application should use WeakReference as a caching mechanism if the useful lifetime of an object's presence in the cache will be comparable to reference lifetime of the object. Suppose, for example, that you have a method which will create a ReadOnlyDictionary based on deserializing a String. If a common usage pattern would be to read a string, create a dictionary, do some stuff with it, abandon it, and start again with another string, WeakReference is probably not ideal. On the other hand, if your objective is to deserialize many strings (quite a few of which will be equal) into ReadOnlyDictionary instances, it may be very useful if repeated attempts to deserialize the same string yield the same instance. Note that the savings would not just come from the fact that one only had to do the work of building the instance once, but also from the facts that (1) it would not be necessary to keep multiple instances in memory, and (2) if ReadOnlyDictionary variables refer to the same instance, they can be known to be equivalent without having to examine the instances themselves. By contrast, determining whether two distinct ReadOnlyDictionary instances were equivalent might require examining all the items in each. Code which would have to do many such comparisons could benefit from using a WeakReference cache so that variables which hold equivalent instances would usually hold the same instance.

I think you have two mechanisms you can rely on for caching, instead of developing your own. The first, as you yourself suggested, was to use a WeakReference, and to let the garbage collector decide when to free this memory up.
You have a second mechanism - memory paging. If the dictionary is created in one swoop, it'll probably be stored in a more or less continuous part of the heap. Just keep the dictionary alive, and let Windows page it out to the swap file if you don't need it. Depending on your usage (how random is your dictionary access), you may end up with better performance than the WeakReference.
This second approach is problematic if you're close to your address space limits (this happens only in 32-bit processes).

How do I clear a System.Runtime.Caching.MemoryCache

I use a System.Runtime.Caching.MemoryCache to hold items which never expire. However, at times I need the ability to clear the entire cache. How do I do that?
I asked a similar question here concerning whether I could enumerate the cache, but that is a bad idea as it needs to be synchronised during enumeration.
I've tried using .Trim(100) but that doesn't work at all.
I've tried getting a list of all the keys via Linq, but then I'm back where I started because evicting items one-by-one can easily lead to race conditions.
I thought to store all the keys, and then issue a .Remove(key) for each one, but there is an implied race condition there too, so I'd need to lock access to the list of keys, and things get messy again.
I then thought that I should be able to call .Dispose() on the entire cache, but I'm not sure if this is the best approach, due to the way it's implemented.
Using ChangeMonitors is not an option for my design, and is unnecassarily complex for such a trivial requirement.
So, how do I completely clear the cache?

I was struggling with this at first. MemoryCache.Default.Trim(100) does not work (as discussed). Trim is a best attempt, so if there are 100 items in the cache, and you call Trim(100) it will remove the ones least used.
Trim returns the count of items removed, and most people expect that to remove all items.
This code removes all items from MemoryCache for me in my xUnit tests with MemoryCache.Default. MemoryCache.Default is the default Region.
foreach (var element in MemoryCache.Default)
{
MemoryCache.Default.Remove(element.Key);
}

You should not call dispose on the Default member of the MemoryCache if you want to be able to use it anymore:
The state of the cache is set to indicate that the cache is disposed.
Any attempt to call public caching methods that change the state of
the cache, such as methods that add, remove, or retrieve cache
entries, might cause unexpected behavior. For example, if you call the
Set method after the cache is disposed, a no-op error occurs. If you
attempt to retrieve items from the cache, the Get method will always
return Nothing.
http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache.dispose.aspx
About the Trim, it's supposed to work:
The Trim property first removes entries that have exceeded either an absolute or sliding expiration. Any callbacks that are registered
for items that are removed will be passed a removed reason of Expired.
If removing expired entries is insufficient to reach the specified trim percentage, additional entries will be removed from the cache
based on a least-recently used (LRU) algorithm until the requested
trim percentage is reached.
But two other users reported it doesnt work on same page so I guess you are stuck with Remove() http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache.trim.aspx
Update
However I see no mention of it being singleton or otherwise unsafe to have multiple instances so you should be able to overwrite your reference.
But if you need to free the memory from the Default instance you will have to clear it manually or destroy it permanently via dispose (rendering it unusable).
Based on your question you could make your own singleton-imposing class returning a Memorycache you may internally dispose at will.. Being the nature of a cache :-)

Here's is what I had made for something I was working on...
public void Flush()
{
List<string> cacheKeys = MemoryCache.Default.Select(kvp => kvp.Key).ToList();
foreach (string cacheKey in cacheKeys)
{
MemoryCache.Default.Remove(cacheKey);
}
}

I know this is an old question but the best option I've come across is to
Dispose the existing MemoryCache and create a new MemoryCache object.
https://stackoverflow.com/a/4183319/880642
The answer doesn't really provide the code to do this in a thread safe way. But this can be achieved using Interlocked.Exchange
var oldCache = Interlocked.Exchange(ref _existingCache, new MemoryCache("newCacheName"));
oldCache.Dispose();
This will swap the existing cache with a new one and allow you to safely call Dispose on the original cache. This avoids needing to enumerate the items in the cache and race conditions caused by disposing a cache while it is in use.
Edit
Here's how I use it in practice accounting for DI
public class CustomCacheProvider : ICustomCacheProvider
{
private IMemoryCache _internalCache;
private readonly ICacheFactory _cacheFactory;
public CustomCacheProvider (ICacheFactory cacheFactory)
{
_cacheFactory = cacheFactory;
_internalCache = _cacheFactory.CreateInstance();
}
public void Set(string key, object item, MemoryCacheEntryOptions policy)
{
_internalCache.Set(key, item, policy);
}
public object Get(string key)
{
return _internalCache.Get(key);
}
// other methods ignored for breviy
public void Dispose()
{
_internalCache?.Dispose();
}
public void EmptyCache()
{
var oldCache = Interlocked.Exchange(ref _internalCache, _cacheFactory.CreateInstance());
oldCache.Dispose();
}
}
The key is controlling access to the internal cache using another singleton which has the ability to create new cache instances using a factory (or manually if you prefer).

The details in #stefan's answer detail the principle; here's how I'd do it.
One should synchronise access to the cache whilst recreating it, to avoid the race condition of client code accessing the cache after it is disposed, but before it is recreated.
To avoid this synchronisation, do this in your adapter class (which wraps the MemoryCache):
public void clearCache() {
var oldCache = TheCache;
TheCache = new MemoryCache("NewCacheName", ...);
oldCache.Dispose();
GC.Collect();
}
This way, TheCache is always in a non-disposed state, and no synchronisation is needed.

I ran into this problem too. .Dispose() did something quite different than what I expected.
Instead, I added a static field to my controller class. I did not use the default cache, to get around this behavior, but created a private one (if you want to call it that). So my implementation looked a bit like this:
public class MyController : Controller
{
static MemoryCache s_cache = new MemoryCache("myCache");
public ActionResult Index()
{
if (conditionThatInvalidatesCache)
{
s_cache = new MemoryCache("myCache");
}
String s = s_cache["key"] as String;
if (s == null)
{
//do work
//add to s_cache["key"]
}
//do whatever next
}
}

Check out this post, and specifically, the answer that Thomas F. Abraham posted.
It has a solution that enables you to clear the entire cache or a named subset.
The key thing here is:
// Cache objects are obligated to remove entry upon change notification.
base.OnChanged(null);
I've implemented this myself, and everything seems to work just fine.