We Keep Coding

Using and Maintaining IGroupedObservable

In my Rx code, I'm using GroupBy to create an IObservable<IGroupedObservable<T>>. From there I am looking to perform some transformations on the IGroupedObservable, while maintaining knowledge of the group (Key).
For example,
IObservable<IGroupedObservable<T>> CreateGroup(this IObservable<T> obs)
{
return obs.GroupBy(o => o.Something);
}
IGroupedObservable<A> Foo(this IGroupedObservable<T> obs)
{
return obs.Select(o => new A(o));
}
IGroupedObservable<B> Bar(this IGroupedObservable<A> obs)
{
return obs.Select(o => new B(o));
}
IObservable<IGroupedObservable<B>> stillGrouped = initialObservable.CreateGroup().Select(grouped => grouped.Foo().Bar());
Of course I can't do this, because the Select() in Foo loses the IGroupedObservable-ness.
Has anyone seen any solutions to this?
EDIT TLDR I'm trying to compose operations that depend on an observable already being grouped, and I'd like the type system to enforce this for me. I suppose I could make these operations take the group Key, and perform a Where on that Key up front, but that's messier and slower.

Hmm.
One brute force way is to replicate the observable methods you need. Something like this:
private class GroupedObservable<TKey, TElement> : IGroupedObservable<TKey, TElement>
{
private readonly IObservable<TElement> _o;
private readonly TKey _k;
public TKey Key { get { return _k } }
public GroupedObservable(TKey key, IObservable<TElement> o)
{
_key = key;
_o = ;
}
public IDisposable Subscribe(IObserver<TElement> observer) { return _o.Subscribe(observer); }
}
public static IGroupedObservable<TKey, TResult> Select<TKey, TSource, TResult>(this IGroupedObservable<TKey, TSource> source, Func<TSource, TResult> selector)
{
return new GroupedObservable<TKey, TResult>(source.Key, ((IObservable<TSource>)source).Select(selector));
}
Over time you'll build up a library of the methods you need.
Another simpler way is to just use Select to transform your IGroupedObservable<TKey, TElement> into IObservable<KeyValuePair<TKey, TElement>>:
public static IObservable<KeyValuePair<TKey, TValue>> ToKV<TKey, TValue>(this IGroupedObservable<TKey, TValue> g)
{
return g.Select(v => new KeyValuePair<TKey, TValue>(g.Key, v));
}
initialObservable.CreateGroup().Select(group => group.ToKV());

How to use Dictionary in a delegate

I have a Dictionary which I want to filter by different conditions, e.g.
IDictionary<string, string> result = collection.Where(r => r.Value == null).ToDictionary(r => r.Key, r => r.Value);
I would like to pass the Where clause as a parameter to a method that performs the actual filtering, e.g.
private static IDictionary<T1, T2> Filter<T1, T2>(Func<IDictionary<T1, T2>, IDictionary<T1, T2>> exp, IDictionary<T1, T2> col)
{
return col.Where(exp).ToDictionary<T1, T2>(r => r.Key, r => r.Value);
}
This does not compile, though.
I have tried to call this method by using
Func<IDictionary<string, string>, IDictionary<string, string>> expression = r => r.Value == null;
var result = Filter<string, string>(expression, collection);
What am I doing wrong?

Where wants a Func<TSource, bool>, in your case Func<KeyValuePair<TKey, TValue>, bool>.
Furthermore, your return type of the method is incorrect. It should use T1 and T2 instead of string. Additionally, it is better to use descriptive names for the generic parameters. Instead of T1 and T2 I use the same names as the dictionary - TKey and TValue:
private static IDictionary<TKey, TValue> Filter<TKey, TValue>(
Func<KeyValuePair<TKey, TValue>, bool> exp, IDictionary<TKey, TValue> col)
{
return col.Where(exp).ToDictionary(r => r.Key, r => r.Value);
}

If you look at the constructor for the Where extension method you will see
Func<KeyValuePair<string, string>, bool>
So this is what you need to filter by, try this extension method.
public static class Extensions
{
public static IDictionairy<TKey, TValue> Filter<TKey, TValue>(this IDictionary<TKey, TValue> source, Func<KeyValuePair<TKey, TValue>, bool> filterDelegate)
{
return source.Where(filterDelegate).ToDictionary(x => x.Key, x => x.Value);
}
}
Call as
IDictionary<string, string> dictionairy = new Dictionary<string, string>();
var result = dictionairy.Filter((x => x.Key == "YourValue"));

Why does ConcurrentDictionary.GetOrAdd(key, valueFactory) allow the valueFactory to be invoked twice?

I am using a concurrent dictionary as a thread-safe static cache and noticed the following behavior:
From the MSDN docs on GetOrAdd:
If you call GetOrAdd simultaneously on different threads,
addValueFactory may be called multiple times, but its key/value pair
might not be added to the dictionary for every call.
I would like to be able to guarantee that the factory is only called once. Is there any way to do this with the ConcurrentDictionary API without resorting to my own separate synchronization (e. g. locking inside valueFactory)?
My use case is that valueFactory is generating types inside a dynamic module so if two valueFactories for the same key run concurrently I hit:
System.ArgumentException: Duplicate type name within an assembly.

You could use a dictionary that is typed like this: ConcurrentDictionary<TKey, Lazy<TValue>>, and then the your value factory would return a Lazy<TValue> object that has been initialized with LazyThreadSafetyMode.ExecutionAndPublication, which is the default option used by Lazy<TValue> if you don't specify it. By specifying the LazyThreadSafetyMode.ExecutionAndPublication you are telling Lazy only one thread may initialize and set the value of the object.
This results in the ConcurrentDictionary only using one instance of the Lazy<TValue> object, and the Lazy<TValue> object protects more than one thread from initializing its value.
i.e.
var dict = new ConcurrentDictionary<int, Lazy<Foo>>();
dict.GetOrAdd(key,
(k) => new Lazy<Foo>(valueFactory)
);
The downside then is you'll need to call *.Value every time you are accessing an object in the dictionary. Here are some extensions that'll help with that.
public static class ConcurrentDictionaryExtensions
{
public static TValue GetOrAdd<TKey, TValue>(
this ConcurrentDictionary<TKey, Lazy<TValue>> #this,
TKey key, Func<TKey, TValue> valueFactory
)
{
return #this.GetOrAdd(key,
(k) => new Lazy<TValue>(() => valueFactory(k))
).Value;
}
public static TValue AddOrUpdate<TKey, TValue>(
this ConcurrentDictionary<TKey, Lazy<TValue>> #this,
TKey key, Func<TKey, TValue> addValueFactory,
Func<TKey, TValue, TValue> updateValueFactory
)
{
return #this.AddOrUpdate(key,
(k) => new Lazy<TValue>(() => addValueFactory(k)),
(k, currentValue) => new Lazy<TValue>(
() => updateValueFactory(k, currentValue.Value)
)
).Value;
}
public static bool TryGetValue<TKey, TValue>(
this ConcurrentDictionary<TKey, Lazy<TValue>> #this,
TKey key, out TValue value
)
{
value = default(TValue);
var result = #this.TryGetValue(key, out Lazy<TValue> v);
if (result) value = v.Value;
return result;
}
// this overload may not make sense to use when you want to avoid
// the construction of the value when it isn't needed
public static bool TryAdd<TKey, TValue>(
this ConcurrentDictionary<TKey, Lazy<TValue>> #this,
TKey key, TValue value
)
{
return #this.TryAdd(key, new Lazy<TValue>(() => value));
}
public static bool TryAdd<TKey, TValue>(
this ConcurrentDictionary<TKey, Lazy<TValue>> #this,
TKey key, Func<TKey, TValue> valueFactory
)
{
return #this.TryAdd(key,
new Lazy<TValue>(() => valueFactory(key))
);
}
public static bool TryRemove<TKey, TValue>(
this ConcurrentDictionary<TKey, Lazy<TValue>> #this,
TKey key, out TValue value
)
{
value = default(TValue);
if (#this.TryRemove(key, out Lazy<TValue> v))
{
value = v.Value;
return true;
}
return false;
}
public static bool TryUpdate<TKey, TValue>(
this ConcurrentDictionary<TKey, Lazy<TValue>> #this,
TKey key, Func<TKey, TValue, TValue> updateValueFactory
)
{
if (!#this.TryGetValue(key, out Lazy<TValue> existingValue))
return false;
return #this.TryUpdate(key,
new Lazy<TValue>(
() => updateValueFactory(key, existingValue.Value)
),
existingValue
);
}
}

This is not uncommon with Non-Blocking Algorithms. They essentially test for a condition confirming there is no contention using Interlock.CompareExchange. They loop around though until the CAS succeeds. Have a look at ConcurrentQueue page (4) as a good intro to Non-Blocking Algorithms
Short answer is no, it's the nature of the beast that it will require multiple attempts to add to the collection under contention.
Other than using the other overload of passing a value, you'd need to protect against multiple calls inside your value factory, perhaps using a double lock / memory barrier.

.net dictionary and lookup add / update

I am sick of doing blocks of code like this for various bits of code I have:
if (dict.ContainsKey[key]) {
dict[key] = value;
}
else {
dict.Add(key,value);
}
and for lookups (i.e. key -> list of value)
if (lookup.ContainsKey[key]) {
lookup[key].Add(value);
}
else {
lookup.Add(new List<valuetype>);
lookup[key].Add(value);
}
Is there another collections lib or extension method I should use to do this in one line of code no matter what the key and value types are?
e.g.
dict.AddOrUpdate(key,value)
lookup.AddOrUpdate(key,value)

As Evgeny says, the indexer will already replace existing values - so if you just want to unconditionally set the value for a given key, you can do
dictionary[key] = value;
The more interesting case is the "get a value, or insert it if necessary". It's easy to do with an extension method:
public static TValue GetOrCreateValue<TKey, TValue>
(this IDictionary<TKey, TValue> dictionary,
TKey key,
TValue value)
{
return dictionary.GetOrCreateValue(key, () => value);
}
public static TValue GetOrCreateValue<TKey, TValue>
(this IDictionary<TKey, TValue> dictionary,
TKey key,
Func<TValue> valueProvider)
{
TValue ret;
if (!dictionary.TryGetValue(key, out ret))
{
ret = valueProvider();
dictionary[key] = ret;
}
return ret;
}
Note the use of a delegate to create the default value - that facilitates scenarios like the "list as value" one; you don't want to create the empty list unless you have to:
dict.GetOrCreateValue(key, () => new List<int>()).Add(item);
Also note how this only performs the lookup once if the key is already present - there's no need to do a ContainsKey and then look up the value. It still requires two lookups when it's creating the new value though.

When updating you don't need to perform a check. Simply use:
dict[key] = value
It will replace any existing value. When retrieving the value unfortunately there is no convenient single method (like setdefault in Python), but you could make your own extension method. Something like this:
if (!lookup.TryGetValue(key, out value))
{
value = new List<T>();
lookup.Add(key, value);
}

ConcurrentDictionary in .NET 4.0 has this nice method. You could also write an extension method for this.

If working with .NET Framework 4 or later, you can use the AddOrUpdate Method
dict.AddOrUpdate(key,value)
add or update is like this
dict[key] = value;

I like AddOrUpdate method of ConcurrentDictionary, but I like performance of Dictionary collection too :) So, this is extension method for all classes implementing IDictionary.
public static TValue AddOrUpdate<TKey, TValue>(
this IDictionary<TKey, TValue> dict,
TKey key,
TValue addValue,
Func<TKey, TValue, TValue> updateValueFactory)
{
TValue existing;
if (dict.TryGetValue(key, out existing))
{
addValue = updateValueFactory(key, existing);
dict[key] = addValue;
}
else
{
dict.Add(key, addValue);
}
return addValue;
}
public static TValue AddOrUpdate<TKey, TValue>(
this IDictionary<TKey, TValue> dict,
TKey key,
Func<TKey, TValue> addValueFactory,
Func<TKey, TValue, TValue> updateValueFactory)
{
TValue existing;
if (dict.TryGetValue(key, out existing))
{
existing = updateValueFactory(key, existing);
dict[key] = existing;
}
else
{
existing = addValueFactory(key);
dict.Add(key, existing);
}
return existing;
}

Im not sure if there is a method like you ask for, but you could write a small function for it, or use the try catch exception, presumably if you try adding a value that already exists it will throw an exception. If you catch that and ignore it... Just a suggestion

Automatically creating a collection in the Dictionary<Key, Collection<Value>>

Lot of times I have to create a Dictionary<KeyType, List<ValueType>>
Before I can start using the dictionary I have to first verify that List has been created for that key.
//Can i remove these two lines?
if(!dict.ContainsKey(key))
dict[key]= new List<ValueType>;
//now use the key
dict[key].Add(value);
I know its only "2 lines" of code but it annoys me and I think it can be removed.
I can extend dictionary in someway but before I do it, I want to know if someone has found a clever way to remove the above if statement.
Basically i want to create a Dictionary<KeyType, Collection<ValueType>> and start using it right away like dict[key].Add(value).

You could create something like Google Java Collection's Multimap... or you could just add an extension method like this:
public static void AddValue<TKey, TValue>
(this IDictionary<TKey, List<TValue>> dictionary, TKey key, TValue value)
{
List<TValue> values;
if (!dictionary.TryGetValue(key, out values))
{
values = new List<TValue>();
dictionary.Add(key, values);
}
values.Add(value);
}
As Bevan says, Lookup can help as well - but you can only create one with the ToLookup method, and you can't modify it afterwards. In many cases that's a thoroughly good thing, but if you need a mutable map then you'll ned something like the above.

Have a look at the LookUp class introduced with Linq in .NET 3.5 - it might be just what you're looking for: a Dictionary like class that supports multiple items per key.
Perhaps the only significant downside is that you have to have all your elements available in one batch, as LookUp is immutable.

The ConcurrentDictionary<T,K>.GetOrAdd method is amazingly useful.
private ConcurrentDictionary<string, ICollection<int>> _dictionary;
private static ICollection<int> CreateEmptyList(string dummyKey)
{
return new List<int>();
}
private void AddValue(string key, int value)
{
ICollection<int> values = _dictionary.GetOrAdd(key, CreateEmptyList);
values.Add(value);
}
Edit: Here's an example of how to implement the feature as an extension method for IDictionary<T,K> (C# 3):
Note that IDictionary<TKey, TValue> is generally not thread safe, so if you want thread safety with this extension method you'll have to manually implement it just as for other operations.
public static TValue GetOrAdd<TKey, TValue>(
this IDictionary<TKey, TValue> dictionary,
TKey key,
Func<TKey, TValue> valueFactory)
{
TValue value;
if (!dictionary.TryGetValue(key, out value))
{
value = valueFactory(key);
dictionary.Add(key, value);
}
return value;
}

To add to the answers, you can also add a more general extension which accepts a delegate for instantiation:
public static TValue GetOrCreate<TKey, TValue>
(this IDictionary<TKey, TValue> dict,
TKey key,
Func<TKey, TValue> getValue)
{
TValue value;
if (!dict.TryGetValue(key, out value))
{
dict.Add(key, getValue(key));
}
return value;
}
and then you can provide whatever instantiation method you like:
Dictionary<int, string> dict = new Dictionary<int, string>();
string result = dict.GetOrCreate(5, i => i.ToString());