How can I route Observable values to different Subscribers? - c#

This is all just pseudo code...
Ok here is my scenario, I have an incoming data stream that gets parsed into packets.
I have an IObservable<Packets> Packets
Each packet has a Packet ID, i.e. 1, 2, 3, 4
I want to create observables that only receive a specific ID.
so I do:
Packets.Where(p=>p.Id == 1)
for example... that gives me an IObservable<Packets> that only gives me packets of Id 1.
I may have several of these:
Packets.Where(p=>p.Id == 2)
Packets.Where(p=>p.Id == 3)
Packets.Where(p=>p.Id == 4)
Packets.Where(p=>p.Id == 5)
This essentially works, but the more Ids I want to select the more processing is required, i.e. the p=>p.Id will be run for every single Id, even after a destination Observable has been found.
How can I do the routing so that it is more efficient, something analogous:
Dictionary listeners;
listeners.GetValue(packet.Id).OnDataReceived(packet)
so that as soon as an id is picked up by one of my IObservables, then none of the others get to see it?
Updates
Added an extension based on Lee Campbell's groupby suggestion:
public static class IObservableExtensions
{
class RouteTable<TKey, TSource>
{
public static readonly ConditionalWeakTable<IObservable<TSource>, IObservable<IGroupedObservable<TKey, TSource>>> s_routes = new ConditionalWeakTable<IObservable<TSource>, IObservable<IGroupedObservable<TKey, TSource>>>();
}
public static IObservable<TSource> Route<TKey, TSource>(this IObservable<TSource> source, Func<TSource, TKey> selector, TKey id)
{
var grouped = RouteTable<TKey, TSource>.s_routes.GetValue(source, s => s.GroupBy(p => selector(p)).Replay().RefCount());
return grouped.Where(e => e.Key.Equals(id)).SelectMany(e => e);
}
}
It would be used like this:
Subject<Packet> packetSubject = new Subject<Packet>();
var packets = packetSubject.AsObservable();
packets.Route((p) => p.Id, 5).Subscribe((p) =>
{
Console.WriteLine("5");
});
packets.Route((p) => p.Id, 4).Subscribe((p) =>
{
Console.WriteLine("4");
});
packets.Route((p) => p.Id, 3).Subscribe((p) =>
{
Console.WriteLine("3");
});
packetSubject.OnNext(new Packet() { Id = 1 });
packetSubject.OnNext(new Packet() { Id = 2 });
packetSubject.OnNext(new Packet() { Id = 3 });
packetSubject.OnNext(new Packet() { Id = 4 });
packetSubject.OnNext(new Packet() { Id = 5 });
packetSubject.OnNext(new Packet() { Id = 4 });
packetSubject.OnNext(new Packet() { Id = 3 });
output is:
3, 4, 5, 4, 3
It only checks the Id for every group when it sees a new packet id.

Here's an operator that I wrote quite some time ago, but I think it does what you're after. I still think that a simple .Where is probably better - even with multiple subscribers.
Nevertheless, I wanted a .ToLookup for observables that operates like the same operator for enumerables.
It isn't memory efficient, but it implements IDisposable so that it can be cleaned up afterwards. It also isn't thread-safe so a little hardening might be required.
Here it is:
public static class ObservableEx
{
public static IObservableLookup<K, V> ToLookup<T, K, V>(this IObservable<T> source, Func<T, K> keySelector, Func<T, V> valueSelector, IScheduler scheduler)
{
return new ObservableLookup<T, K, V>(source, keySelector, valueSelector, scheduler);
}
internal class ObservableLookup<T, K, V> : IDisposable, IObservableLookup<K, V>
{
private IDisposable _subscription = null;
private readonly Dictionary<K, ReplaySubject<V>> _lookups = new Dictionary<K, ReplaySubject<V>>();
internal ObservableLookup(IObservable<T> source, Func<T, K> keySelector, Func<T, V> valueSelector, IScheduler scheduler)
{
_subscription = source.ObserveOn(scheduler).Subscribe(
t => this.GetReplaySubject(keySelector(t)).OnNext(valueSelector(t)),
ex => _lookups.Values.ForEach(rs => rs.OnError(ex)),
() => _lookups.Values.ForEach(rs => rs.OnCompleted()));
}
public void Dispose()
{
if (_subscription != null)
{
_subscription.Dispose();
_subscription = null;
_lookups.Values.ForEach(rs => rs.Dispose());
_lookups.Clear();
}
}
private ReplaySubject<V> GetReplaySubject(K key)
{
if (!_lookups.ContainsKey(key))
{
_lookups.Add(key, new ReplaySubject<V>());
}
return _lookups[key];
}
public IObservable<V> this[K key]
{
get
{
if (_subscription == null) throw new ObjectDisposedException("ObservableLookup");
return this.GetReplaySubject(key).AsObservable();
}
}
}
}
public interface IObservableLookup<K, V> : IDisposable
{
IObservable<V> this[K key] { get; }
}
You would use it like this:
IObservable<Packets> Packets = ...
IObservableLookup<int, Packets> lookup = Packets.ToLookup(p => p.Id, p => p, Scheduler.Default);
lookup[1].Subscribe(p => { });
lookup[2].Subscribe(p => { });
// etc
The nice thing with this is that you can subscribe to values by key before a value with that key has been produced by the source observable.
Don't forget to call lookup.Dispose() when done to clean up the resources.

I would suggest looking at GroupBy and then checking if there is a performance pay off. I assume there is, but is it significant?
Packets.GroupBy(p=>p.Id)
Example code with tests on how to use GroupBy as a type of router
var scheduler = new TestScheduler();
var source = scheduler.CreateColdObservable(
ReactiveTest.OnNext(100, 1),
ReactiveTest.OnNext(200, 2),
ReactiveTest.OnNext(300, 3),
ReactiveTest.OnNext(400, 4),
ReactiveTest.OnNext(500, 5),
ReactiveTest.OnNext(600, 6),
ReactiveTest.OnNext(700, 7),
ReactiveTest.OnNext(800, 8),
ReactiveTest.OnNext(900, 9),
ReactiveTest.OnNext(1000, 10),
ReactiveTest.OnNext(1100, 11)
);
var router = source.GroupBy(i=>i%4)
.Publish()
.RefCount();
var zerosObserver = scheduler.CreateObserver<int>();
router.Where(grp=>grp.Key == 0)
.Take(1)
.SelectMany(grp=>grp)
.Subscribe(zerosObserver);
var onesObserver = scheduler.CreateObserver<int>();
router.Where(grp => grp.Key == 1)
.Take(1)
.SelectMany(grp => grp)
.Subscribe(onesObserver);
var twosObserver = scheduler.CreateObserver<int>();
router.Where(grp => grp.Key == 2)
.Take(1)
.SelectMany(grp => grp)
.Subscribe(twosObserver);
var threesObserver = scheduler.CreateObserver<int>();
router.Where(grp => grp.Key == 3)
.Take(1)
.SelectMany(grp => grp)
.Subscribe(threesObserver);
scheduler.Start();
ReactiveAssert.AreElementsEqual(new[] { ReactiveTest.OnNext(400, 4), ReactiveTest.OnNext(800, 8)}, zerosObserver.Messages);
ReactiveAssert.AreElementsEqual(new[] { ReactiveTest.OnNext(100, 1), ReactiveTest.OnNext(500, 5), ReactiveTest.OnNext(900, 9)}, onesObserver.Messages);
ReactiveAssert.AreElementsEqual(new[] { ReactiveTest.OnNext(200, 2), ReactiveTest.OnNext(600, 6), ReactiveTest.OnNext(1000, 10) }, twosObserver.Messages);
ReactiveAssert.AreElementsEqual(new[] { ReactiveTest.OnNext(300, 3), ReactiveTest.OnNext(700, 7), ReactiveTest.OnNext(1100, 11)}, threesObserver.Messages);

You can use GroupBy to split the data. I would suggest you set up all subscriptions first and then activate your source. Doing so would result in one huge nested GroupBy query, but it is also possible to multi-cast your groups and subscribe to them individually. I wrote a small helper utility to do so below.
Because you still might want to add new routes after the source has been activated (done trough Connect), we use Replay to replay the groups. Replay is also a multi-cast operator so we wont need Publish to multi-cast.
public sealed class RouteData<TKey, TSource>
{
private IConnectableObservable<IGroupedObservable<TKey, TSource>> myRoutes;
public RouteData(IObservable<TSource> source, Func<TSource, TKey> keySelector)
{
this.myRoutes = source.GroupBy(keySelector).Replay();
}
public IDisposable Connect()
{
return this.myRoutes.Connect();
}
public IObservable<TSource> Get(TKey id)
{
return myRoutes.FirstAsync(e => e.Key.Equals(id)).Merge();
}
}
public static class myExtension
{
public static RouteData<TKey, TSource> RouteData<TKey, TSource>(this IObservable<TSource> source, Func<TSource, TKey> keySelector)
{
return new RouteData<TKey, TSource>(source, keySelector);
}
}
Example usage:
public class myPackage
{
public int Id;
public myPackage(int id)
{
this.Id = id;
}
}
class program
{
static void Main()
{
var source = new[] { 0, 1, 2, 3, 4, 5, 4, 3 }.ToObservable().Select(i => new myPackage(i));
var routes = source.RouteData(e => e.Id);
var subscription = new CompositeDisposable(
routes.Get(5).Subscribe(Console.WriteLine),
routes.Get(4).Subscribe(Console.WriteLine),
routes.Get(3).Subscribe(Console.WriteLine),
routes.Connect());
Console.ReadLine();
}
}

You may want to consider writing a custom IObserver that does your bidding. I've included an example below.
void Main()
{
var source = Observable.Range(1, 10);
var switcher = new Switch<int, int>(i => i % 3);
switcher[0] = Observer.Create<int>(val => Console.WriteLine($"{val} Divisible by three"));
source.Subscribe(switcher);
}
class Switch<TKey,TValue> : IObserver<TValue>
{
private readonly IDictionary<TKey, IObserver<TValue>> cases;
private readonly Func<TValue,TKey> idExtractor;
public IObserver<TValue> this[TKey decision]
{
get
{
return cases[decision];
}
set
{
cases[decision] = value;
}
}
public Switch(Func<TValue,TKey> idExtractor)
{
this.cases = new Dictionary<TKey, IObserver<TValue>>();
this.idExtractor = idExtractor;
}
public void OnNext(TValue next)
{
IObserver<TValue> nextCase;
if (cases.TryGetValue(idExtractor(next), out nextCase))
{
nextCase.OnNext(next);
}
}
public void OnError(Exception e)
{
foreach (var successor in cases.Values)
{
successor.OnError(e);
}
}
public void OnCompleted()
{
foreach (var successor in cases.Values)
{
successor.OnCompleted();
}
}
}
You would obviously need to implement idExtractor to extract the ids from your packet.

Related

.NET 6 IntersectBy and ExceptBy examples

Could someone provide me a small example on how to Use the .NET 6 LINQ IntersectBy and ExceptBy methods? MSDN hasn't got any examples and the one I tried doesn't compile due to CS0411 error. The example I tried:
namespace Test
{
internal struct Example
{
public int X { get; set; }
public int Y { get; set; }
public override string ToString()
{
return $"{X}, {Y}";
}
}
public class Program
{
public static void Main()
{
var elements = new List<Example>
{
new Example { X = 10, Y = 20 },
new Example { X = 11, Y = 23 },
};
var elements2 = new List<Example>
{
new Example { X = 10, Y = 12 },
new Example { X = 44, Y = 20 },
};
//ok
var union = elements.UnionBy(elements2, x => x.X);
//CS0411 - Why ?
var intersect = elements.IntersectBy(elements2, x => x.X);
//CS0411 - Why ?
var except = elements.ExceptBy(elements2, x => x.X);
Console.ReadKey();
}
}
}
Granted the documentation doesn't have any examples, it states that the selector function should select TKey i.e. the type of the second collection. The following should work:
var intersect = elements.IntersectBy(elements2, x => x);
var except = elements.ExceptBy(elements2, x => x);
Although I think this may be closer to what you want:
var intersect = elements.IntersectBy(elements2.Select(e => e.X), x => x.X);
For more complex types, you may want to consider implementing an IEqualityComparer and using the overloads that take one as an argument.
I made my own ExceptByProperty method like this
Usage:
var new = items.ExceptByProperty(duplicateItems, x => x.Id).ToList();
Code:
public static class EnumerableExtensions
{
public static IEnumerable<TSource> ExceptByProperty<TSource, TProperty>(this IEnumerable<TSource> first, IEnumerable<TSource> second, Func<TSource, TProperty> keySelector)
{
return first.ExceptBy(second, x => x, GenericComparer<TSource, TProperty>.Comparer(keySelector));
}
}
public sealed class GenericComparer<T, TProperty> : IEqualityComparer<T>
{
public static IEqualityComparer<T> Comparer(Func<T, TProperty> selector)
{
return new GenericComparer<T, TProperty>(selector);
}
private readonly Func<T, TProperty> selector;
public GenericComparer(Func<T, TProperty> selector)
{
this.selector = selector;
}
public bool Equals(T? x, T? y)
{
if (x == null || y == null) return false;
return Equals(selector(x), selector(y));
}
public int GetHashCode([DisallowNull] T obj)
{
object? value = selector(obj);
if (value == null) return obj.GetHashCode();
return value.GetHashCode();
}
}

Linq group by Chunks [duplicate]

Let's take a class called Cls:
public class Cls
{
public int SequenceNumber { get; set; }
public int Value { get; set; }
}
Now, let's populate some collection with following elements:
Sequence
Number Value
======== =====
1 9
2 9
3 15
4 15
5 15
6 30
7 9
What I need to do, is to enumerate over Sequence Numbers and check if the next element has the same value. If yes, values are aggregated and so, desired output is as following:
Sequence Sequence
Number Number
From To Value
======== ======== =====
1 2 9
3 5 15
6 6 30
7 7 9
How can I perform this operation using LINQ query?
You can use Linq's GroupBy in a modified version which groups only if the two items are adjacent, then it's easy as:
var result = classes
.GroupAdjacent(c => c.Value)
.Select(g => new {
SequenceNumFrom = g.Min(c => c.SequenceNumber),
SequenceNumTo = g.Max(c => c.SequenceNumber),
Value = g.Key
});
foreach (var x in result)
Console.WriteLine("SequenceNumFrom:{0} SequenceNumTo:{1} Value:{2}", x.SequenceNumFrom, x.SequenceNumTo, x.Value);
DEMO
Result:
SequenceNumFrom:1 SequenceNumTo:2 Value:9
SequenceNumFrom:3 SequenceNumTo:5 Value:15
SequenceNumFrom:6 SequenceNumTo:6 Value:30
SequenceNumFrom:7 SequenceNumTo:7 Value:9
This is the extension method to to group adjacent items:
public static IEnumerable<IGrouping<TKey, TSource>> GroupAdjacent<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector)
{
TKey last = default(TKey);
bool haveLast = false;
List<TSource> list = new List<TSource>();
foreach (TSource s in source)
{
TKey k = keySelector(s);
if (haveLast)
{
if (!k.Equals(last))
{
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
list = new List<TSource>();
list.Add(s);
last = k;
}
else
{
list.Add(s);
last = k;
}
}
else
{
list.Add(s);
last = k;
haveLast = true;
}
}
if (haveLast)
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
}
}
and the class used:
public class GroupOfAdjacent<TSource, TKey> : IEnumerable<TSource>, IGrouping<TKey, TSource>
{
public TKey Key { get; set; }
private List<TSource> GroupList { get; set; }
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return ((System.Collections.Generic.IEnumerable<TSource>)this).GetEnumerator();
}
System.Collections.Generic.IEnumerator<TSource> System.Collections.Generic.IEnumerable<TSource>.GetEnumerator()
{
foreach (var s in GroupList)
yield return s;
}
public GroupOfAdjacent(List<TSource> source, TKey key)
{
GroupList = source;
Key = key;
}
}
You can use this linq query
Demo
var values = (new[] { 9, 9, 15, 15, 15, 30, 9 }).Select((x, i) => new { x, i });
var query = from v in values
let firstNonValue = values.Where(v2 => v2.i >= v.i && v2.x != v.x).FirstOrDefault()
let grouping = firstNonValue == null ? int.MaxValue : firstNonValue.i
group v by grouping into v
select new
{
From = v.Min(y => y.i) + 1,
To = v.Max(y => y.i) + 1,
Value = v.Min(y => y.x)
};
MoreLinq provides this functionality out of the box
It's called GroupAdjacent and is implemented as extension method on IEnumerable:
Groups the adjacent elements of a sequence according to a specified key selector function.
enumerable.GroupAdjacent(e => e.Key)
There is even a Nuget "source" package that contains only that method, if you don't want to pull in an additional binary Nuget package.
The method returns an IEnumerable<IGrouping<TKey, TValue>>, so its output can be processed in the same way output from GroupBy would be.
You can do it like this:
var all = new [] {
new Cls(1, 9)
, new Cls(2, 9)
, new Cls(3, 15)
, new Cls(4, 15)
, new Cls(5, 15)
, new Cls(6, 30)
, new Cls(7, 9)
};
var f = all.First();
var res = all.Skip(1).Aggregate(
new List<Run> {new Run {From = f.SequenceNumber, To = f.SequenceNumber, Value = f.Value} }
, (p, v) => {
if (v.Value == p.Last().Value) {
p.Last().To = v.SequenceNumber;
} else {
p.Add(new Run {From = v.SequenceNumber, To = v.SequenceNumber, Value = v.Value});
}
return p;
});
foreach (var r in res) {
Console.WriteLine("{0} - {1} : {2}", r.From, r.To, r.Value);
}
The idea is to use Aggregate creatively: starting with a list consisting of a single Run, examine the content of the list we've got so far at each stage of aggregation (the if statement in the lambda). Depending on the last value, either continue the old run, or start a new one.
Here is a demo on ideone.
I was able to accomplish it by creating a custom extension method.
static class Extensions {
internal static IEnumerable<Tuple<int, int, int>> GroupAdj(this IEnumerable<Cls> enumerable) {
Cls start = null;
Cls end = null;
int value = Int32.MinValue;
foreach (Cls cls in enumerable) {
if (start == null) {
start = cls;
end = cls;
continue;
}
if (start.Value == cls.Value) {
end = cls;
continue;
}
yield return Tuple.Create(start.SequenceNumber, end.SequenceNumber, start.Value);
start = cls;
end = cls;
}
yield return Tuple.Create(start.SequenceNumber, end.SequenceNumber, start.Value);
}
}
Here's the implementation:
static void Main() {
List<Cls> items = new List<Cls> {
new Cls { SequenceNumber = 1, Value = 9 },
new Cls { SequenceNumber = 2, Value = 9 },
new Cls { SequenceNumber = 3, Value = 15 },
new Cls { SequenceNumber = 4, Value = 15 },
new Cls { SequenceNumber = 5, Value = 15 },
new Cls { SequenceNumber = 6, Value = 30 },
new Cls { SequenceNumber = 7, Value = 9 }
};
Console.WriteLine("From To Value");
Console.WriteLine("===== ===== =====");
foreach (var item in items.OrderBy(i => i.SequenceNumber).GroupAdj()) {
Console.WriteLine("{0,-5} {1,-5} {2,-5}", item.Item1, item.Item2, item.Item3);
}
}
And the expected output:
From To Value
===== ===== =====
1 2 9
3 5 15
6 6 30
7 7 9
Here is an implementation without any helper methods:
var grp = 0;
var results =
from i
in
input.Zip(
input.Skip(1).Concat(new [] {input.Last ()}),
(n1, n2) => Tuple.Create(
n1, (n2.Value == n1.Value) ? grp : grp++
)
)
group i by i.Item2 into gp
select new {SequenceNumFrom = gp.Min(x => x.Item1.SequenceNumber),SequenceNumTo = gp.Max(x => x.Item1.SequenceNumber), Value = gp.Min(x => x.Item1.Value)};
The idea is:
Keep track of your own grouping indicator, grp.
Join each item of the collection to the next item in the collection (via Skip(1) and Zip).
If the Values match, they are in the same group; otherwise, increment grp to signal the start of the next group.
Untested dark magic follows. The imperative version seems like it would be easier in this case.
IEnumerable<Cls> data = ...;
var query = data
.GroupBy(x => x.Value)
.Select(g => new
{
Value = g.Key,
Sequences = g
.OrderBy(x => x.SequenceNumber)
.Select((x,i) => new
{
x.SequenceNumber,
OffsetSequenceNumber = x.SequenceNumber - i
})
.GroupBy(x => x.OffsetSequenceNumber)
.Select(g => g
.Select(x => x.SequenceNumber)
.OrderBy(x => x)
.ToList())
.ToList()
})
.SelectMany(x => x.Sequences
.Select(s => new { First = s.First(), Last = s.Last(), x.Value }))
.OrderBy(x => x.First)
.ToList();
Let me propose another option, which yields lazily both sequence of groups and
elements inside groups.
Demonstration in .NET Fiddle
Implementation:
public static class EnumerableExtensions
{
public static IEnumerable<IGrouping<TKey, TSource>> GroupAdjacent<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector,
IEqualityComparer<TKey>? comparer = null)
{
var comparerOrDefault = comparer ?? EqualityComparer<TKey>.Default;
using var iterator = new Iterator<TSource>(source.GetEnumerator());
iterator.MoveNext();
while (iterator.HasCurrent)
{
var key = keySelector(iterator.Current);
var elements = YieldAdjacentElements(iterator, key, keySelector, comparerOrDefault);
yield return new Grouping<TKey, TSource>(key, elements);
while (iterator.HasCurrentWithKey(key, keySelector, comparerOrDefault))
{
iterator.MoveNext();
}
}
}
static IEnumerable<TSource> YieldAdjacentElements<TKey, TSource>(
Iterator<TSource> iterator,
TKey key,
Func<TSource, TKey> keySelector,
IEqualityComparer<TKey> comparer)
{
while (iterator.HasCurrentWithKey(key, keySelector, comparer))
{
yield return iterator.Current;
iterator.MoveNext();
}
}
private static bool HasCurrentWithKey<TKey, TSource>(
this Iterator<TSource> iterator,
TKey key,
Func<TSource, TKey> keySelector,
IEqualityComparer<TKey> comparer) =>
iterator.HasCurrent && comparer.Equals(keySelector(iterator.Current), key);
private sealed class Grouping<TKey, TElement> : IGrouping<TKey, TElement>
{
public Grouping(TKey key, IEnumerable<TElement> elements)
{
Key = key;
Elements = elements;
}
public TKey Key { get; }
public IEnumerable<TElement> Elements { get; }
public IEnumerator<TElement> GetEnumerator() => Elements.GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => Elements.GetEnumerator();
}
private sealed class Iterator<T> : IDisposable
{
private readonly IEnumerator<T> _enumerator;
public Iterator(IEnumerator<T> enumerator)
{
_enumerator = enumerator;
}
public bool HasCurrent { get; private set; }
public T Current => _enumerator.Current;
public void MoveNext()
{
HasCurrent = _enumerator.MoveNext();
}
public void Dispose()
{
_enumerator.Dispose();
}
}
}
Notice, that it is impossible to achieve such level of laziness with regular GroupBy operation, since it needs to look through the whole collection before yielding the first group.
Particularly, in my case migration from GroupBy to GroupAdjacent in connection with lazy handling of whole pipeline helped to resolve memory consumption issues for large sequences.
In general, GroupAdjacent can be used as lazy and more efficient alternative of GroupBy, provided that input collection satisfies condition, that keys are sorted (or at least not fragmented), and provided that all operations in pipeline are lazy.

Reactive extensions C# use Catch with anonymous source type

Oversimplified example:
[TestMethod]
public void ReactiveCatch() {
var source = new[] { new { i = 1 }, new { i = 0 } }.ToObservable();
var a = source
.Do(x => { var j = 1 / x.i; })
.Catch((Exception exc) => Observable.Empty(new { i = 0 }))
.ToEnumerable()
.ToArray();
Assert.AreEqual(new { i = 1 }, a[0]);
}
Is there a way to provide a generic handler to the Catch method, which will return an empty default IObservable?
Came up with this solution after carefully reading the comment.
static class Mixin {
public static IObservable<T> CatchAndStop<T>(this IObservable<T> source)
=> source.Catch((Exception exc) => Observable.Empty<T>());
}
[TestClass]
public class ReactiveExtensions {
[TestMethod]
public void ReactiveCatch() {
var source = new[] { new { i = 1 }, new { i = 0 } }.ToObservable();
var a = source
.Do(x => { var j = 1 / x.i; })
.CatchAndStop()
.ToEnumerable()
.ToArray();
Assert.AreEqual(new { i = 1 }, a[0]);
}
But what if I need to Catch an exception of specific type? Something like .CatchAndStop<TimeoutException>().
IObservable<TSource> Catch<TSource, TException>(...) has two generic parameters. Both of them need to be inferred in order to not specify them.
A dummy Func<T> to the rescue.
static class Mixin {
public static IObservable<TSource> CatchAndStop<TSource>(
this IObservable<TSource> source) => source.CatchAndStop(() => new Exception());
public static IObservable<TSource> CatchAndStop<TSource, TException>(
this IObservable<TSource> source, Func<TException> witness
) where TException : Exception
=> source.Catch((TException exc) => Observable.Empty<TSource>());
}
[TestClass]
public class ReactiveExtensions {
[TestMethod]
public void ReactiveCatch() {
var source = new[] { new { i = 1 }, new { i = 0 } }.ToObservable();
var a = source
.Do(x => { var j = 1 / x.i; })
.CatchAndStop(() => new TimeoutException())
.CatchAndStop()
.ToEnumerable()
.ToArray();
Assert.AreEqual(new { i = 1 }, a[0]);
}
TSource is inferred from the IObservable<TSource> source
TException will be inferred from the second parameter - witness, which serves purely as a holder for TException's generic parameter type.
Not ideal, but whatever works ...

Transforming an observable to another with different emit intervals

I have a hot observable that emits at random intervals like different numbers.
1--1-----1--1-2--4
I am looking for a way when there are duplicates within a predefined interval to take this item and merge it back to the sequence until it finds space to bypass the interval threshold.
I have implemented a solution which I believe is not optimum as when I test it in production with real objects instead of Integers it creates kind of back-pressure in the system and I see the CPU going mad. Below is the test of what I have up to now.
using System;
using System.Collections.Generic;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using Microsoft.Reactive.Testing;
using Xunit;
using Xunit.Abstractions;
namespace Specs{
public class CollectDuplicatesSpecs:ReactiveTest{
private readonly ITestOutputHelper _outputHelper;
public CollectDuplicatesSpecs(ITestOutputHelper outputHelper){
_outputHelper = outputHelper;
}
[Fact]
public void MethodName(){
var testScheduler = new TestScheduler();
var hotObservable = testScheduler.CreateHotObservable(OnNext(10, 1), OnNext(20, 1), OnNext(30, 1),OnNext(40, 1));
var subject = new Subject<int>();
hotObservable.Merge(subject).Window(TimeSpan.FromTicks(20), testScheduler).Select(observable => {
observable.CollectDuplicates(i => i).Delay(TimeSpan.FromTicks(1), testScheduler).Subscribe(subject);
return observable.Distinct();
}).SelectMany(observable => observable).Subscribe(i => _outputHelper.WriteLine($"{testScheduler.Clock}-{i}"));
testScheduler.AdvanceBy(160);
}
}
public static class RxEx{
public static IObservable<TSource> CollectDuplicates<TSource>(this IObservable<TSource> source, Func<TSource, int> keySelector = null) {
return Observable.Create<TSource>(observer => {
var dubplicateCollector = new DubplicateCollector<TSource>(keySelector);
var duplicateCollectorSubscription = dubplicateCollector.Matches.Subscribe(observer);
var disposable = source.Distinct(dubplicateCollector).Finally(dubplicateCollector.Dispose).Subscribe();
return new CompositeDisposable(disposable, duplicateCollectorSubscription, dubplicateCollector);
});
}
}
public class DubplicateCollector<TSource> : IEqualityComparer<TSource>,IDisposable {
private readonly Func<TSource, int> _keySelector;
readonly Subject<TSource> _matches = new Subject<TSource>();
public DubplicateCollector(Func<TSource, int> keySelector) {
_keySelector = keySelector;
}
public IObservable<TSource> Matches => _matches;
public bool Equals(TSource x, TSource y) {
var equals = IsMatch(x, y);
if (equals)
_matches.OnNext(x);
return equals;
}
private bool IsMatch(TSource x, TSource y) {
if (_keySelector != null)
return _keySelector(x).Equals(_keySelector(y));
var equals = x != null && x.Equals(y);
return equals;
}
public int GetHashCode(TSource obj) {
return _keySelector(obj);
}
public void Dispose(){
_matches?.Dispose();
}
}
}
which prints
10-1
21-1
40-1
60-1
I'm struggling to get what you want: Some marble diagrams may help. I'm assuming you essentially want something like a smoothing operator: If messages come in bursts, then smooth them out over time somehow.
Based on this answer, you can create an operator that handles the smoothing:
public static class ObservableDrainExtensions
{
public static IObservable<T> TimeDrained<T>(this IObservable<T> source, TimeSpan ts, IScheduler scheduler)
{
return source.Drain(x => Observable.Empty<T>().Delay(ts, scheduler).StartWith(x));
}
public static IObservable<T> TimeDrained<T>(this IObservable<T> source, TimeSpan ts)
{
return TimeDrained(source, ts, Scheduler.Default);
}
public static IObservable<TOut> Drain<TSource, TOut>(this IObservable<TSource> source,
Func<TSource, IObservable<TOut>> selector)
{
return Observable.Defer(() =>
{
BehaviorSubject<Unit> queue = new BehaviorSubject<Unit>(new Unit());
return source
.Zip(queue, (v, q) => v)
.SelectMany(v => selector(v)
.Do(_ => { }, () => queue.OnNext(new Unit()))
);
});
}
}
Drain can linearly smooth things out, TimeDrained does so based on a TimeSpan. You can combine this with GroupBy to add the distinct element to it:
[Fact]
public void MethodName()
{
var testScheduler = new TestScheduler();
var hotObservable = testScheduler.CreateHotObservable(
OnNext(10, 1),
OnNext(20, 1),
OnNext(30, 1),
OnNext(40, 1)
);
var ts = TimeSpan.FromTicks(20);
hotObservable
.GroupBy(i => i) //comparison key
.Select(g => g.TimeDrained(ts, testScheduler))
.Merge()
.Subscribe(i => Console.WriteLine($"{testScheduler.Clock}-{i}"));
testScheduler.AdvanceBy(160);
}
Output is:
10-1
30-1
50-1
70-1
If this isn't what you're looking for, then please clear up the question.

Merge elements in list by property

Context
I have a list of time intervals. Time interval type is HistoMesures.
Each HistoMesure is defined by a Debut (begin) property, a Fin (end) property, and a Commentaires (a little note) property.
My list is made in such a way that :
All HistoMesure are exclusive, I mean that they can't be overlapping each other.
The list is sorted by Debut, so by the beggining of the interval.
Edit : All HistoMesure are contiguous in this configuration.
Question
I want to merge (transform two little intervals in one big interval) all adjacent HistoMesure which have the same Commentaires. Currently I achieve this that way :
//sortedHistos type is List<HistoMesure>
int i = 0;
while (i < sortedHistos.Count - 1)
{
if (sortedHistos[i].Commentaires == sortedHistos[i + 1].Commentaires)
{
sortedHistos[i].Fin = sortedHistos[i + 1].Fin;
sortedHistos.RemoveAt(i + 1);
}
else
{
++i;
}
}
But I feel that it exists a more elegant way to do this, maybe with LINQ. Do you have any suggestion ?
Your solution works fine, I would keep it.
Don't try too hard to use LINQ if it doesn't match your requirements. LINQ is great to write queries (this is the Q of LINQ), not so great to modify existing lists.
This code will produce overlapping merged intervals. I.e. if you have intervals A, B, C where A and C have same commentaries, result will be AC, B:
var result = from h in sortedHistos
group h by h.Commentaires into g
select new HistoMesure {
Debut = g.First().Debut, // thus you have sorted entries
Fin = g.Last().Fin,
Commentaires = g.Key
};
You can use Min and Max if intervals are not sorted.
UPDATE: There is no default LINQ operator which allows you to create adjacent groups. But you always can create one. Here is IEnumerable<T> extension (I skipped arguments check):
public static IEnumerable<IGrouping<TKey, TElement>> GroupAdjacent<TKey, TElement>(
this IEnumerable<TElement> source, Func<TElement, TKey> keySelector)
{
using (var iterator = source.GetEnumerator())
{
if(!iterator.MoveNext())
{
yield break;
}
else
{
var comparer = Comparer<TKey>.Default;
var group = new Grouping<TKey, TElement>(keySelector(iterator.Current));
group.Add(iterator.Current);
while(iterator.MoveNext())
{
TKey key = keySelector(iterator.Current);
if (comparer.Compare(key, group.Key) != 0)
{
yield return group;
group = new Grouping<TKey, TElement>(key);
}
group.Add(iterator.Current);
}
if (group.Any())
yield return group;
}
}
}
This extension creates groups of adjacent elements which have same key value. Unfortunately all implementations of IGrouping in .NET are internal, so you need yours:
public class Grouping<TKey, TElement> : IGrouping<TKey, TElement>
{
private List<TElement> elements = new List<TElement>();
public Grouping(TKey key)
{
Key = key;
}
public TKey Key { get; private set; }
public IEnumerator<TElement> GetEnumerator()
{
return elements.GetEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
public void Add(TElement element)
{
elements.Add(element);
}
}
And now your code will look like:
var result = sortedHistos.GroupAdjacent(h => h.Commentaries)
.Select(g => new HistoMesure {
Debut = g.Min(h => h.Debut),
Fin = g.Max(h => h.Fin),
Commentaries = g.Key
});
Using Linq and borrowing from this article to group by adjacent values, this should work:
Your query:
var filteredHistos = sortedHistos
.GroupAdjacent(h => h.Commentaires)
.Select(g => new HistoMesure
{
Debut = g.First().Debut,
Fin = g.Last().Fin,
Commentaires = g.Key
});
And copying from the article, the rest of the code to group by:
public class GroupOfAdjacent<TSource, TKey> : IEnumerable<TSource>, IGrouping<TKey, TSource>
{
public TKey Key { get; set; }
private List<TSource> GroupList { get; set; }
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return ((System.Collections.Generic.IEnumerable<TSource>)this).GetEnumerator();
}
System.Collections.Generic.IEnumerator<TSource> System.Collections.Generic.IEnumerable<TSource>.GetEnumerator()
{
foreach (var s in GroupList)
yield return s;
}
public GroupOfAdjacent(List<TSource> source, TKey key)
{
GroupList = source;
Key = key;
}
}
public static class LocalExtensions
{
public static IEnumerable<IGrouping<TKey, TSource>> GroupAdjacent<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector)
{
TKey last = default(TKey);
bool haveLast = false;
List<TSource> list = new List<TSource>();
foreach (TSource s in source)
{
TKey k = keySelector(s);
if (haveLast)
{
if (!k.Equals(last))
{
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
list = new List<TSource>();
list.Add(s);
last = k;
}
else
{
list.Add(s);
last = k;
}
}
else
{
list.Add(s);
last = k;
haveLast = true;
}
}
if (haveLast)
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
}
}
If I understood you correctly, you need something like this:
var mergedMesures = mesures
.GroupBy(_ => _.Commentaires)
.Select(_ => new HistoMesures
{
Debut = _.Min(item => item.Debut),
Fin = _.Max(item => item.Fin),
Commentaires = _.Key
});

Categories