In this answer to a question about Subject<T> Enigmativity mentioned:
as an aside, you should try to avoid using subjects at all. The
general rule is that if you're using a subject then you're doing
something wrong.
I often use subjects as backing fields for IObservable properties, which would have probably been .NET events in the days before Rx. e.g. instead of something like
public class Thing
{
public event EventHandler SomethingHappened;
private void DoSomething()
{
Blah();
SomethingHappened(this, EventArgs.Empty);
}
}
I might do
public class Thing
{
private readonly Subject<Unit> somethingHappened = new Subject<Unit>();
public IObservable<Unit> SomethingHappened
{
get { return somethingHappened; }
}
private void DoSomething()
{
Blah();
somethingHappened.OnNext(Unit.Default);
}
}
So, if I want to avoid using Subject what would be the correct way of doing this kind of thing? Or I should I stick to using .NET events in my interfaces, even when they'll be consumed by Rx code (so probably FromEventPattern)?
Also, a bit more details on why using Subject like this is a bad idea would be helpful.
Update: To make this question a bit more concrete, I'm talking about using Subject<T> as a way to get from non-Rx code (maybe you're working with some other legacy code) into the Rx world. So, something like:
class MyVolumeCallback : LegacyApiForSomeHardware
{
private readonly Subject<int> volumeChanged = new Subject<int>();
public IObservable<int> VolumeChanged
{
get
{
return volumeChanged.AsObservable();
}
}
protected override void UserChangedVolume(int newVolume)
{
volumeChanged.OnNext(newVolume);
}
}
Where, instead of using events, the LegacyApiForSomeHardware type makes you override virtual methods as a way of getting "this just happened" notifications.
For one thing, someone can cast the SomethingHappened back to an ISubject and feed things into it from the outside. At the very least, apply AsObservable to it in order to hide the subject-ness of the underlying object.
Also, subject broadcasting of callbacks doesn't do strictly more than a .NET event. For example, if one observer throws, the ones that are next in the chain won't be called.
static void D()
{
Action<int> a = null;
a += x =>
{
Console.WriteLine("1> " + x);
};
a += x =>
{
Console.WriteLine("2> " + x);
if (x == 42)
throw new Exception();
};
a += x =>
{
Console.WriteLine("3> " + x);
};
a(41);
try
{
a(42); // 2> throwing will prevent 3> from observing 42
}
catch { }
a(43);
}
static void S()
{
Subject<int> s = new Subject<int>();
s.Subscribe(x =>
{
Console.WriteLine("1> " + x);
});
s.Subscribe(x =>
{
Console.WriteLine("2> " + x);
if (x == 42)
throw new Exception();
});
s.Subscribe(x =>
{
Console.WriteLine("3> " + x);
});
s.OnNext(41);
try
{
s.OnNext(42); // 2> throwing will prevent 3> from observing 42
}
catch { }
s.OnNext(43);
}
In general, the caller is dead once an observer throws, unless you protect every On* call (but don't swallow exceptions arbitrarily, as shown above). This is the same for multicast delegates; exceptions will swing back at you.
Most of the time, you can achieve what you want to do without a subject, e.g. by using Observable.Create to construct a new sequence. Such sequences don't have an "observer list" that results from multiple subscriptions; each observer has its own "session" (the cold observable model), so an exception from an observer is nothing more than a suicide command in a confined area rather than blowing yourself up in the middle of a square.
Essentially, subjects are best used at the edges of the reactive query graph (for ingress streams that need to be addressable by another party that feeds in the data, though you could use regular .NET events for this and bridge them to Rx using FromEvent* methods) and for sharing subscriptions within a reactive query graph (using Publish, Replay, etc. which are Multicast calls in disguise, using a subject). One of the dangers of using subjects - which are very stateful due to their observer list and potential recording of messages - is to use them when trying to write a query operator using subjects. 99.999% of the time, such stories have a sad ending.
In an answer on the Rx forum, Dave Sexton (of Rxx), said as part an answer to something:
Subjects are the stateful components of Rx. They are useful for when
you need to create an event-like observable as a field or a local
variable.
Which is exactly what's happening with this question, he also wrote an in-depth follow up blog post on To Use Subject Or Not To Use Subject? which concludes with:
When should I use a subject?
When all of the following are true:
you don't have an observable or anything that can be converted into one.
you require a hot observable.
the scope of your observable is a type.
you don't need to define a similar event and no similar event already exists.
Why should I use a subject in that case?
Because you've got no choice!
So, answering the inner question of "details on why using Subject like this is a bad idea" - it's not a bad idea, this is one of the few places were using a Subject is the correct way to do things.
While I can't speak for Enigmativity directly, I imagine it's because it's very low-level, something you don't really need to use directly; everything that's offered by the Subject<T> class can be achieved by using the classes in the System.Reactive.Linq namespace.
Taking the example from the Subject<T> documentation:
Subject<string> mySubject = new Subject<string>();
//*** Create news feed #1 and subscribe mySubject to it ***//
NewsHeadlineFeed NewsFeed1 = new NewsHeadlineFeed("Headline News Feed #1");
NewsFeed1.HeadlineFeed.Subscribe(mySubject);
//*** Create news feed #2 and subscribe mySubject to it ***//
NewsHeadlineFeed NewsFeed2 = new NewsHeadlineFeed("Headline News Feed #2");
NewsFeed2.HeadlineFeed.Subscribe(mySubject);
This is easily achieved with the Merge extension method on the Observable class:
IObservable<string> feeds =
new NewsHeadlineFeed("Headline News Feed #1").HeadlineFeed.Merge(
new NewsHeadlineFeed("Headline News Feed #2").HeadlineFeed);
Which you can then subscribe to normally. Using Subject<T> just makes the code more complex. If you're going to use Subject<T> then you should be doing some very low-level processing of observables where the extension methods fail you.
One approach for classes which have simple one-off events, is to provide a ToObservable method which creates a meaningful cold observable based on an event.
This is more readable than using the Observable factory methods, and allows developers who don't use Rx to make use of the API.
public IObservable<T> ToObservable()
{
return Observable.Create<T>(observer =>
{
Action notifier = () =>
{
switch (Status)
{
case FutureStatus.Completed:
observer.OnNext(Value);
observer.OnCompleted();
break;
case FutureStatus.Cancelled:
observer.OnCompleted();
break;
case FutureStatus.Faulted:
observer.OnError(Exception);
break;
}
};
Resolve += notifier;
return () => Resolve -= notifier;
});
}
Related
I have code that uses a subject to create an observable. I'm able to pass in a value to the observable stream using subject.onNext(value). The problem is if I subscribe to that observable after a value was passed in, I would like to still get that value. From what I can tell, subjects don't hold onto value, they just pass them along like an event. So how do I change this code from using subject, to using an Observable.
private readonly Subject<int> _valueSubject = new Subject<int>();
public IObservable<int> ValueObservable => _valueSubject ;
public void SetValue(int valuePassedIn)
{
_valueSubject.OnNext(valuePassedIn);
}
Edit: Found out you can get latest values using BehaviorSubject, but would still like to know how to use Observable.Create instead of Subject in a scenario like this.
Assuming that you don't need to call the OnNext directly, a cold observable might be what you want:
IObservable<int> coldObservable = Observable.Create<int>(obs =>
{
obs.OnNext(1);
obs.OnNext(2);
obs.OnNext(3);
obs.OnCompleted();
return () => { };
});
Otherwise a ReplaySubject will allow you to keep a sized buffer of values that will be emitted as each observer subscribes. Not exactly the same as remembering all values I realize but this would not be a good idea anyway due to memory usage. Might be closer than the single value BehaviourSubject provides.
The following will allow 2 values to be available to new subscribers:
ISubject<int> replaySubject = new ReplaySubject<int>(2);
IObservable<int> observable;
[TestMethod]
public void TestMethod()
{
observable = replaySubject;
replaySubject.OnNext(1);
replaySubject.OnNext(2);
replaySubject.OnNext(3);
replaySubject.OnNext(4);
replaySubject.OnNext(5);
observable.Subscribe(OnValue);
}
Output:
Observed value:4
Observed value:5
How should one approach on transforming exceptions in an IObservable stream to plain domain objects and to resubscribe to the stream transparently?
Addendum: As James points out in the comment, my use case idea was something like having a should-be continuous stream over an unreliable source, e.g. a network. In case of a glitch, just try to reconnect to the source, but notify the downstream processor.
In fact, this relates my other question at Translating a piece of asynchronous C# code to F# (with Reactive Extensions and FSharpx), which in turn stems from How to implement polling using Observables?.
In fact, now that I think of it, I could first use code at How to write a generic, recursive extension method in F#? ("RetryAfterDelay") (with some more parameters to adjust the RetryAfterDelay behavior) and chain it with this implementation. When the retries are exhausted, a domain error will be produced and the poller will be reinitiated. Granted, there problably will be a more efficient way, but nevertheless. :) ... Or provide just a call-back function to log error instead of transforming them to domain events, well, choices abound...
But back to the original code...
For instance, if I have
public enum EventTypeEnum
{
None = 0,
Normal = 1,
Faulted = 2
}
public class Event
{
public EventTypeEnum Type { get; set; }
}
private static IObservable<int> FaultingSequence1()
{
var subject = new ReplaySubject<int>();
subject.OnNext(1);
subject.OnNext(2);
subject.OnError(new InvalidOperationException("Something went wrong!"));
return subject;
}
private static IEnumerable<int> FaultingSequence2()
{
for(int i = 0; i < 3; ++i)
{
yield return 1;
}
throw new InvalidOperationException("Something went wrong!");
}
//Additional pondering: Why isn't FaultingSequence2().ToObservable() too be procted by Catch?
//
//This part is for illustratory purposes here. This is the piece I'd like
//behave so that exceptions would get transformed to Events with EventTypeEnum.Faulted
//and passed along to the stream that's been subscribed to while resubscribing to
//FaultingSequence1. That is, the subscribed would learn about the fault through a
//domain event type.
//Retry does the resubscribing, but only on OnError.
var stream = FaultingSequence1().Catch<int, Exception>(ex =>
{
Console.WriteLine("Exception: {0}", ex);
return Observable.Throw<int>(ex);
}).Retry().Select(i => new Event { Type = EventTypeEnum.Normal });
//How to get this to print "Event type: Normal", "Event type: Normal", "Event type: Faulted"?
stream.Subscribe(i => Console.WriteLine("Event type: {0}", i.Type));
This problem has really got me now! Any advice?
There's an operator called Materialize which converts each event into a Notification<T>:
OnNext:
OnNext a Notification<T> with Kind OnNext containing a value.
OnError:
OnNext a Notification<T> with Kind OnError containing an exception.
OnCompleted.
OnCompleted:
OnNext a Notification<T> with Kind OnCompleted
OnCompleted.
So the subscription still completes when either OnError or OnCompleted is invoked, but OnError is never invoked on the Subscriber. So you could be able to do something like this...
source
.Materialize()
.Repeat();
However, this will resubscribe to the source even when the original subscription completes naturally (via OnCompleted).
So maybe you still want OnError to be invoked, but you also want the exception from the original OnError to be passed through OnNext inside of a Notification<T>. For that, you could use something like this:
source
.Materialize()
.SelectMany(notification =>
notification.Kind == NotificationKind.OnError
? Observable.Return(notification).Concat(Observable.Exception(notification.Exception))
: Observable.Return(notification)
)
.Retry();
In this manner, if the subscription completes naturally (via OnCompleted), then the source will not be resubscribed.
Once you have that set up, it's each enough to map each type of notification to whatever domain object you want to use:
source
.Materialize()
.SelectMany(notification =>
notification.Kind == NotificationKind.OnError
? Observable.Return(notification).Concat(Observable.Exception(notification.Exception))
: Observable.Return(notification)
)
.Retry()
.Map(notification => {
switch (notification.Kind) {
case (NotificationKind.OnNext): return // something.
case (NotificationKind.OnError): return // something.
case (NotificationKind.OnCompleted): return // something.
default: throw new NotImplementedException();
}
});
I'm trying to expose an observable sequence that gives observers all existing records in a database table plus any future items. For the sake of argument, lets say it's log entries. Therefore, I'd have something like this:
public class LogService
{
private readonly Subject<LogEntry> entries;
public LogService()
{
this.entries = new Subject<LogEntry>();
this.entries
.Buffer(...)
.Subscribe(async x => WriteLogEntriesToDatabaseAsync(x));
}
public IObservable<LogEntry> Entries
{
get { return this.entries; }
}
public IObservable<LogEntry> AllLogEntries
{
get
{
// how the heck?
}
}
public void Log(string message)
{
this.entries.OnNext(new LogEntry(message));
}
private async Task<IEnumerable<LogEntry>> GetLogEntriesAsync()
{
// reads existing entries from DB table and returns them
}
private async Task WriteLogEntriesToDatabaseAsync(IList<LogEntry> entries)
{
// writes entries to the database
}
}
My initial thought for the implementation of AllLogEntries was something like this:
return Observable.Create<LogEntry>(
async observer =>
{
var existingEntries = await this.GetLogEntriesAsync();
foreach (var existingEntry in existingEntries)
{
observer.OnNext(existingEntry);
}
return this.entries.Subscribe(observer);
});
But the problem with this is that there could log entries that have been buffered and not yet written to the database. Hence, those entries will be missed because they are not in the database and have already passed through the entries observable.
My next thought was to separate the buffered entries from the non-buffered and use the buffered when implementing AllLogEntries:
return Observable.Create<LogEntry>(
async observer =>
{
var existingEntries = await this.GetLogEntriesAsync();
foreach (var existingEntry in existingEntries)
{
observer.OnNext(existingEntry);
}
return this.bufferedEntries
.SelectMany(x => x)
.Subscribe(observer);
});
There are two problems with this:
It means clients of AllLogEntries also have to wait for the buffer timespan to pass before they receive their log entries. I want them to see log entries instantaneously.
There is still a race condition in that log entries could be written to the database between the point at which I finish reading the existing ones and the point at which I return the future entries.
So my question is: how would I actually go about achieving my requirements here with no possibility of race conditions, and avoiding any major performance penalties?
To do this via the client code, you will probably have to implement a solution using polling and then look for differences between calls. Possibly combining a solution with
Observable.Interval() : http://rxwiki.wikidot.com/101samples#toc28 , and
Observable.DistinctUntilChanged()
will give you sufficient solution.
Alternatively, I'd suggest you try to find a solution where the clients are notified when the DB/table is updated. In a web application, you could use something like SignalR to do this.
For example: http://techbrij.com/database-change-notifications-asp-net-signalr-sqldependency
If its not a web-application, a similar update mechanism via sockets may work.
See these links (these came from the accepted answer of SignalR polling database for updates):
http://xsockets.net/api/net-c#snippet61
https://github.com/codeplanner/XSocketsPollingLegacyDB
I have two streams of objects that each have a Timestamp value. Both streams are in order, so for example the timestamps might be Ta = 1,3,6,6,7 in one stream and Tb = 1,2,5,5,6,8 in the other. Objects in both streams are of the same type.
What I'd like to be able to do is to put each of these events on the bus in order of timestamp, i.e., put A1, then B1, B2, A3 and so on. Furthermore, since some streams have several (sequential) elements with the same timestamp, I want those elements grouped so that each new event is an array. So we would put [A3] on the bus, followed by [A15,A25] and so on.
I've tried to implement this by making two ConcurrentQueue structures, putting each event at the back of the queue, then looking at each front of the queue, choosing first the earlier event and then traversing the queue such that all events with this timestamp are present.
However, I've encountered two problems:
If I leave these queues unbounded, I quickly run out of memory as the read op is a lot faster than the handlers receiving the events. (I've got a few gigabytes of data).
I sometimes end up with a situation where I handle the event, say, A15 before A25 has arrived. I somehow need to guard against this.
I'm thinking that Rx can help in this regard but I don't see an obvious combinator(s) to make this possible. Thus, any advice is much appreciated.
Rx is indeed a good fit for this problem IMO.
IObservables can't 'OrderBy' for obvious reasons (you would have to observe the entire stream first to guarantee the correct output order), so my answer below makes the assumption (that you stated) that your 2 source event streams are in order.
It was an interesting problem in the end. The standard Rx operators are missing a GroupByUntilChanged that would have solved this easily, as long as it called OnComplete on the previous group observable when the first element of the next group was observed. However looking at the implementation of DistinctUntilChanged it doesn't follow this pattern and only calls OnComplete when the source observable completes (even though it knows there will be no more elements after the first non-distinct element... weird???). Anyway, for those reasons, I decided against a GroupByUntilChanged method (to not break Rx conventions) and went instead for a ToEnumerableUntilChanged.
Disclaimer: This is my first Rx extension so would appreciate feedback on my choices made. Also, one main concern of mine is the anonymous observable holding the distinctElements list.
Firstly, your application code is quite simple:
public class Event
{
public DateTime Timestamp { get; set; }
}
private IObservable<Event> eventStream1;
private IObservable<Event> eventStream2;
public IObservable<IEnumerable<Event>> CombineAndGroup()
{
return eventStream1.CombineLatest(eventStream2, (e1, e2) => e1.Timestamp < e2.Timestamp ? e1 : e2)
.ToEnumerableUntilChanged(e => e.Timestamp);
}
Now for the ToEnumerableUntilChanged implementation (wall of code warning):
public static IObservable<IEnumerable<TSource>> ToEnumerableUntilChanged<TSource,TKey>(this IObservable<TSource> source, Func<TSource,TKey> keySelector)
{
// TODO: Follow Rx conventions and create a superset overload that takes the IComparer as a parameter
var comparer = EqualityComparer<TKey>.Default;
return Observable.Create<IEnumerable<TSource>>(observer =>
{
var currentKey = default(TKey);
var hasCurrentKey = false;
var distinctElements = new List<TSource>();
return source.Subscribe((value =>
{
TKey elementKey;
try
{
elementKey = keySelector(value);
}
catch (Exception ex)
{
observer.OnError(ex);
return;
}
if (!hasCurrentKey)
{
hasCurrentKey = true;
currentKey = elementKey;
distinctElements.Add(value);
return;
}
bool keysMatch;
try
{
keysMatch = comparer.Equals(currentKey, elementKey);
}
catch (Exception ex)
{
observer.OnError(ex);
return;
}
if (keysMatch)
{
distinctElements.Add(value);
return;
}
observer.OnNext( distinctElements);
distinctElements.Clear();
distinctElements.Add(value);
currentKey = elementKey;
}), observer.OnError, () =>
{
if (distinctElements.Count > 0)
observer.OnNext(distinctElements);
observer.OnCompleted();
});
});
}
I am trying to build a subscription list. Let's take the example:
list of Publishers, each having a list of Magazines, each having a list of subscribers
Publishers --> Magazines --> Subscribers
Makes sense to use of a Dictionary within a Dictionary within a Dictionary in C#. Is it possible to do this without locking the entire structure when adding/removing a subscriber without race conditions?
Also the code gets messy very quickly in C# which makes me think I am not going down the right path. Is there an easier way to do this? Here are the constructor and subscribe method:
Note: The code uses Source, Type, Subscriber instead of the names above
Source ---> Type ---> Subscriber
public class SubscriptionCollection<SourceT, TypeT, SubscriberT>
{
// Race conditions here I'm sure! Not locking anything yet but should revisit at some point
ConcurrentDictionary<SourceT, ConcurrentDictionary<TypeT, ConcurrentDictionary<SubscriberT, SubscriptionInfo>>> SourceTypeSubs;
public SubscriptionCollection()
{
SourceTypeSubs = new ConcurrentDictionary<SourceT, ConcurrentDictionary<TypeT, ConcurrentDictionary<SubscriberT, SubscriptionInfo>>>();
}
public void Subscribe(SourceT sourceT, TypeT typeT, SubscriberT subT) {
ConcurrentDictionary<TypeT, ConcurrentDictionary<SubscriberT, SubscriptionInfo>> typesANDsubs;
if (SourceTypeSubs.TryGetValue(sourceT, out typesANDsubs))
{
ConcurrentDictionary<SubscriberT, SubscriptionInfo> subs;
if (typesANDsubs.TryGetValue(typeT, out subs))
{
SubscriptionInfo subInfo;
if (subs.TryGetValue(subT, out subInfo))
{
// Subscription already exists - do nothing
}
else
{
subs.TryAdd(subT, new SubscriptionInfo());
}
}
else
{
// This type does not exist - first add type, then subscription
var newType = new ConcurrentDictionary<SubscriberT, SubscriptionInfo>();
newType.TryAdd(subT, new SubscriptionInfo());
typesANDsubs.TryAdd(typeT, newType);
}
}
else
{
// this source does not exist - first add source, then type, then subscriptions
var newSource = new ConcurrentDictionary<TypeT, ConcurrentDictionary<SubscriberT, SubscriptionInfo>>();
var newType = new ConcurrentDictionary<SubscriberT, SubscriptionInfo>();
newType.TryAdd(subT, new SubscriptionInfo());
newSource.TryAdd(typeT, newType);
SourceTypeSubs.TryAdd(sourceT, newSource);
};
}
If you use ConcurrentDictionary, like you already do, you don't need locking, that's already taken care of.
But you still have to think about race conditions and how to deal with them. Fortunately, ConcurrentDictionary gives you exactly what you need. For example, if you have two threads, that both try to subscribe to source that doesn't exist yet at the same time, only one of them will succeed. But that's why TryAdd() returns whether the addition was successful. You can't just ignore its return value. If it returns false, you know some other thread already added that source, so you can retrieve the dictionary now.
Another option is to use the GetOrAdd() method. It retrieves already existing value, and creates it if it doesn't exist yet.
I would rewrite your code like this (and make it much simpler along the way):
public void Subscribe(SourceT sourceT, TypeT typeT, SubscriberT subT)
{
var typesAndSubs = SourceTypeSubs.GetOrAdd(sourceT,
_ => new ConcurrentDictionary<TypeT, ConcurrentDictionary<SubscriberT, SubscriptionInfo>>());
var subs = typesAndSubs.GetOrAdd(typeT,
_ => new ConcurrentDictionary<SubscriberT, SubscriptionInfo>());
subs.GetOrAdd(subT, _ => new SubscriptionInfo());
}