I rewrote some old async code of mine that makes that makes SOAP calls. The fetch() method would go out, get the result from the SOAP interface, and then add it to a DataTable that is bound to my WPF view. The new code uses Reactive Extensions to get a list of strings and creates an IObservable from the list. I thought it would return the results asynchronously, but the entire UI locks up until the entire result set is ready. I'm new to Reactive Extensions so I'm hoping I'm just missing something simple.
The Code:
(from click event)
private void fetchSoapRows()
{
var strings = (txtInput.Text.Split('*')).ToObservable();
strings.Subscribe(s=> SoapQueryEngine.Fetch(s));
}
Also, does anyone know how I could write a test to make certain this method doesn't block the application in the future?
There are two parts to an observable query, the Query itself and the Subscription.
Your Query is an IEnumerable<string> producing values as fast as the computer can do it.
Your Subscription is
SoapQueryEngine.Fetch(s);
This runs Fetch for each string produced by the Query in the Subscriber thread which tends to be the thread where you're setting up your Subscription (although it isn't necessarily).
The issue has to do with the intention and design of Rx. It's intended that the Query is the long-running process and the Subscription is a short method that deals with the results. If you want to run a long running function as an Rx Observable your best option is to use Observable.ToAsync.
You should also take a look at this question to see a similar problem which shows more of what's going on in the background.
There is nothing inherently concurrent about Rx. If you want to make your calls to Fetch you will need to change SoapQueryEngine so that it is async or call it on another thread and then bring the results back to the UI thread.
Try this way. Instead of subscribing to the event text changed event, create an observable on the event and observe it on the thread pool:
Observable.FromEventPattern(<subscribe event>, <unsubscribe event>)
.ObserveOn(ThreadPoolScheduler.Instance)
.SelectMany(s => s.Split('*'))
.Subscribe(s=> SoapQueryEngine.Fetch(s));
Related
I have a simple BackgroundWorker that should generate results, but I don't know how to get them back to my calling method.
I have read a number of answers, and they all show me how the RunWorkerCompleted event can receive the results of the DoWork event handler, and to prove this they show a message box with the result. That's all good, but how do I get a result back to the actual method that called RunWorkerAsync?
myResult=myBackGroundWorker.RunWorkerAsync(); // This obviously doesn't compile
This answer in particular has been very useful:
How to make BackgroundWorker return an object
but I still don't know how to access the results.
This answer also mentions the use of a delegate:
BackgroundWorker Return A Value?
but I don't understand if this would solve my problem, or if I would still just get the result inside the RunWorkCompleted event handler.
Short answer: you can't.
Think about what it means to return the value back to the method that called RunWorkerAsync. This would mean that the method has to pause its execution, wait for the background worker to complete its work, then continue again. But you still want other methods to run during the waiting, otherwise the UI would freeze.
Does that sound familiar to you? That's basically what an async method does when it encounters an await expression!
BackgroundWorker is quite an old API, and there wasn't the fancy feature of async/await when it came out (async/await is a language feature anyway), which is why it uses events to do the trick. Back then, there wasn't a way to do an operation asynchronously, and return a value back to the caller elegantly.
Now that we have async/await, you can use it to do what you want:
someReuslt = await Task.Run(() => { ... });
If you haven't heard about async/await at all, this is a good place to start.
What would be the most correct way to use async method in databound property getter? I am talking about solid, scientific arguments, not personal preferences. I've read many threads about the problem, but not this specific case. Some of the solutions don't work in all the cases, and some of the suggestion, well... they were just too subjective or just wrong.
What I don't accept and why:
You can't - Actually, it is possible. There are many posts like "there are no such things like async properties", "it is against the design of the language" etc. but also there are many sensible explanations why such expressions are false
This should me method, not property - It can't be. I want to databind it. I provide property "proxies" for people using this code because in the future there may be different method to calculate this pseudo-property. And I want the View-side of the binding to be simple as possible
Use property to store the cached result of the method - that would defeat the purpose, it is actually something that changes dynamically and the class is an ORM Entity so it would store redundant data to the DB.
Use SomeTask.Result; or SomeTask.GetAwaiter().GetResult() - In most cases I would just use it. I've successfully used those in many cases i.e. Console applications. It's nice, clear and easily readable. But when I use it in databound property I get a deadlock
Problem background (simplified)
Let's say that I am responsible for developing ORM mechanism in a project. There was a first stable version, but now I want to add some properties to the Entities for the DataBinders who are responsible for the layout. I can edit Entity layer, but I can't edit Mapping and Repository layers. (I am not held againt my will, this situation is fictional simplification). All the methods in repositories are async. All I can do is ask someone responsible to provide identical synchronous methods for all of the methods, but it would be stupid to this kind of redundant work.
Only solution I can use now
_something = Task.Run(async () => await AnotherRepository.CalculateStuff(this)).Result;
And it just doesn't look right to me. It works, but I have to await my method inside the lambda in Task.Run(). I am stuck with it for the time being, and I want to know the simplest and correct approach.
Repository method pseudo-code
public async static Task<IList<CalculatedStuff>> CalculateStuff(SomeClass class)
{
return await Task.Run(() =>
{
using (var session = Helper.OpenSession())
return session.CreateCriteria(typeof(CalculatedStuff)).Add(Restrictions.Eq("SomeClass", class))
///...
.List<CalculatedStuff>();
});
}
there are no such things like async properties
I have a blog post and MSDN article on "async properties" for data binding. I do take the stance that they are not natural, which is based on these (objective) observations:
Properties read by data binding must return immediately (synchronously).
Asynchronous operations are asynchronous (that is, they complete after some time).
Clearly, these are at complete odds with one another.
Now, there are a few different solutions, but any solution that attempts to violate one of these observations is going to be dubious, at best.
For example, you can attempt to violate the second observation by trying to run the asynchronous operation synchronously. As you discovered, Result / Wait / GetAwaiter().GetResult() will deadlock (for reasons described in detail on my blog). Task.Run(() => ...).GetAwaiter().GetResult() will avoid the deadlock but will execute the code in a free-threaded context (which is OK for most code but not all). These are two different kinds of sync-over-async; I call them the "Blocking Hack" and the "Thread Pool Hack" in my Async Brownfield article, which also covers two other kinds of sync-over-async patterns.
Unfortunately, there is no solution for sync-over-async that works in every scenario. Even if you get it to work, your users would get a substandard experience (blocking the UI thread for an indefinite amount of time), and you may have problems with app stores (I believe MS's at least will actively check for blocking the UI thread and auto-reject). IMO, sync-over-async is best avoided.
However, we obviously cannot violate the first observation, either. If we're data binding to the result of some asynchronous operation, we can't very well return it before the operation completes!
Or can we?
What if we change what the data binding is attaching to? Say, introduce a property that has a default value before the operation is completed, and changes (via INotifyPropertyChanged) to the result of the operation when the operation completes. That sounds reasonable... And we can stick in another property to indicate to the UI that the operation is in progress! And maybe another one to indicate if the operation failed...
This is the line of thinking that resulted in my NotifyTaskCompletion type in the article on data binding (updated NotifyTask type here). It is essentially a data-bindable wrapper for Task<T>, so the UI can respond dynamically to the asynchronous operation without trying to force it to be synchronous.
This does require some changes to the bindings, but you get a nice side effect that your UX is better (non-blocking).
This should me method, not property
Well, you can do this as a property:
TEntity Entity { get { return NotifyTask.Create(() => Repository.GetEntityAsync()); } }
// Data bind to Entity.Result for the results.
// Data bind to Entity.IsNotCompleted for a busy spinner.
However, I would say that it's surprising behavior to have a property read kick off something significant like a database query or HTTP download. That's a pretty wide definition of "property". IMO, this would be better represented as a method, which connotates action more than a property does (or perhaps as part of an asynchronous initialization, which I also describe on my blog). Put another way: I prefer my properties without side effects. Reading a property more than once and having it return different values is counterintuitive. This final paragraph is entirely my own opinion. :)
If you have access to the source code of AnotherRepository.CalculateStuff, you can implement it in a way that won't deadlock when called from bound property. First short summary of why it deadlocks. When you await something, current synchronization context is remembered and the rest of the method (after async) is executed on that context. For UI applications that means the rest of the method is executed on UI thread. But in your case UI thread is already blocked by waiting for the Result of task - hence deadlock.
But there is method of Task named ConfigureAwait. If you pass false for it's only argument (named continueOnCapturedContext) and await task returned by this method - it won't continue on captured context, which will solve your problem. So suppose you have:
// this is UI bound
public string Data
{
get { return GetData().Result; }
}
static async Task<string> GetData() {
await Task.Run(() =>
{
Thread.Sleep(2000);
});
return "test!";
}
This will deadlock when called from UI thread. But if you change it:
static async Task<string> GetData() {
await Task.Run(() =>
{
Thread.Sleep(2000);
}).ConfigureAwait(false);
return "test!";
}
It won't any more.
For those who might read this later - don't do it this way, only if for temporary debugging purposes. Instead return dummy object from your property getter with some IsLoading flag set to true, and meanwhile load data in background and fill dummy object properties when done. This will not freeze your UI during long blocking operation.
I have an "autocomplete" textbox that will invoke a WCF method each time a key is pressed.
The WCF server, in turn, will run an SQL query, return the first 15 results and send them.
However, this results in a noticeable latency when typing in the box.
What I'm about to do instead is this:
Create a new thread when a text_changed event is fired, make that thread wait 1000ms using Stopwatch.ElapsedMilliseconds. During this waiting time, the thread can be stopped permanently.
If it was not stopped, the thread will send the request to the server (and repopulate the auto complete box).
As soon as a new "text_changed" event is fired, I will stop the current thread and start a new one.
Is there a better approach or is this the way to go?
So basically wait for 1 second for the user to stop typing before requesting results.
That's a good solution for conserving server resources, but you are actually adding latency by making the user wait for a minimum of 1000ms.
My guess is that your original issue was that this is a winforms app and the request you made was synchronous by default. As a result, the textbox wasn't accepting user input while the app was waiting for a response. Just making the call asynchronous should solve that issue without making the typing slower.
Another approach is to use the Rx (Reactive Extensions) framework to encapsulate automplete with some very interesting characteristics.
With Rx you get ability to compose (multiple event sources), throttle the user input so that you don't overwhelm the source, and on top of that you can ignore the old result if user typed more in the search box (TakeUntil).
More info:
Rx: Curing your asynchronous programming blues
Curing the asynchronous blues with the Reactive Extensions for .NET
Example:
SO: RX AutoCompleteBox
RxProperty = Observable.FromEvent<TextChangedEventHandler, TextChangedEventArgs>(
h => new TextChangedEventHandler(h),
h => AssociatedObject.TextChanged += h,
h => AssociatedObject.TextChanged -= h)
.Select(t => ((TextBox)t.Sender).Text)
.Throttle(TimeSpan.FromMilliseconds(400))
.SubscribeOnDispatcher()
.Take(10)
.TakeUntil(AssociatedObject.TextChanged );
Instead of FromEvent you can use FromAsync and use proxy Beginxxx Endxxx methods.
I am currently getting to grips with the Reactive Extensions framework for .NET and I am working my way through the various introduction resources I've found (mainly http://www.introtorx.com)
Our application involves a number of hardware interfaces that detect network frames, these will be my IObservables, I then have a variety of components that will consume those frames or perform some manner of transform on the data and produce a new type of frame. There will also be other components that need to display every n'th frame for example.
I am convinced that Rx is going to be useful for our application, however I am struggling with the implementation details for the IObserver interface.
Most (if not all) of the resources I have been reading have said that I should not implement the IObservable interface myself but use one of the provided functions or classes.
From my research it appears that creating a Subject<IBaseFrame> would provide me what I need, I would have my single thread that reads data from the hardware interface and then calls the OnNext function of my Subject<IBaseFrame> instance. The different IObserver components would then receive their notifications from that Subject.
My confusion is coming from the advice give in the appendix of this tutorial where it says:
Avoid the use of the subject types. Rx is effectively a functional programming paradigm. Using subjects means we are now managing state, which is potentially mutating. Dealing with both mutating state and asynchronous programming at the same time is very hard to get right. Furthermore, many of the operators (extension methods) have been carefully written to ensure that correct and consistent lifetime of subscriptions and sequences is maintained; when you introduce subjects, you can break this. Future releases may also see significant performance degradation if you explicitly use subjects.
My application is quite performance critical, I am obviously going to test the performance of using the Rx patterns before it goes in to production code; however I am worried that I am doing something that is against the spirit of the Rx framework by using the Subject class and that a future version of the framework is going to hurt performance.
Is there a better way of doing what I want? The hardware polling thread is going to be running continuously whether there are any observers or not (the HW buffer will back up otherwise), so this is a very hot sequence. I need to then pass the received frames out to multiple observers.
Any advice would be greatly appreciated.
Ok,
If we ignore my dogmatic ways and ignore "subjects are good/bad" all together. Let us look at the problem space.
I bet you either have 1 of 2 styles of system you need to ingrate to.
The system raises an event or a call back when a message arrives
You need to poll the system to see if there are any message to process
For option 1, easy, we just wrap it with the appropriate FromEvent method and we are done. To the Pub!
For option 2, we now need to consider how we poll this and how to do this effciently. Also when we get the value, how do we publish it?
I would imagine that you would want a dedicated thread for polling. You wouldn't want some other coder hammering the ThreadPool/TaskPool and leaving you in a ThreadPool starvation situation. Alternatively you don't want the hassle of context switching (I guess). So assume we have our own thread, we will probably have some sort of While/Sleep loop that we sit in to poll. When the check finds some messages we publish them. Well all of this sounds perfect for Observable.Create. Now we probably cant use a While loop as that wont allow us to ever return a Disposable to allow cancellation. Luckily you have read the whole book so are savvy with Recursive scheduling!
I imagine something like this could work. #NotTested
public class MessageListener
{
private readonly IObservable<IMessage> _messages;
private readonly IScheduler _scheduler;
public MessageListener()
{
_scheduler = new EventLoopScheduler();
var messages = ListenToMessages()
.SubscribeOn(_scheduler)
.Publish();
_messages = messages;
messages.Connect();
}
public IObservable<IMessage> Messages
{
get {return _messages;}
}
private IObservable<IMessage> ListenToMessages()
{
return Observable.Create<IMessage>(o=>
{
return _scheduler.Schedule(recurse=>
{
try
{
var messages = GetMessages();
foreach (var msg in messages)
{
o.OnNext(msg);
}
recurse();
}
catch (Exception ex)
{
o.OnError(ex);
}
});
});
}
private IEnumerable<IMessage> GetMessages()
{
//Do some work here that gets messages from a queue,
// file system, database or other system that cant push
// new data at us.
//
//This may return an empty result when no new data is found.
}
}
The reason I really don't like Subjects, is that is usually a case of the developer not really having a clear design on the problem. Hack in a subject, poke it here there and everywhere, and then let the poor support dev guess at WTF was going on. When you use the Create/Generate etc methods you are localizing the effects on the sequence. You can see it all in one method and you know no-one else is throwing in a nasty side effect. If I see a subject fields I now have to go looking for all the places in a class it is being used. If some MFer exposes one publicly, then all bets are off, who knows how this sequence is being used!
Async/Concurrency/Rx is hard. You don't need to make it harder by allowing side effects and causality programming to spin your head even more.
In general you should avoid using Subject, however for the thing you are doing here I think they work quite well. I asked a similar question when I came across the "avoid subjects" message in Rx tutorials.
To quote Dave Sexton (of Rxx)
"Subjects are the stateful components of Rx. They are useful for when
you need to create an event-like observable as a field or a local
variable."
I tend to use them as the entry point into Rx. So if I have some code that needs to say 'something happened' (like you have), I would use a Subject and call OnNext. Then expose that as an IObservable for others to subscribe to (you can use AsObservable() on your subject to make sure nobody can cast to a Subject and mess things up).
You could also achieve this with a .NET event and use FromEventPattern, but if I'm only going to turn the event into an IObservable anyway, I don't see the benefit of having an event instead of a Subject (which might mean I'm missing something here)
However, what you should avoid quite strongly is subscribing to an IObservable with a Subject, i.e. don't pass a Subject into the IObservable.Subscribe method.
Often when you're managing a Subject, you're actually just reimplementing features already in Rx, and probably in not as robust, simple and extensible a way.
When you're trying to adapt some asynchronous data flow into Rx (or create an asynchronous data flow from one that's not currently asynchronous), the most common cases are usually:
The source of data is an event: As Lee says, this is the simplest case: use FromEvent and head to the pub.
The source of data is from a synchronous operation and you want polled updates, (eg a webservice or database call): In this case you could use Lee's suggested approach, or for simple cases, you could use something like Observable.Interval.Select(_ => <db fetch>). You may want to use DistinctUntilChanged() to prevent publishing updates when nothing has changed in the source data.
The source of data is some kind of asynchronous api that calls your callback: In this case, use Observable.Create to hook up your callback to call OnNext/OnError/OnComplete on the observer.
The source of data is a call that blocks until new data is available (eg some synchronous socket read operations): In this case, you can use Observable.Create to wrap the imperative code that reads from the socket and publishes to the Observer.OnNext when data is read. This may be similar to what you're doing with the Subject.
Using Observable.Create vs creating a class that manages a Subject is fairly equivalent to using the yield keyword vs creating a whole class that implements IEnumerator. Of course, you can write an IEnumerator to be as clean and as good a citizen as the yield code, but which one is better encapsulated and feels a neater design? The same is true for Observable.Create vs managing Subjects.
Observable.Create gives you a clean pattern for lazy setup and clean teardown. How do you achieve this with a class wrapping a Subject? You need some kind of Start method... how do you know when to call it? Or do you just always start it, even when no one is listening? And when you're done, how do you get it to stop reading from the socket/polling the database, etc? You have to have some kind of Stop method, and you have to still have access not just to the IObservable you're subscribed to, but the class that created the Subject in the first place.
With Observable.Create, it's all wrapped up in one place. The body of Observable.Create is not run until someone subscribes, so if no one subscribes, you never use your resource. And Observable.Create returns a Disposable that can cleanly shutdown your resource/callbacks, etc - this is called when the Observer unsubscribes. The lifetimes of the resources you're using to generate the Observable are neatly tied to the lifetime of the Observable itself.
The quoted block text pretty much explains why you shouldn't be using Subject<T>, but to put it simpler, you are combining the functions of observer and observable, while injecting some sort of state in between (whether you're encapsulating or extending).
This is where you run into trouble; these responsibilities should be separate and distinct from each other.
That said, in your specific case, I'd recommend that you break your concerns into smaller parts.
First, you have your thread that is hot, and always monitoring the hardware for signals to raise notifications for. How would you do this normally? Events. So let's start with that.
Let's define the EventArgs that your event will fire.
// The event args that has the information.
public class BaseFrameEventArgs : EventArgs
{
public BaseFrameEventArgs(IBaseFrame baseFrame)
{
// Validate parameters.
if (baseFrame == null) throw new ArgumentNullException("IBaseFrame");
// Set values.
BaseFrame = baseFrame;
}
// Poor man's immutability.
public IBaseFrame BaseFrame { get; private set; }
}
Now, the class that will fire the event. Note, this could be a static class (since you always have a thread running monitoring the hardware buffer), or something you call on-demand which subscribes to that. You'll have to modify this as appropriate.
public class BaseFrameMonitor
{
// You want to make this access thread safe
public event EventHandler<BaseFrameEventArgs> HardwareEvent;
public BaseFrameMonitor()
{
// Create/subscribe to your thread that
// drains hardware signals.
}
}
So now you have a class that exposes an event. Observables work well with events. So much so that there's first-class support for converting streams of events (think of an event stream as multiple firings of an event) into IObservable<T> implementations if you follow the standard event pattern, through the static FromEventPattern method on the Observable class.
With the source of your events, and the FromEventPattern method, we can create an IObservable<EventPattern<BaseFrameEventArgs>> easily (the EventPattern<TEventArgs> class embodies what you'd see in a .NET event, notably, an instance derived from EventArgs and an object representing the sender), like so:
// The event source.
// Or you might not need this if your class is static and exposes
// the event as a static event.
var source = new BaseFrameMonitor();
// Create the observable. It's going to be hot
// as the events are hot.
IObservable<EventPattern<BaseFrameEventArgs>> observable = Observable.
FromEventPattern<BaseFrameEventArgs>(
h => source.HardwareEvent += h,
h => source.HardwareEvent -= h);
Of course, you want an IObservable<IBaseFrame>, but that's easy, using the Select extension method on the Observable class to create a projection (just like you would in LINQ, and we can wrap all of this up in an easy-to-use method):
public IObservable<IBaseFrame> CreateHardwareObservable()
{
// The event source.
// Or you might not need this if your class is static and exposes
// the event as a static event.
var source = new BaseFrameMonitor();
// Create the observable. It's going to be hot
// as the events are hot.
IObservable<EventPattern<BaseFrameEventArgs>> observable = Observable.
FromEventPattern<BaseFrameEventArgs>(
h => source.HardwareEvent += h,
h => source.HardwareEvent -= h);
// Return the observable, but projected.
return observable.Select(i => i.EventArgs.BaseFrame);
}
It is bad to generalize that Subjects are not good to use for a public interface.
While it is certainly true, that this is not the way a reactive programming approach should look like, it is definitively a good improvement/refactoring option for your classic code.
If you have a normal property with an public set accessor and you want to notify about changes, there speaks nothing against replacing it with a BehaviorSubject.
INPC or additional other events are just not that clean and it personally wears me off.
For this purpose you can and should use BehaviorSubjects as public properties instead of normal properties and ditch INPC or other events.
Additionally the Subject-interface makes the users of your interface more aware about the functionality of your properties and are more likely to subscribe instead of just getting the value.
It is the best to use if you want others to listen/subscribe to changes of a property.
I've built my system in c-sharp (winforms) and I've run into a problem. In my view - my graphical interface - I'm starting a pretty heavy algorithm, which in each loop adds a result to a list in my view. The algorithm runs in a presenter (MVP pattern), using a backgroundworker - enabling the view not to freeze. As I said before, the algorithm runs in a loop, and since it's so heavy, I want to process the results of the algorithm as they come in.
View:
...
public List<string> Results { get; }
...
_presenter.RunAlgorithmAsync();
//Start processing results
...
Backgroundworker in presenter:
...
_view.Results.Add(result);
...
To sum it up, how can I start processing the list while the backgroundworker adds to it? Of course, the backgroundworker can work faster than the processing of the list, and vice versa - the processing may have to wait for results to arrive to the list, and the list need to be able up build up a stack of results.
I realize this question may be blurry, but if you ask me questions, I'm sure I can define the problem better.
Use a queue and have the two threads treat it as a producer and consumer.
Make the BackgroundWorker call a method in the view which adds the item to the list and processes it.
Use a threadsafe queue to drive your producer/consumer pattern, such as the .NET 4 ConcurrentQueue: http://www.codethinked.com/post/2010/02/04/NET-40-and-System_Collections_Concurrent_ConcurrentQueue.aspx
Is it something that you can use the ObservableCollection and catch the CollectionChanged event to catch and process each item as it's added to the collection?