To clarify, i have a method:
public static IObservable<Node> GetNodes()
{
var computers = GetComputersInLan();
return computers.Select(computerAddress => GetNode(computerAddress));
}
GetComputersInLan method returns IObservable of IPAddress
private static IObservable<IPAddress> GetComputersInLan()
{
var tasks = new List<Task<PingReply>>();
for (int i = 1; i < 255; i++)
{
Ping p = new Ping();
ipBytes[3] = (byte)(++ipBytes[3]);
IPAddress address = new IPAddress(ipBytes);
tasks.Add(p.SendPingAsync(address, 2000));
}
return tasks.ToObservable().Where(x => x.Result.Status == IPStatus.Success).Select(y => y.Result.Address);
}
GetNode method constructs a Node.
private static Node GetNode(IPAddress ipAddress)
{
return new Node(ipAddress, (IHandler)Activator.GetObject(typeof(Handler), "tcp://" + ipAddress + ":1337/handler"));
}
public class Node
{
private IHandler Handler { get; set; }
public IPAddress Address { get; set; }
public int AvailableCores { get; set; }
public async Task<TResult> Invoke<TResult>(Func<TResult> method)
{
AvailableCores--;
var result = await Task.Run<TResult>(() => Handler.Invoke(method));
AvailableCores++;
return result;
}
}
Handler is a remote computer, and AvailableCores represents its cpu cores.
What I want is to await method GetNodes to return the first Node that has more than 0 AvailableCores.
await GetNodes().FirstAsync(node => node.AvailableCore > 0)
But what happens, is that after enough calls to method Invoke, instead of waiting for cores to become available, it fires an exception "sequence contains no elements".
That is expected behavior for this method. FirstAsync will only check the current state of the items you pass to it, either returning the first match or throwing the exception you are encountering if there is no match.
You will have to manage the case of waiting until a core becomes available yourself. You could try FirstOrDefaultAsync to return null instead of throwing an exception when all cores are busy. From there, you will need some scheme to detect when a core becomes available for the next unit of work, be that an event or polling.
Related
So I am trying to take an IAsyncEnumerable method and relay the results of said method in PowerShell. I understand that PowerShell does not have async support, and generally, people use the Task.GetAwaiter().GetResult() as the means to get their result.
However, this approach does not work (or at least I don't know how to implement it) for IAsyncEnumerable methods.
My specific case is a little more sophisticated, but let's take this example:
namespace ServerMetadataCache.Client.Powershell
{
[Cmdlet(VerbsDiagnostic.Test,"SampleCmdlet")]
[OutputType(typeof(FavoriteStuff))]
public class TestSampleCmdletCommand : PSCmdlet
{
[Parameter(
Mandatory = true,
Position = 0,
ValueFromPipeline = true,
ValueFromPipelineByPropertyName = true)]
public int FavoriteNumber { get; set; } = default!;
[Parameter(
Position = 1,
ValueFromPipelineByPropertyName = true)]
[ValidateSet("Cat", "Dog", "Horse")]
public string FavoritePet { get; set; } = "Dog";
private IAsyncEnumerable<int> InternalMethodAsync()
{
for (int i = 1; i <= 10; i++)
{
await Task.Delay(1000);//Simulate waiting for data to come through.
yield return i;
}
}
protected override void EndProcessing()
{
//this is the issue... how can I tell powershell to read/process results of
//InternalMethodAsync()? Regularly trying to read the method doesn't work
//and neither does wrapping the method with a task and using GetAwaiter().GetResult()
}
public class FavoriteStuff
{
public int FavoriteNumber { get; set; } = default!;
public string FavoritePet { get; set; } = default!;
}
}
This cmdlet is of course a dummy that just takes in a integer and either "Dog", "Cat" or "Horse", but my bigger concern is how to process the InternalMethodAsync() in the Cmdlet. The challenge is getting around the IAsyncEnumerable.
Make an async wrapper method that takes a concurrent collection - like a ConcurrentQueue<int> - as a parameter and fills it with the items from the IAsyncEnumerable<int>, then start reading from it before the task completes:
private async Task InternalMethodExchangeAsync(IProducerConsumerCollection<int> outputCollection)
{
await foreach(var item in InternalMethodAsync())
outputCollection.TryAdd(item)
}
Then in EndProcessing():
protected override void EndProcessing()
{
var resultQueue = new ConcurrentQueue<int>();
var task = InternalMethodExchangeAsync(resultQueue);
// Task still running? Let's try reading from the queue!
while(!task.IsCompleted)
if(resultQueue.TryDequeue(out int preliminaryResult))
WriteObject(preliminaryResult);
// Process remaining items
while(resultQueue.Count > 0)
if(resultQueue.TryDequeue(out int trailingResult))
WriteObject(trailingResult);
}
This question really helped me figure it out.
It Goes back to get Task.GetAwaiter().GetResult() - which works on regular tasks. You need to process the enumerator's results as tasks (which I was trying to do earlier but not in the right way). So based on the InternalMethodAsync() in my question, one could process the results like so:
private void processDummy(){
var enumerator = InternalMethodAsync().GetAsyncEnumerator();
while (enumerator.MoveNextAsync().AsTask().GetAwaiter().GetResult()){
Console.WriteLine(enumerator.Current);
}
}
The AsTask() was the key piece here.
What I need to do
I need to start different instances of a class in a synchronous context using an async method.
Application structure
In my console application I've declared a List<Bot> class:
private List<Bot> _bots = new List<Bot>(new Bot[10]);
the class Bot contains some methods that takes data from internet, so these methods need to be waited. The method structure looks like this:
public class Bot
{
Competition Comp { get; set; }
public async Task StartAsync(int instance)
{
string url = "";
//based on the instance I take the data from different source.
switch(instance)
{
case 0:
url = "www.google.com";
break;
case 1:
url = "www.bing.com";
break;
}
//Comp property contains different groups.
Comp.Groups = await GetCompetitionAsync(Comp, url);
if(Comp.Groups.Count > 0)
{
foreach(var gp in group)
{
//add data inside database.
}
}
}
}
the Competition class have the following design:
public class Competition
{
public string Name { get; set; }
public List<string> Groups { get; set; }
}
I start all the instances of Bot class using the following code:
for(int i = 0; i < _bots.Count - 1; i++)
{
_bots[i].StartAsync(i);
}
this code will call different times StartAsync of Bot class, in this way, I can manage each instance of the bot, and I can eventually stop or start a specific instance in a separate method.
The problem
The method GetCompetitionAsync create a List<string>:
public async Task<List<string>> GetCompetitionAsync(Competition comp, string url)
{
if(comp == null)
comp = new Competition();
List<string> groups = new List<string();
using (var httpResonse = await httpClient.GetAsync(url))
{
string content = await httpResponse.Content.ReadAsStringAsync();
//fill list groups
}
return groups;
}
essentially this method will fill the List<string> available in Comp. Now, if I execute a single instance of StartAsync all works well, but when I run multiple instances (as the for above), the Comp object (which contains the Competition) have all the properties NULL.
So seems that when I have multiple Task running the synchronous context doesn't wait the async context, which in this case fill the List<string>.
When the code reach this line: if(Competition.Groups.Count > 0) I get a NULL exception, because Groups is null, and other Comp properties are NULL.
How can I manage this situation?
UPDATE
After other attempts, I though to create a List<Task> instead of a List<Bot>:
List<Task> tasks = new List<Task>(new Task[10]);
then instead of:
for(int i = 0; i < _bots.Count - 1; i++)
{
_bots[i].StartAsync(i);
}
I did:
for (int i = 0; i < tasks.Count - 1; i++)
{
Console.WriteLine("Starting " + i);
if (tasks[i] == null)
tasks[i] = new Task(async () => await new Bot().StartAsync(i));
apparently all is working well, I got no errors. The problem is: why? I though to something like a deadlock, that I can't even solve using ConfigureAwait(false);.
The last solution also doesn't allow me to access to Bot method because is now a Task.
UPDATE 2
Okay maybe I gotcha the issue. Essentially the await inside the asynchronous method StartAsync is trying to comeback on the main thread, meanwhile the main thread is busy waiting the task to complete, and this will create a deadlock.
That's why moving the StartAsync() inside a List<Task> has worked, 'cause now the async call is now running on a thread pool thread, it doesn't try to comeback to the main thread, and everything seems to works. But I can't use this solution for the reasons explained above.
I'm prefer use Threads instead of Tasks. IMHO, Threads more simple for understanding.
Note: seems that property Bot.Comp in your code is NOT initialized! I fix this issue.
My version of your code:
public class Bot
{
Competition Comp { get; set; }
System.Thread _thread;
private int _instance;
public Bot()
{
Comp = new Competition ();
}
public void Start(int instance)
{
_instance = instance;
_thread = new Thread(StartAsync);
_thread.Start();
}
private void StartAsync()
{
string url = "";
//based on the instance I take the data from different source.
switch(_instance)
{
case 0:
url = "www.google.com";
break;
case 1:
url = "www.bing.com";
break;
}
//Comp property contains different groups.
GetCompetitionAsync(Comp, url);
if(Comp.Groups.Count > 0)
{
foreach(var gp in group)
{
//add data inside database.
}
}
}
public List<string> GetCompetitionAsync(Competition comp, string url)
{
if(comp.groups == null) comp.groups = new List<string>();
using (var httpResonse = httpClient.GetAsync(url))
{
string content = await httpResponse.Content.ReadAsStringAsync();
//fill list groups
}
return groups;
}
}
Then we run threads:
for(int i = 0; i < _bots.Count - 1; i++)
{
_bots[i].Start(i);
}
Each instance of Bot starts method private void StartAsync() in it's own thread.
Note a implementation of method Bot.Start():
public void Start(int instance)
{
_instance = instance;
_thread = new Thread(StartAsync); //At this line: set method Bot.StartAsync as entry point for new thread.
_thread.Start();//At this line: call of _thread.Start() starts new thread and returns **immediately**.
}
This sort of thing is far simpler if you think in terms of lists and "pure" functions-- functions that accept input and return output. Don't pass in something for them to fill or mutate.
For example, this function accepts a string and returns the groups:
List<string> ExtractGroups(string content)
{
var list = new List<string>();
//Populate list
return list;
}
This function accepts a URL and returns its groups.
async Task<List<string>> GetCompetitionAsync(string url)
{
using (var httpResponse = await httpClient.GetAsync(url))
{
string content = await httpResponse.Content.ReadAsStringAsync();
return ExtractGroups(content);
}
}
And this function accepts a list of URLs and returns all of the groups as one list.
async Task<List<string>> GetAllGroups(string[] urls)
{
var tasks = urls.Select( u => GetCompetitionAsync(u) );
await Task.WhenAll(tasks);
return tasks.SelectMany( t => t.Result );
}
You can then stuff the data into the database as you had planned.
var groups = GetAllGroups( new string[] { "www.google.com", "www.bing.com" } );
foreach(var gp in groups)
{
//add data inside database.
}
See how much simpler it is when you break it down this way?
I have a situation where new tasks are being constantly generated and added to a ConcurrentBag<Tasks>.
I need to wait all tasks to complete.
Waiting for all the tasks in the ConcurrentBag via WaitAll is not enough as the number of tasks would have grown while the previous wait is completed.
At the moment I am waiting it in the following way:
private void WaitAllTasks()
{
while (true)
{
int countAtStart = _tasks.Count();
Task.WaitAll(_tasks.ToArray());
int countAtEnd = _tasks.Count();
if (countAtStart == countAtEnd)
{
break;
}
#if DEBUG
if (_tasks.Count() > 100)
{
tokenSource.Cancel();
break;
}
#endif
}
}
I am not very happy with the while(true) solution.
Can anyone suggest a better more efficient way to do this (without having to pool the processor constantly with a while(true))
Additional context information as requested in the comments. I don't think though this is relevant to the question.
This piece of code is used in a web crawler. The crawler scans page content and looks for two type of information. Data Pages and Link Pages. Data pages will be scanned and data will be collected, Link Pages will be scanned and more links will be collected from them.
As each of the tasks carry-on the activities and find more links, they add the links to an EventList. There is an event OnAdd on the list (code below) that is used to trigger other task to scan the newly added URLs. And so forth.
The job is complete when there are no more running tasks (so no more links will be added) and all items have been processed.
public IEventList<ISearchStatus> CurrentLinks { get; private set; }
public IEventList<IDataStatus> CurrentData { get; private set; }
public IEventList<System.Dynamic.ExpandoObject> ResultData { get; set; }
private readonly ConcurrentBag<Task> _tasks = new ConcurrentBag<Task>();
private readonly CancellationTokenSource tokenSource = new CancellationTokenSource();
private readonly CancellationToken token;
public void Search(ISearchDefinition search)
{
CurrentLinks.OnAdd += UrlAdded;
CurrentData.OnAdd += DataUrlAdded;
var status = new SearchStatus(search);
CurrentLinks.Add(status);
WaitAllTasks();
_exporter.Export(ResultData as IList<System.Dynamic.ExpandoObject>);
}
private void DataUrlAdded(object o, EventArgs e)
{
var item = o as IDataStatus;
if (item == null)
{
return;
}
_tasks.Add(Task.Factory.StartNew(() => ProcessObjectSearch(item), token));
}
private void UrlAdded(object o, EventArgs e)
{
var item = o as ISearchStatus;
if (item==null)
{
return;
}
_tasks.Add(Task.Factory.StartNew(() => ProcessFollow(item), token));
_tasks.Add(Task.Factory.StartNew(() => ProcessData(item), token));
}
public class EventList<T> : List<T>, IEventList<T>
{
public EventHandler OnAdd { get; set; }
private readonly object locker = new object();
public new void Add(T item)
{
//lock (locker)
{
base.Add(item);
}
OnAdd?.Invoke(item, null);
}
public new bool Contains(T item)
{
//lock (locker)
{
return base.Contains(item);
}
}
}
I think that this task can be done with TPL Dataflow library with very basic setup. You'll need a TransformManyBlock<Task, IEnumerable<DataTask>> and an ActionBlock (may be more of them) for actual data processing, like this:
// queue for a new urls to parse
var buffer = new BufferBlock<ParseTask>();
// parser itself, returns many data tasks from one url
// similar to LINQ.SelectMany method
var transform = new TransformManyBlock<ParseTask, DataTask>(task =>
{
// get all the additional urls to parse
var parsedLinks = GetLinkTasks(task);
// get all the data to parse
var parsedData = GetDataTasks(task);
// setup additional links to be parsed
foreach (var parsedLink in parsedLinks)
{
buffer.Post(parsedLink);
}
// return all the data to be processed
return parsedData;
});
// actual data processing
var consumer = new ActionBlock<DataTask>(s => ProcessData(s));
After that you need to link the blocks between each over:
buffer.LinkTo(transform, new DataflowLinkOptions { PropagateCompletion = true });
transform.LinkTo(consumer, new DataflowLinkOptions { PropagateCompletion = true });
Now you have a nice pipeline which will execute in background. At the moment you realize that everything you need is parsed, you simply call the Complete method for a block so it stops accepting news messages. After the buffer became empty, it will propagate the completion down the pipeline to transform block, which will propagate it down to consumer(s), and you need to wait for Completion task:
// no additional links would be accepted
buffer.Complete();
// after all the tasks are done, this will get fired
await consumer.Completion;
You can check the moment for a completion, for example, if both buffer' Count property and transform' InputCount and transform' CurrentDegreeOfParallelism (this is internal property for the TransformManyBlock) are equal to 0.
However, I suggested you to implement some additional logic here to determine current transformers number, as using the internal logic isn't a great solution. As for cancelling the pipeline, you can create a TPL block with a CancellationToken, either the one for all, or a dedicated for each block, getting the cancellation out of box.
Why not write one function that yields your tasks as necessary, when they are created? This way you can just use Task.WhenAll to wait for them to complete or, have I missed the point? See this working here.
using System;
using System.Threading.Tasks;
using System.Collections.Generic;
public class Program
{
public static void Main()
{
try
{
Task.WhenAll(GetLazilyGeneratedSequenceOfTasks()).Wait();
Console.WriteLine("Fisnished.");
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
public static IEnumerable<Task> GetLazilyGeneratedSequenceOfTasks()
{
var random = new Random();
var finished = false;
while (!finished)
{
var n = random.Next(1, 2001);
if (n < 50)
{
finished = true;
}
if (n > 499)
{
yield return Task.Delay(n);
}
Task.Delay(20).Wait();
}
yield break;
}
}
Alternatively, if your question is not as trivial as my answer may suggest, I'd consider a mesh with TPL Dataflow. The combination of a BufferBlock and an ActionBlock would get you very close to what you need. You could start here.
Either way, I'd suggest you want to include a provision for accepting a CancellationToken or two.
If I have an INPC supporting class Numbers with two properties A and B. I can write code like
Numbers numbers = new Numbers();
IObservable<double> o = numbers.WhenAnyValue(p=>p.A,p=>p.B,(a,b)=>a/b);
WhenAnyValue is a utility method in the ReactiveUI library for composing observables from property change events. If I then write.
o.Subscribe(v=>Console.WriteLine(v));
it will print a/b whenever A or B changes. This is all good until I set
numbers.B = 0;
Now a/b will throw a DivideByZeroException and the observable will terminate. However this is a UI. I don't want the observable to terminate. I just either wish to ignore the exception or log it and move on. First attempt is to see that IObservable contains an extension method called Retry which will reconnect to the observable after an exception. We try
Numbers numbers = new Numbers();
IObservable<double> o = numbers
.WhenAnyValue(p=>p.A,p=>p.B,(a,b)=>a/b)
.Retry();
o.Subscribe(v=>Console.WriteLine(v));
However when I do numbers.B = 0 then the Retry will ignore the exception and reconnect and will immediately fail again and again and again because WhenAnyValue always delivers an event on subscription.
So it seems what I need is a Retry that will ignore the first input after reconnection iff it is the same as the input that caused the error that disconnected the first one except I don't think this is possible with RX.
Any ideas?
Full Test Case
The below test case does not terminate.
public class Numbers : ReactiveObject
{
int _A;
public int A
{
get { return _A; }
set { this.RaiseAndSetIfChanged(ref _A, value); }
}
int _B;
public int B
{
get { return _B; }
set { this.RaiseAndSetIfChanged(ref _B, value); }
}
}
[Fact]
public void TestShouldTerminate()
{
var numbers = new Numbers();
var o = numbers
.WhenAnyValue(p => p.A, p => p.B, Tuple.Create)
.Select(v=>v.Item1/v.Item2)
.Select(v=>v+1)
.Retry();
double value = 0;
o.Subscribe(v => value = v);
numbers.A = 10;
numbers.B = 20;
value.Should().Be(1.5);
}
}
}
}
This can't be handled in vanilla RX. I've created a wrapper called
IObservableExceptional
IObserverExceptional
that changes the standard contract for error handling in RX. Errors now no longer terminate the observable. It supports LINQ and should be fairly transparent for most uses. The test case that passes is
[Fact]
public void ErrorsCanBePropogated()
{
var numbers = new Numbers();
var list = new List<double>();
var errors = new List<Exception>();
numbers
.WhenAnyValue(p => p.A, p => p.B, Tuple.Create)
.ToObservableExceptional()
.Select(v => v.Item1/v.Item2)
.Subscribe(onNext: val=>list.Add(val), onError:err=>errors.Add(err));
list.Count.Should().Be(0);
errors.Count.Should().Be(1);
numbers.A = 10;
list.Count.Should().Be(0);
errors.Count.Should().Be(2);
numbers.B = 5;
list.Count.Should().Be(1);
list[0].Should().Be(2.0);
errors.Count.Should().Be(2);
}
The three new interfaces are
public interface IObservableExceptional<T>
{
void Subscribe(IObserverExceptional<T> observer);
IObservable<IExceptional<T>> Observable { get; }
}
public interface IObserverExceptional<T>
{
void OnNext(IExceptional<T> t);
void OnCompleted();
IObserver<IExceptional<T>> Observer { get; }
}
public interface IExceptional<out T> : IEnumerable<T>
{
bool HasException { get; }
Exception Exception { get; }
T Value { get; }
string ToMessage();
void ThrowIfHasException();
}
IObservableException and IExceptional both support LINQ ( ie they are Monads )
Any exceptions thrown within the Select or SelectMany combinators of IObservableExceptional are wrapped as IExceptional objects and passed on to the subscriber. An error does not terminate the subscription.
The repository is at
https://github.com/Weingartner/Exceptional
Update - solved
The final solution differs a bit from Brandon's suggestion but his answer brought me on the right track.
class State
{
public int Offset { get; set; }
public HashSet<string> UniqueImageUrls = new HashSet<string>();
}
public IObservable<TPicture> GetPictures(ref object _state)
{
var localState = (State) _state ?? new State();
_state = localState;
return Observable.Defer(()=>
{
return Observable.Defer(() => Observable.Return(GetPage(localState.Offset)))
.SubscribeOn(TaskPoolScheduler.Default)
.Do(x=> localState.Offset += 20)
.Repeat()
.TakeWhile(x=> x.Count > 0)
.SelectMany(x=> x)
.Where(x=> !localState.UniqueImageUrls.Contains(x.ImageUrl))
.Do(x=> localState.UniqueImageUrls.Add(x.ImageUrl));
});
}
IList<TPicture> GetPage(int offset)
{
...
return result;
}
Original Question
I'm currently struggling with the following problem. The PictureProvider implementation shown below is working with an offset variable used for paging results of a backend service providing the actual data. What I would like to implement is an elegant solution making the current offset available to the consumer of the observable to allow for resuming the observable sequence at a later time at the correct offset. Resuming is already accounted for by the intialState argument to GetPictures().
Recommendations for improving the code in a more RX like fashion would be welcome as well. I'm actually not so sure if the Task.Run() stuff is appropriate here.
public class PictureProvider :
IPictureProvider<Picture>
{
#region IPictureProvider implementation
public IObservable<Picture> GetPictures(object initialState)
{
return Observable.Create<Picture>((IObserver<Picture> observer) =>
{
var state = new ProducerState(initialState);
ProducePictures(observer, state);
return state;
});
}
#endregion
void ProducePictures(IObserver<Picture> observer, ProducerState state)
{
Task.Run(() =>
{
try
{
while(!state.Terminate.WaitOne(0))
{
var page = GetPage(state.Offset);
if(page.Count == 0)
{
observer.OnCompleted();
break;
}
else
{
foreach(var picture in page)
observer.OnNext(picture);
state.Offset += page.Count;
}
}
}
catch (Exception ex)
{
observer.OnError(ex);
}
state.TerminateAck.Set();
});
}
IList<Picture> GetPage(int offset)
{
var result = new List<Picture>();
... boring web service call here
return result;
}
public class ProducerState :
IDisposable
{
public ProducerState(object initialState)
{
Terminate = new ManualResetEvent(false);
TerminateAck = new ManualResetEvent(false);
if(initialState != null)
Offset = (int) initialState;
}
public ManualResetEvent Terminate { get; private set; }
public ManualResetEvent TerminateAck { get; private set; }
public int Offset { get; set; }
#region IDisposable implementation
public void Dispose()
{
Terminate.Set();
TerminateAck.WaitOne();
Terminate.Dispose();
TerminateAck.Dispose();
}
#endregion
}
}
I suggest refactoring your interface to yield the state as part of the data. Now the client has what they need to resubscribe where they left off.
Also, once you start using Rx, you should find that using synchronization primitives like ManualResetEvent are rarely necessary. If you refactor your code so that retrieving each page is its own Task, then you can eliminate all of that synchronization code.
Also, if you are calling a "boring web service" in GetPage, then just make it async. This gets rid of the need to call Task.Run among other benefits.
Here is a refactored version, using .NET 4.5 async/await syntax. It could also be done without async/await. I also added a GetPageAsync method that uses Observable.Run just in case you really cannot convert your webservice call to be asynchronous
/// <summary>A set of pictures</summary>
public struct PictureSet
{
public int Offset { get; private set; }
public IList<Picture> Pictures { get; private set; }
/// <summary>Clients will use this property if they want to pick up where they left off</summary>
public int NextOffset { get { return Offset + Pictures.Count; } }
public PictureSet(int offset, IList<Picture> pictures)
:this() { Offset = offset; Pictures = pictures; }
}
public class PictureProvider : IPictureProvider<PictureSet>
{
public IObservable<PictureSet> GetPictures(int offset = 0)
{
// use Defer() so we can capture a copy of offset
// for each observer that subscribes (so multiple
// observers do not update each other's offset
return Observable.Defer<PictureSet>(() =>
{
var localOffset = offset;
// Use Defer so we re-execute GetPageAsync()
// each time through the loop.
// Update localOffset after each GetPageAsync()
// completes so that the next call to GetPageAsync()
// uses the next offset
return Observable.Defer(() => GetPageAsync(localOffset))
.Select(pictures =>
{
var s = new PictureSet(localOffset, pictures);
localOffset += pictures.Count;
})
.Repeat()
.TakeWhile(pictureSet => pictureSet.Pictures.Count > 0);
});
}
private async Task<IList<Picture>> GetPageAsync(int offset)
{
var data = await BoringWebServiceCallAsync(offset);
result = data.Pictures.ToList();
}
// this version uses Observable.Run() (which just uses Task.Run under the hood)
// in case you cannot convert your
// web service call to be asynchronous
private IObservable<IList<Picture>> GetPageAsync(int offset)
{
return Observable.Run(() =>
{
var result = new List<Picture>();
... boring web service call here
return result;
});
}
}
Clients just need to add a SelectMany call to get their IObservable<Picture>. They can choose to store the pictureSet.NextOffset if they wish.
pictureProvider
.GetPictures()
.SelectMany(pictureSet => pictureSet.Pictures)
.Subscribe(picture => whatever);
Instead of thinking about how to save the subscription state, I would think about how to replay the state of the inputs (i.e. I'd try to create a serializable ReplaySubject that, on resume, would just resubscribe and catch back up to the current state).