Let's say I have a data service class that fetches the data batch by batch and its chunks to the subscribers.
public class DataService {
public IObservable<IList<T>> QuerySegmentedObservable<T>(string tableName) where T : TableEntity, new(){
return Observable.Create<IList<T>>(async (observer, token) =>{
TableContinuationToken continuationToken = null;
do{
var currentSegment = CallData();
observer.OnNext(currentSegment.Results);
continuationToken = currentSegment.ContinuationToken;
} while (continuationToken != null);
observer.OnCompleted();
}
}
}
I am subscribing this observable as below.
public async Task<bool> MyMethod()
{
var tcs = new TaskCompletionSource<bool>();
var observable = _dataService.QuerySegmentedObservable<TSource>(_sourceTableName);
var dataCount = 0;
_databaseService.OpenConnection();
observable.Subscribe(async data =>
{
await _databaseService.DoSomething(data);
dataCount += data.Count;
Console.WriteLine($"Processing - {dataCount}");
},
err =>
{
Console.WriteLine($"Error - {err.Message}");
tcs.SetResult(false);
},
() =>
{
_databaseService.CloseConnection();
Console.WriteLine($"Finished");
tcs.SetResult(true);
}
);
return await tcs.Task;
}
The problem is that OnComplete() is called before the last OnNext() is finished. So, I ended up closing the connection before finishing the task that I am doing in Subscribe();
Is there any way to fix it? Thanks.
Rx does support async/await within operators. You're using it though within a subscription. So (hopefully) you can change your code to something like this:
public async Task<bool> MyMethod()
{
var tcs = new TaskCompletionSource<bool>();
_databaseService.OpenConnection();
var dataCount = 0;
_dataService.QuerySegmentedObservable<TSource>(_sourceTableName)
.SelectMany(async data =>
{
await _databaseService.DoSomething(data);
return data;
})
//.Finally(() => _databaseService.CloseConnection()) //This would be called on OnComplete and OnError, just like try-finally
.Subscribe(data =>
{
dataCount += data.Count;
Console.WriteLine($"Processing - {dataCount}");
},
err =>
{
Console.WriteLine($"Error - {err.Message}");
tcs.SetResult(false);
},
() =>
{
_databaseService.CloseConnection(); //Maybe move this to a Finally call?
Console.WriteLine($"Finished");
tcs.SetResult(true);
}
);
return await tcs.Task;
}
I can't really test it, so I hope that sets you on the right path. If you need more help, then please post a better MCVE.
Related
I am trying to use asynchronous programming with async and await to calculate some values and save in a database periodically using ASP.NET Core 2.x. The method shown below is not executing from the thread pool.
public async void CalculateCIndex(int CIndexId)
{
List<string> UserIds = _context.ApplicationUsers
.Where(m => m.EmailConfirmed == true)
.Select(m => m.Id).ToList();
for (int i = 0; i < UserIds.Count; i += 10)
{
var temp = UserIds.Skip(i).Take(10);
foreach (var Id in temp)
{
await Task.Run(async () =>
{
await CIndexCal(Id, CIndexId);
});
}
await Task.Delay(6000);
}
}
public async Task CIndexCal(string Id, int CIndexId)
{
_context.CIndexMember.Add(new CIndexMember
{
NCIndex = (decimal)((AindexAfterRar + Power) / CI.TotalCindex)
});
_context.SaveChanges();
}
You can find good answer here:
async/await - when to return a Task vs void?
Your method CIndexCal is declared as async but it is not async at all due to fact you are not using async methods inside. In order to change this you could change:
_context.SaveChanges();
to
await _context.SaveChangesAsync();
Then instead of:
await Task.Run(async () =>
{
await CIndexCal(Id, CIndexId);
});
you just need:
await CIndexCal(Id, CIndexId);
and I am not sure why you are making delay and I think you don't need this:
await Task.Delay(6000);
I am starting 2 channels in the mediaservices azure portal.
Starting a channel takes a long time to complete, about 25-30 seconds per channel. Hence, multithreading :)
However, the following is not clear to me:
I have 2 methods:
public async Task<bool> StartAsync(string programName, CancellationToken token = default(CancellationToken))
{
var workerThreads = new List<Thread>();
var results = new List<bool>();
foreach (var azureProgram in _accounts.GetPrograms(programName))
{
var thread = new Thread(() =>
{
var result = StartChannelAsync(azureProgram).Result;
lock (results)
{
results.Add(result);
}
});
workerThreads.Add(thread);
thread.Start();
}
foreach (var thread in workerThreads)
{
thread.Join();
}
return results.All(r => r);
}
and
private async Task<bool> StartChannelAsync(IProgram azureProgram)
{
var state = _channelFactory.ConvertToState(azureProgram.Channel.State);
if (state == State.Running)
{
return true;
}
if (state.IsTransitioning())
{
return false;
}
await azureProgram.Channel.StartAsync();
return true;
}
in the first method I use
var result = StartChannelAsync(azureProgram).Result;
In this case everything works fine. But if I use
var result = await StartChannelAsync(azureProgram);
Executing is not awaited and my results has zero entries.
What am I missing here?
And is this a correct way?
Any comments on the code is appreciated. I am not a multithreading king ;)
Cheers!
Don't span new Thread instances to execute tasks in parallel, instead use Task.WhenAll:
public async Task<bool> StartAsync(string programName, CancellationToken token = default(CancellationToken))
{
// Create a task for each program and fire them "at the same time"
Task<bool>[] startingChannels = _accounts.GetPrograms(programName))
.Select(n => StartChannelAsync(n))
.ToArray();
// Create a task that will be completed when all the supplied tasks are done
bool[] results = await Task.WhenAll(startingChannels);
return results.All(r => r);
}
Note: I see that you're passing a CancellationToken to your StartAsync method, but you're not actually using it. Consider passing it as an argument to StartChannelAsync, and then use it when calling azureProgram.Channel.StartAsync
If you love one-liners:
public async Task<bool> StartAsync(string programName, CancellationToken token = default(CancellationToken))
{
return (await Task.WhenAll(_accounts.GetPrograms(programName)
.Select(p => StartChannelAsync(p))
.ToArray())).All(r => r);
}
I want to execute several asynchronous tasks concurrently. Each task will run an HTTP request that can either complete successfully or throw an exception. I need to await until the first task completes successfully, or until all the tasks have failed.
How can I implement an overload of the Task.WhenAny method that accepts a predicate, so that I can exclude the non-successfully completed tasks?
Wait for any task and return the task if the condition is met. Otherwise wait again for the other tasks until there is no more task to wait for.
public static async Task<Task> WhenAny( IEnumerable<Task> tasks, Predicate<Task> condition )
{
var tasklist = tasks.ToList();
while ( tasklist.Count > 0 )
{
var task = await Task.WhenAny( tasklist );
if ( condition( task ) )
return task;
tasklist.Remove( task );
}
return null;
}
simple check for that
var tasks = new List<Task> {
Task.FromException( new Exception() ),
Task.FromException( new Exception() ),
Task.FromException( new Exception() ),
Task.CompletedTask, };
var completedTask = WhenAny( tasks, t => t.Status == TaskStatus.RanToCompletion ).Result;
if ( tasks.IndexOf( completedTask ) != 3 )
throw new Exception( "not expected" );
public static Task<T> GetFirstResult<T>(
ICollection<Func<CancellationToken, Task<T>>> taskFactories,
Predicate<T> predicate) where T : class
{
var tcs = new TaskCompletionSource<T>();
var cts = new CancellationTokenSource();
int completedCount = 0;
// in case you have a lot of tasks you might need to throttle them
//(e.g. so you don't try to send 99999999 requests at the same time)
// see: http://stackoverflow.com/a/25877042/67824
foreach (var taskFactory in taskFactories)
{
taskFactory(cts.Token).ContinueWith(t =>
{
if (t.Exception != null)
{
Console.WriteLine($"Task completed with exception: {t.Exception}");
}
else if (predicate(t.Result))
{
cts.Cancel();
tcs.TrySetResult(t.Result);
}
if (Interlocked.Increment(ref completedCount) == taskFactories.Count)
{
tcs.SetException(new InvalidOperationException("All tasks failed"));
}
}, cts.Token);
}
return tcs.Task;
}
Sample usage:
using System.Net.Http;
var client = new HttpClient();
var response = await GetFirstResult(
new Func<CancellationToken, Task<HttpResponseMessage>>[]
{
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
ct => client.GetAsync("http://microsoft123456.com", ct),
},
rm => rm.IsSuccessStatusCode);
Console.WriteLine($"Successful response: {response}");
public static Task<Task<T>> WhenFirst<T>(IEnumerable<Task<T>> tasks, Func<Task<T>, bool> predicate)
{
if (tasks == null) throw new ArgumentNullException(nameof(tasks));
if (predicate == null) throw new ArgumentNullException(nameof(predicate));
var tasksArray = (tasks as IReadOnlyList<Task<T>>) ?? tasks.ToArray();
if (tasksArray.Count == 0) throw new ArgumentException("Empty task list", nameof(tasks));
if (tasksArray.Any(t => t == null)) throw new ArgumentException("Tasks contains a null reference", nameof(tasks));
var tcs = new TaskCompletionSource<Task<T>>();
var count = tasksArray.Count;
Action<Task<T>> continuation = t =>
{
if (predicate(t))
{
tcs.TrySetResult(t);
}
if (Interlocked.Decrement(ref count) == 0)
{
tcs.TrySetResult(null);
}
};
foreach (var task in tasksArray)
{
task.ContinueWith(continuation);
}
return tcs.Task;
}
Sample usage:
var task = await WhenFirst(tasks, t => t.Status == TaskStatus.RanToCompletion);
if (task != null)
var value = await task;
Note that this doesn't propagate exceptions of failed tasks (just as WhenAny doesn't).
You can also create a version of this for the non-generic Task.
Here is an attempted improvement of the excellent Eli Arbel's answer. These are the improved points:
An exception in the predicate is propagated as a fault of the returned task.
The predicate is not called after a task has been accepted as the result.
The predicate is executed in the original SynchronizationContext. This makes it possible to access UI elements (if the WhenFirst method is called from a UI thread)
The source IEnumerable<Task<T>> is enumerated directly, without being converted to an array first.
public static Task<Task<T>> WhenFirst<T>(IEnumerable<Task<T>> tasks,
Func<Task<T>, bool> predicate)
{
if (tasks == null) throw new ArgumentNullException(nameof(tasks));
if (predicate == null) throw new ArgumentNullException(nameof(predicate));
var tcs = new TaskCompletionSource<Task<T>>(
TaskCreationOptions.RunContinuationsAsynchronously);
var pendingCount = 1; // The initial 1 represents the enumeration itself
foreach (var task in tasks)
{
if (task == null) throw new ArgumentException($"The {nameof(tasks)}" +
" argument included a null value.", nameof(tasks));
Interlocked.Increment(ref pendingCount);
HandleTaskCompletion(task);
}
if (Interlocked.Decrement(ref pendingCount) == 0) tcs.TrySetResult(null);
return tcs.Task;
async void HandleTaskCompletion(Task<T> task)
{
try
{
await task; // Continue on the captured context
}
catch { } // Ignore exception
if (tcs.Task.IsCompleted) return;
try
{
if (predicate(task))
tcs.TrySetResult(task);
else
if (Interlocked.Decrement(ref pendingCount) == 0)
tcs.TrySetResult(null);
}
catch (Exception ex)
{
tcs.TrySetException(ex);
}
}
}
Another way of doing this, very similar to Sir Rufo's answer, but using AsyncEnumerable and Ix.NET
Implement a little helper method to stream any task as soon as it's completed:
static IAsyncEnumerable<Task<T>> WhenCompleted<T>(IEnumerable<Task<T>> source) =>
AsyncEnumerable.Create(_ =>
{
var tasks = source.ToList();
Task<T> current = null;
return AsyncEnumerator.Create(
async () => tasks.Any() && tasks.Remove(current = await Task.WhenAny(tasks)),
() => current,
async () => { });
});
}
One can then process the tasks in completion order, e.g. returning the first matching one as requested:
await WhenCompleted(tasks).FirstOrDefault(t => t.Status == TaskStatus.RanToCompletion)
Just wanted to add on some of the answers #Peebo and #SirRufo that are using List.Remove (because I can't comment yet)
I would consider using:
var tasks = source.ToHashSet();
instead of:
var tasks = source.ToList();
so removing would be more efficient
Something is definitely flawed in my understanding of async/await. I want a piece of code named SaveSearchCase to run asynchronously in background.
I want it to be fired and forget about it and continue with the current method's return statement.
public IList<Entities.Case.CreateCaseOutput> createCase(ARC.Donor.Data.Entities.Case.CreateCaseInput CreateCaseInput, ARC.Donor.Data.Entities.Case.SaveCaseSearchInput SaveCaseSearchInput)
{
..........
..........
..........
var AcctLst = rep.ExecuteStoredProcedure<Entities.Case.CreateCaseOutput>(strSPQuery, listParam).ToList();
if (!string.IsNullOrEmpty(AcctLst.ElementAt(0).o_case_seq.ToString()))
{
Task<IList<Entities.Case.SaveCaseSearchOutput>> task = saveCaseSearch(SaveCaseSearchInput, AcctLst.ElementAt(0).o_case_seq);
Task t = task.ContinueWith(
r => { Console.WriteLine(r.Result); }
);
}
Console.WriteLine("After the async call");
return AcctLst;
}
And the SaveCaseSearch looks like
public async Task<IList<Entities.Case.SaveCaseSearchOutput>> saveCaseSearch(ARC.Donor.Data.Entities.Case.SaveCaseSearchInput SaveCaseSearchInput,Int64? case_key)
{
Repository rep = new Repository();
string strSPQuery = string.Empty;
List<object> listParam = new List<object>();
SQL.CaseSQL.getSaveCaseSearchParameters(SaveCaseSearchInput, case_key,out strSPQuery, out listParam);
var AcctLst = await rep.ExecuteStoredProcedureAsync<Entities.Case.SaveCaseSearchOutput>(strSPQuery, listParam);
return (System.Collections.Generic.IList<ARC.Donor.Data.Entities.Case.SaveCaseSearchOutput>)AcctLst;
}
But when I see the debugger createCase method waits for SaveCaseSearch to complete first and then only
it prints "After Async Call "
and then returns . Which I do not want definitely .
So which way is my understanding flawed ? Please help to make it run async and continue with current method's print and return statement .
UPDATE
I updated the SaveCaseSearch method to reflect like :
public async Task<IList<Entities.Case.SaveCaseSearchOutput>> saveCaseSearch(ARC.Donor.Data.Entities.Case.SaveCaseSearchInput SaveCaseSearchInput,Int64? case_key)
{
return Task.Run<IList<Entities.Case.SaveCaseSearchOutput>>(async (SaveCaseSearchInput, case_key) =>
{
Repository rep = new Repository();
string strSPQuery = string.Empty;
List<object> listParam = new List<object>();
SQL.CaseSQL.getSaveCaseSearchParameters(SaveCaseSearchInput, case_key, out strSPQuery, out listParam);
var AcctLst = await rep.ExecuteStoredProcedureAsync<Entities.Case.SaveCaseSearchOutput>(strSPQuery, listParam);
return (System.Collections.Generic.IList<ARC.Donor.Data.Entities.Case.SaveCaseSearchOutput>)AcctLst;
});
}
But there is something wrong with the params. It says
Error 4 A local variable named 'SaveCaseSearchInput' cannot be declared in this scope because it would give a different meaning to 'SaveCaseSearchInput', which is already used in a 'parent or current' scope to denote something else C:\Users\m1034699\Desktop\Stuart_V2_12042016\Stuart Web Service\ARC.Donor.Data\Case\Search.cs 43 79 ARC.Donor.Data
Well this saveCaseSearch() method runs synchronously in main thread and this is the main problem here. Instead of returning result with a task you should return Task with operation itself. Here is some simplified example :
Runs synchronously and waits 5 seconds
public IList<int> A()
{
var AcctLst = new List<int> { 0, 2, 5, 8 };
if (true)
{
Task<IList<int>> task = saveCaseSearch();
Task t = task.ContinueWith(
r => { Console.WriteLine(r.Result[0]); }
);
}
Console.WriteLine("After the async call");
return AcctLst;
}
// runs sync and in the end returns Task that is never actually fired
public async Task<IList<int>> saveCaseSearch()
{
Thread.Sleep(5000);
return new List<int>() { 10, 12, 16 };
}
Runs asynchronously - fires task & forgets :
public IList<int> A()
{
... same code as above
}
// notice that we removed `async` keyword here because we just return task.
public Task<IList<int>> saveCaseSearch()
{
return Task.Run<IList<int>>(() =>
{
Thread.Sleep(5000);
return new List<int>() { 10, 12, 16 };
});
}
Here is full code for this example
Against all that I believe in pertaining to "fire-and-forget" you can do this by writing your code this way:
public Task<SaveCaseSearchOutput> SaveCaseSearch(
SaveCaseSearchInput saveCaseSearchInput,
long? caseKey)
{
var rep = new Repository();
var query = string.Empty;
var listParam = new List<object>();
SQL.CaseSQL
.getSaveCaseSearchParameters(
saveCaseSearchInput,
caseKey,
out query,
out listParam);
return rep.ExecuteStoredProcedureAsync<SaveCaseSearchOutput>(
strSPQuery,
istParam);
}
And then if the place where you would like to fire it and log when it returns (which is really what you have -- so you're not forgetting about it), do this:
public IList<CreateCaseOutput> CreateCase(
CreateCaseInput createCaseInput,
SaveCaseSearchInput saveCaseSearchInput)
{
// Omitted for brevity...
var AcctLst =
rep.ExecuteStoredProcedure<CreateCaseOutput>(
strSPQuery,
listParam)
.ToList();
if (!string.IsNullOrEmpty(AcctLst.ElementAt(0).o_case_seq.ToString()))
{
SaveCaseSearch(saveCaseSearchInput,
AcctLst.ElementAt(0).o_case_seq)
.ContinueWith(r => Console.WriteLine(r.Result));
}
Console.WriteLine("After the async call");
return AcctLst;
}
The issue was that you were using async and await in the SaveSearchCase function, and this basically means that your code is the opposite of "fire-and-forget".
As a side note, you should really just use async and await, and avoid the "fire-and-forget" idea! Make your DB calls asynchronous and leverage this paradigm for what it's worth!
Consider the following:
The SaveCaseSearch call can stay as I have defined it above.
public Task<SaveCaseSearchOutput> SaveCaseSearch(
SaveCaseSearchInput saveCaseSearchInput,
long? caseKey)
{
var rep = new Repository();
var query = string.Empty;
var listParam = new List<object>();
SQL.CaseSQL
.getSaveCaseSearchParameters(
saveCaseSearchInput,
caseKey,
out query,
out listParam);
return rep.ExecuteStoredProcedureAsync<SaveCaseSearchOutput>(
strSPQuery,
istParam);
}
Then in your call to it, do this instead:
public async Task<IList<CreateCaseOutput>> CreateCase(
CreateCaseInput createCaseInput,
SaveCaseSearchInput saveCaseSearchInput)
{
// Omitted for brevity...
var AcctLst =
await rep.ExecuteStoredProcedureAsync<CreateCaseOutput>(
strSPQuery,
listParam)
.ToList();
if (!string.IsNullOrEmpty(AcctLst.ElementAt(0).o_case_seq.ToString()))
{
await SaveCaseSearch(saveCaseSearchInput,
AcctLst.ElementAt(0).o_case_seq)
.ContinueWith(r => Console.WriteLine(r.Result));
}
Console.WriteLine("After the async call");
return AcctLst;
}
This makes for a much better solution!
I'll start off by publishing the code that is troubled:
public async Task main()
{
Task t = func();
await t;
list.ItemsSource = jlist; //jlist previously defined
}
public async Task func()
{
TwitterService service = new TwitterService(_consumerKey, _consumerSecret);
service.AuthenticateWith(_accessToken, _accessTokenSecret);
TwitterGeoLocationSearch g = new TwitterGeoLocationSearch(40.758367, -73.982706, 25, 0);
SearchOptions s = new SearchOptions();
s.Geocode = g;
s.Q = "";
s.Count = 1;
service.Search(s, (statuses, response) => get_tweets(statuses, response));
void get_tweets(TwitterSearchResult statuses, TwitterResponse response)
{
//unimportant code
jlist.Add(info);
System.Diagnostics.Debug.WriteLine("done with get_tweets, jlist created");
}
I am having issues with the get_tweets(..) function running (on what I believe a different thread) and the Task t is not awaited like I have in the main function. Basically, my issue is that the list.Itemsource = jlist is ran before the get_tweets function is finished. Does anyone have a solution or the right direction to point me in?
First, create a TAP wrapper for TwitterService.Search, using TaskCompletionSource. So something like:
public static Task<Tuple<TwitterSearchResult, TwitterResponse>> SearchAsync(this TwitterService service, SearchOptions options)
{
var tcs = new TaskCompletionSource<Tuple<TwitterSearchResult, TwitterResponse>>();
service.Search(options, (status, response) => tcs.SetResult(Tuple.Create(status, response)));
return tcs.Task;
}
Then you can consume it using await:
SearchOptions s = new SearchOptions();
s.Geocode = g;
s.Q = "";
s.Count = 1;
var result = await service.SearchAsync(s);
get_tweets(result.Item1, result.Item2);