I'm working on a .Net core solution that takes backup of storage files from another microservice and because this process takes too long time, we decided to build this routine under a background task.By following this link:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-2.1
I have implemented the background by using Queued background tasks like the following :
public interface IBackgroundTaskQueue
{
void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem);
Task<Func<CancellationToken, Task>> DequeueAsync(
CancellationToken cancellationToken);
}
public class BackgroundTaskQueue : IBackgroundTaskQueue
{
private ConcurrentQueue<Func<CancellationToken, Task>> _workItems =
new ConcurrentQueue<Func<CancellationToken, Task>>();
private SemaphoreSlim _signal = new SemaphoreSlim(0);
public void QueueBackgroundWorkItem(
Func<CancellationToken, Task> workItem)
{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
_workItems.Enqueue(workItem);
_signal.Release();
}
public async Task<Func<CancellationToken, Task>> DequeueAsync(
CancellationToken cancellationToken)
{
await _signal.WaitAsync(cancellationToken);
_workItems.TryDequeue(out var workItem);
return workItem;
}
}
public class QueuedHostedService : BackgroundService
{
private readonly ILogger _logger;
public QueuedHostedService(IBackgroundTaskQueue taskQueue,
ILoggerFactory loggerFactory)
{
TaskQueue = taskQueue;
_logger = loggerFactory.CreateLogger<QueuedHostedService>();
}
public IBackgroundTaskQueue TaskQueue { get; }
protected async override Task ExecuteAsync(
CancellationToken cancellationToken)
{
_logger.LogInformation("Queued Hosted Service is starting.");
while (!cancellationToken.IsCancellationRequested)
{
var workItem = await TaskQueue.DequeueAsync(cancellationToken);
try
{
await workItem(cancellationToken);
}
catch (Exception ex)
{
_logger.LogError(ex,
$"Error occurred executing {nameof(workItem)}.");
}
}
_logger.LogInformation("Queued Hosted Service is stopping.");
}
}
}
and in the controller action method I did that:
[HttpPost]
[ValidateAntiForgeryToken]
public IActionResult TakeBackup()
{
// Process #1: update latest backup time in setting table.
var _setting = _settingService.FindByKey("BackupData");
var data = JsonConvert.DeserializeObject<BackUpData>(_setting.Value);
data.LatestBackupTime = DateTime.UtcNow;
_setting.Value = JsonConvert.SerializeObject(data);
_settingService.AddOrUpdate(_setting);
// Process #2: Begin a background service to excaute the backup task.
_queue.QueueBackgroundWorkItem(async token =>
{
// instead of this staff I will replace by the API I want to consume.
var guid = Guid.NewGuid().ToString();
for (int delayLoop = 0; delayLoop < 3; delayLoop++)
{
_logger.LogInformation(
$"Queued Background Task {guid} is running. {delayLoop}/3");
await Task.Delay(TimeSpan.FromSeconds(5), token);
}
_logger.LogInformation(
$"Queued Background Task {guid} is complete. 3/3");
// Here I need to redirect to the index view after the task is finished (my issue) ..
RedirectToAction("Index",new {progress="Done"});
});
return RedirectToAction("Index");
}
}
The logger information displays successfully
All what I need is to find away to be able to reload the index controller after the background task is done successfully but for some reason I don't know it can't be redirected.
The Index action method is like that :
public async Task<IActionResult> Index()
{
var links = new List<LinkObject>();
var files = await _storageProvider.GetAllFiles(null, "backup");
foreach (var f in files)
{
var file = f;
if (f.Contains("/devstoreaccount1/"))
{
file = file.Replace("/devstoreaccount1/", "");
}
file = file.TrimStart('/');
links.Add(new LinkObject()
{
Method = "GET",
Href = await _storageProvider.GetSasUrl(file),
Rel = f
});
}
return View(links);
}
Thanks !
If you want the current page to interact with a long running task, you don't necessarily need the overhead of BackgroundService. That feature is for cases where there is no page to interact with.
First, the server cannot call a client to tell it to reload. At least not without the use of WebSockets, which would definitely be overkill for this. Instead, you will use Javascript (AJAX) to make background calls to poll for the status of your task. This is a common pattern used by any complex web application.
On the server, you'll create a normal async action method that takes all the time it needs to complete the task.
The web page (after it has loaded) will call this action method using AJAX and will ignore the response. That call will eventually time out, but it's not a concern, you don't need the response and the server will continue processing the action even though the socket connection has terminated.
The web page will subsequently begin polling (using AJAX) a different action method which will tell you whether the task has completed or not. You'll need some shared state on the server, perhaps a database table that gets updated by your background task, etc. This method should always return very quickly - all it needs to do is read the present state of the task and return that status.
The web page will continue polling that method until the response changes (e.g. from RUNNING to COMPLETED.) Once the status changes, then you can reload the page using Javascript or whatever you need to do in response to the task completing.
Note: There are some nuances here, like the cost of holding client connections that you expect to time out. If you care you can optimize these away but in most cases it won't be an issue and it adds complexity.
I have modified the Xamarin/Azure TODO example. But the code is stuck in
IMobileServiceTable.ToListAsync()
This is my IO class:
class DataIO
{
BackgroundWorker DatabaseWorker = new BackgroundWorker();
IMobileServiceTable<UserPosition> PositionTable;
MobileServiceClient client;
public DataIO()
{
Init();
}
public void Init()
{
client = new MobileServiceClient(Constants.ApplicationURL);
PositionTable = client.GetTable<UserPosition>();
}
async void AddEntry(UserPosition entry)
{
await PositionTable.InsertAsync(entry);
}
public async Task<List<UserPosition>> GetEntries()
{
List<UserPosition> Entries = await PositionTable.ToListAsync();
return Entries;
}
public async void DeleteEntry(UserPosition entry)
{
await PositionTable.DeleteAsync(entry);
}
public async void AddToDatabase(UserPosition item)
{
await PositionTable.InsertAsync(item);
}
}
The debugger dosen't neither step over it nor throws an error.
How to handle that?
In an earlier call, there wasn't any problem.
EDIT:
I've rewritten the GetEntries() method to:
public async Task<List<UserPosition>> GetEntries()
{
Task<List<UserPosition>> task = PositionTable.ToListAsync();
List<UserPosition> entries = await task;
return entries;
}
according to this example. But the debugger just stays in the line
Task<List<UserPosition>> task = PositionTable.ToListAsync();
AFAIK, IMobileServiceTable.ToListAsync() would send the request as follows for retrieving the result:
Get https://<your-app-name>.azurewebsites.net/tables/UserPosition
I would recommend you using Fiddler to collect the network traces when calling IMobileServiceTable.ToListAsync(). Also, you could access the table endpoint from your mobile app via the browser to make sure your mobile app could work as expected. Additionally, here is a great tutorial about Handling Data in Mobile Clients, you could refer to it.
We are working with .NET Core Web Api, and looking for a lightweight solution to log requests with variable intensity into database, but don't want client's to wait for the saving process.
Unfortunately there's no HostingEnvironment.QueueBackgroundWorkItem(..) implemented in dnx, and Task.Run(..) is not safe.
Is there any elegant solution?
As #axelheer mentioned IHostedService is the way to go in .NET Core 2.0 and above.
I needed a lightweight like for like ASP.NET Core replacement for HostingEnvironment.QueueBackgroundWorkItem, so I wrote DalSoft.Hosting.BackgroundQueue which uses.NET Core's 2.0 IHostedService.
PM> Install-Package DalSoft.Hosting.BackgroundQueue
In your ASP.NET Core Startup.cs:
public void ConfigureServices(IServiceCollection services)
{
services.AddBackgroundQueue(onException:exception =>
{
});
}
To queue a background Task just add BackgroundQueue to your controller's constructor and call Enqueue.
public EmailController(BackgroundQueue backgroundQueue)
{
_backgroundQueue = backgroundQueue;
}
[HttpPost, Route("/")]
public IActionResult SendEmail([FromBody]emailRequest)
{
_backgroundQueue.Enqueue(async cancellationToken =>
{
await _smtp.SendMailAsync(emailRequest.From, emailRequest.To, request.Body);
});
return Ok();
}
QueueBackgroundWorkItem is gone, but we've got IApplicationLifetime instead of IRegisteredObject, which is being used by the former one. And it looks quite promising for such scenarios, I think.
The idea (and I'm still not quite sure, if it's a pretty bad one; thus, beware!) is to register a singleton, which spawns and observes new tasks. Within that singleton we can furthermore register a "stopped event" in order to proper await still running tasks.
This "concept" could be used for short running stuff like logging, mail sending, and the like. Things, that should not take much time, but would produce unnecessary delays for the current request.
public class BackgroundPool
{
protected ILogger<BackgroundPool> Logger { get; }
public BackgroundPool(ILogger<BackgroundPool> logger, IApplicationLifetime lifetime)
{
if (logger == null)
throw new ArgumentNullException(nameof(logger));
if (lifetime == null)
throw new ArgumentNullException(nameof(lifetime));
lifetime.ApplicationStopped.Register(() =>
{
lock (currentTasksLock)
{
Task.WaitAll(currentTasks.ToArray());
}
logger.LogInformation(BackgroundEvents.Close, "Background pool closed.");
});
Logger = logger;
}
private readonly object currentTasksLock = new object();
private readonly List<Task> currentTasks = new List<Task>();
public void SendStuff(Stuff whatever)
{
var task = Task.Run(async () =>
{
Logger.LogInformation(BackgroundEvents.Send, "Sending stuff...");
try
{
// do THE stuff
Logger.LogInformation(BackgroundEvents.SendDone, "Send stuff returns.");
}
catch (Exception ex)
{
Logger.LogError(BackgroundEvents.SendFail, ex, "Send stuff failed.");
}
});
lock (currentTasksLock)
{
currentTasks.Add(task);
currentTasks.RemoveAll(t => t.IsCompleted);
}
}
}
Such a BackgroundPool should be registered as a singleton and can be used by any other component via DI. I'm currently using it for sending mails and it works fine (tested mail sending during app shutdown too).
Note: accessing stuff like the current HttpContext within the background task should not work. The old solution uses UnsafeQueueUserWorkItem to prohibit that anyway.
What do you think?
Update:
With ASP.NET Core 2.0 there's new stuff for background tasks, which get's better with ASP.NET Core 2.1: Implementing background tasks in .NET Core 2.x webapps or microservices with IHostedService and the BackgroundService class
You can use Hangfire (http://hangfire.io/) for background jobs in .NET Core.
For example :
var jobId = BackgroundJob.Enqueue(
() => Console.WriteLine("Fire-and-forget!"));
Here is a tweaked version of Axel's answer that lets you pass in delegates and does more aggressive cleanup of completed tasks.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Logging;
namespace Example
{
public class BackgroundPool
{
private readonly ILogger<BackgroundPool> _logger;
private readonly IApplicationLifetime _lifetime;
private readonly object _currentTasksLock = new object();
private readonly List<Task> _currentTasks = new List<Task>();
public BackgroundPool(ILogger<BackgroundPool> logger, IApplicationLifetime lifetime)
{
if (logger == null)
throw new ArgumentNullException(nameof(logger));
if (lifetime == null)
throw new ArgumentNullException(nameof(lifetime));
_logger = logger;
_lifetime = lifetime;
_lifetime.ApplicationStopped.Register(() =>
{
lock (_currentTasksLock)
{
Task.WaitAll(_currentTasks.ToArray());
}
_logger.LogInformation("Background pool closed.");
});
}
public void QueueBackgroundWork(Action action)
{
#pragma warning disable 1998
async Task Wrapper() => action();
#pragma warning restore 1998
QueueBackgroundWork(Wrapper);
}
public void QueueBackgroundWork(Func<Task> func)
{
var task = Task.Run(async () =>
{
_logger.LogTrace("Queuing background work.");
try
{
await func();
_logger.LogTrace("Background work returns.");
}
catch (Exception ex)
{
_logger.LogError(ex.HResult, ex, "Background work failed.");
}
}, _lifetime.ApplicationStopped);
lock (_currentTasksLock)
{
_currentTasks.Add(task);
}
task.ContinueWith(CleanupOnComplete, _lifetime.ApplicationStopping);
}
private void CleanupOnComplete(Task oldTask)
{
lock (_currentTasksLock)
{
_currentTasks.Remove(oldTask);
}
}
}
}
I know this is a little late, but we just ran into this issue too. So after reading lots of ideas, here's the solution we came up with.
/// <summary>
/// Defines a simple interface for scheduling background tasks. Useful for UnitTesting ASP.net code
/// </summary>
public interface ITaskScheduler
{
/// <summary>
/// Schedules a task which can run in the background, independent of any request.
/// </summary>
/// <param name="workItem">A unit of execution.</param>
[SecurityPermission(SecurityAction.LinkDemand, Unrestricted = true)]
void QueueBackgroundWorkItem(Action<CancellationToken> workItem);
/// <summary>
/// Schedules a task which can run in the background, independent of any request.
/// </summary>
/// <param name="workItem">A unit of execution.</param>
[SecurityPermission(SecurityAction.LinkDemand, Unrestricted = true)]
void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem);
}
public class BackgroundTaskScheduler : BackgroundService, ITaskScheduler
{
public BackgroundTaskScheduler(ILogger<BackgroundTaskScheduler> logger)
{
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
_logger.LogTrace("BackgroundTaskScheduler Service started.");
_stoppingToken = stoppingToken;
_isRunning = true;
try
{
await Task.Delay(-1, stoppingToken);
}
catch (TaskCanceledException)
{
}
finally
{
_isRunning = false;
_logger.LogTrace("BackgroundTaskScheduler Service stopped.");
}
}
public void QueueBackgroundWorkItem(Action<CancellationToken> workItem)
{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
if (!_isRunning)
throw new Exception("BackgroundTaskScheduler is not running.");
_ = Task.Run(() => workItem(_stoppingToken), _stoppingToken);
}
public void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem)
{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
if (!_isRunning)
throw new Exception("BackgroundTaskScheduler is not running.");
_ = Task.Run(async () =>
{
try
{
await workItem(_stoppingToken);
}
catch (Exception e)
{
_logger.LogError(e, "When executing background task.");
throw;
}
}, _stoppingToken);
}
private readonly ILogger _logger;
private volatile bool _isRunning;
private CancellationToken _stoppingToken;
}
The ITaskScheduler (which we already defined in our old ASP.NET client code for UTest test purposes) allows a client to add a background task. The main purpose of the BackgroundTaskScheduler is to capture the stop cancellation token (which is own by the Host) and to pass it into all the background Tasks; which by definition, runs in the System.Threading.ThreadPool so there is no need to create our own.
To configure Hosted Services properly see this post.
Enjoy!
I have used Quartz.NET (does not require SQL Server) with the following extension method to easily set up and run a job:
public static class QuartzUtils
{
public static async Task<JobKey> CreateSingleJob<JOB>(this IScheduler scheduler,
string jobName, object data) where JOB : IJob
{
var jm = new JobDataMap { { "data", data } };
var jobKey = new JobKey(jobName);
await scheduler.ScheduleJob(
JobBuilder.Create<JOB>()
.WithIdentity(jobKey)
.Build(),
TriggerBuilder.Create()
.WithIdentity(jobName)
.UsingJobData(jm)
.StartNow()
.Build());
return jobKey;
}
}
Data is passed as an object that must be serializable. Create an IJob that processes the job like this:
public class MyJobAsync :IJob
{
public async Task Execute(IJobExecutionContext context)
{
var data = (MyDataType)context.MergedJobDataMap["data"];
....
Execute like this:
await SchedulerInstance.CreateSingleJob<MyJobAsync>("JobTitle 123", myData);
The original HostingEnvironment.QueueBackgroundWorkItem was a one-liner and very convenient to use.
The "new" way of doing this in ASP Core 2.x requires reading pages of cryptic documentation and writing considerable amount of code.
To avoid this you can use the following alternative method
public static ConcurrentBag<Boolean> bs = new ConcurrentBag<Boolean>();
[HttpPost("/save")]
public async Task<IActionResult> SaveAsync(dynamic postData)
{
var id = (String)postData.id;
Task.Run(() =>
{
bs.Add(Create(id));
});
return new OkResult();
}
private Boolean Create(String id)
{
/// do work
return true;
}
The static ConcurrentBag<Boolean> bs will hold a reference to the object, this will prevent garbage collector from collecting the task after the controller returns.
Hello I am trying to create a logging system which sends logs to a WCF. In principal it has a Log(string text) method, which can be called multiple times before the actual logging action is made to reduce network chatter. To achieve this I've created a queue (a list) of logs and a timer, which performs the actual logging with a set frequency.
When I want to log something in my program I use the Log method. It looks a bit like this:
private readonly List<string> _currentLogQueue = new List<string>();
public void Log(string logText)
{
lock (_currentLogQueue)
{
_currentLogQueue.Add(logText);
}
}
The queue is then periodically sent to the WCF. The periodic sending is done like so:
private void SetUpQueue(TimeSpan queueFlushPeriod)
{
Task.Run(async () =>
{
while (true)
{
SendQueue(); // Does the actual communication and clears queue on success.
await Task.Delay(queueFlushPeriod);
}
});
}
How can I enable the program using this logger to react to errors during the SendQueue()? I can modify SendQueue to return some kind of error if needed. Right now I only can think of a callback in the form of a delegate passes to the Log() method, but it seems very passé and not fun in the age of async await.
To answer your question:
You can have a TaskCompletionSource indicating success/failure of the logged message:
private readonly List<Tuple<string, TaskCompletionSource<object>> _currentLogQueue = ...;
public Task LogAsync(string logText)
{
var tcs = new TaskCompletionSource<object>();
lock (_currentLogQueue)
{
_currentLogQueue.Add(Tuple.Create(logText, tcs));
}
return tcs.Task;
}
// (Within SendQueue)
var message = queueElement.Item1;
var tcs = queueElement.Item2;
try
{
SendMessage(message);
tcs.TrySetResult(null);
}
catch (Exception ex)
{
tcs.TrySetException(ex);
}
However, I don't think this would really be helpful. What meaningful action can the program take if logging failed?
Today I read a lot about async/await and it completely blew my mind.
I can't understand why the following test passed.
[Test]
public void Test()
{
var listener = new AsyncHttpListener();
listener.ListeningAsync();
try
{
new WebClient().DownloadString("http://localhost:8080/");
}
catch (Exception)
{
}
listener.Close();
}
public class AsyncHttpListener
{
private readonly HttpListener listener;
public AsyncHttpListener()
{
listener = new HttpListener();
listener.Prefixes.Add("http://localhost:8080/");
listener.Start();
}
public void Close()
{
listener.Close();
}
public async void ListeningAsync()
{
var context = await listener.GetContextAsync();
HandleContext(context);
}
private void HandleContext(HttpListenerContext context)
{
throw new Exception("test excpetion");
}
}
Test passed, but output contains:
System.Exception
test excpetion
at AsyncHttpListenerTest.AsyncHttpListener.HandleContext(HttpListenerContext context) in AsyncHttpListener.cs: line 30
at AsyncHttpListenerTest.AsyncHttpListener.d__0.MoveNext() in AsyncHttpListener.cs: line 25
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.b__1(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
I expect that exception will be transmitted from task thread (HandleContext() method) to caller context and test fail. How can i get this behavior?
Make your method async Task instead of async void, and make your test method async Task instead of void:
public async Task ListeningAsync()
{
var context = await listener.GetContextAsync();
HandleContext(context);
}
[Test]
public async Task Test()
{
var listener = new AsyncHttpListener();
await listener.ListeningAsync();
try
{
new WebClient().DownloadString("http://localhost:8080/");
}
catch (Exception)
{
}
listener.Close();
}
There are several good reasons to avoid async void. Error handling is one of them. Errors raised from async void methods go straight to the SynchronizationContext that was current when the method started.
The reason your test passed is because async methods may return to their caller before they complete. The test runner sees the test method return (without throwing an exception yet), and marks it as "passed". If you return Task from your test method, then the test runner knows to wait for the Task to complete before considering the test complete.