All the code samples I've seen so far for Azure WebJobs rely on some kind of trigger (e.g. TimerTrigger or QueueTrigger).
I am looking specifically at WebJobs SDK 3.x, by the way.
So. For a triggerless WebJob (Windows Service-alike one), am I expected to use NoAutomaticTrigger and find a way to kickoff my "main" code manually?
Or should I resort to implementing and registering a class that implements the IHostedService interface?
So far that's the approach I'm taking but it feels more of a hack than a recommended way.
I have not even tried to deploy this code and only ran it on my local machine, so I am afraid that the publishing process will confirm my code is not suitable for Azure WebJobs in its current form.
EntryPoint.cs
This is how the application is being bootstrap when the process is starting.
using Microsoft.Azure.ServiceBus;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
namespace AbcCorp.Jobs
{
public static class Program
{
static async Task Main(string[] args)
{
var config = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json", false)
.Build();
var hostBuilder = new HostBuilder()
.ConfigureWebJobs(builder => { builder.AddAzureStorageCoreServices(); })
.ConfigureServices(serviceCollection =>
{
ConfigureServices(serviceCollection, config);
serviceCollection.AddHostedService<ConsoleApplication>();
});
using (var host = hostBuilder.Build())
await host.RunAsync();
}
private static IServiceCollection ConfigureServices(IServiceCollection services, IConfigurationRoot configuration)
{
services.AddTransient<ConsoleApplication>();
// ... more DI registrations
return services;
}
}
}
ConsoleApplication.cs
This would normally be implemented as a function with a trigger.
The thing is, I want this code to only run once on the process startup.
It will start listening on the service bus events using the regular Microsoft.Azure.ServiceBus SDK package.
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;
using AbcCorp.Internal.Microsoft.Azure.ServiceBus;
using AbcCorp.Api.Messaging;
namespace AbcCorp.Jobs
{
public sealed class ConsoleApplication: IHostedService
{
private readonly IReceiver<SubmissionNotification> _messageReceiver;
private readonly MessageHandler _messageHandler;
public ConsoleApplication(IReceiver<SubmissionNotification> messageReceiver, MessageHandler messageHandler)
{
_messageReceiver = messageReceiver;
_messageHandler = messageHandler;
}
public Task StartAsync(CancellationToken cancellationToken)
{
_messageReceiver.StartListening(_messageHandler.HandleMessage, _messageHandler.HandleException);
return Task.Delay(Timeout.Infinite);
}
public Task StopAsync(CancellationToken cancellationToken)
{
_messageReceiver.Dispose();
return Task.CompletedTask;
}
}
}
So you want a console application to run in a WebJob and listen to messages. You don't really care about WebJob magic like triggers, it's just a place to run your console app. I've done the exact same thing before.
I found the IHostedService abstraction to be very helpful, but I didn't like their SDK. I found it bloated and hard to use. I didn't want to take a large dependency in order use a large array of special magic Azure stuff, when all I wanted to do was run a console application in a WebJob for now, and maybe move it elsewhere later.
So I ended just deleting that dependency, stealing the Shutdown code from the SDK and writing my own Service Host. The result is on my Github Repo azure-webjob-host. Feel free to use it or raid it for ideas. I don't know, maybe if I did it again I'd have another attempt at getting the SDK to work, but I present this is a bit of an alternative to the SDK.
Basically I wrote an IServiceHost not too different from yours (except that StartAsync exited when stuff started instead of just hanging). Then I wrote my own service host, which is basically just a loop:
await _service.StartAsync(cancellationToken);
while (!token.IsCancellationRequested){await Task.Delay(1000);}
await _service.StopAsync(default);
Then I stole the WebJobsShutdownWatcher code from their repo.
Then I created an IServiceHost that started my message handler. (I was using Rabbit, which has nothing to do with triggers or azure stuff)
public class MessagingService : IHostedService, IDisposable
{
public MessagingService(ConnectionSettings connectionSettings,
AppSubscriberSettings subscriberSettings,
MessageHandlerTypeMapping[] messageHandlerTypeMappings,
ILogger<MessagingService> logger)
{
....
}
public async Task StartAsync(CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
await Task.WhenAll(subscribers.Value.Select(s => s.StartSubscriptionAsync()));
}
public async Task StopAsync(CancellationToken cancellationToken)
{
...
}
public void Dispose()
{
...
}
}
Then I put that all together into something like this:
IHostedService myService = new MyService();
using (var host = new ServiceHostBuilder().HostService(myService))
{
await host.RunAsync(default);
}
I have some workers attached to service bus topics and what we do is the following (ServiceBusClient is a custom Class that contains our Subscription Client):
public override Task StartAsync(CancellationToken cancellationToken)
{
_serviceBusClient.RegisterOnMessageHandlerAndReceiveMessages(MessageReceivedAsync);
_logger.LogDebug($"Started successfully the Import Client. Listening for messages...");
return base.StartAsync(cancellationToken);
}
public void RegisterOnMessageHandlerAndReceiveMessages(Func<Message, CancellationToken, Task> ProcessMessagesAsync)
{
// Configure the message handler options in terms of exception handling, number of concurrent messages to deliver, etc.
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
// Maximum number of concurrent calls to the callback ProcessMessagesAsync(), set to 1 for simplicity.
// Set it according to how many messages the application wants to process in parallel.
MaxConcurrentCalls = 1,
// Indicates whether MessagePump should automatically complete the messages after returning from User Callback.
// False below indicates the Complete will be handled by the User Callback as in `ProcessMessagesAsync` below.
AutoComplete = false
};
// Register the function that processes messages.
SubscriptionClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
And then you can use DI to instantiate your service bus client and inject on the constructor of your Worker class.
Here i have the initialization of the singleton instance of my custom class Service Bus Client
services.AddSingleton<IServiceBusClient, ServiceBusClient>((p) =>
{
var diagnostics = p.GetService<EventHandling>();
var sbc = new ServiceBusClient(
programOptions.Endpoint,
programOptions.TopicName,
programOptions.Subscriber,
programOptions.SubscriberKey);
sbc.Exception += exception => diagnostics.HandleException(exception);
return sbc;
});
Then on this custom class, i initialize my subscription client
public ServiceBusClient(
string endpoint,
string topicName,
string subscriberName,
string subscriberKey, ReceiveMode mode = ReceiveMode.PeekLock)
{
var connBuilder = new ServiceBusConnectionStringBuilder(endpoint, topicName, subscriberName, subscriberKey);
var connectionString = connBuilder.GetNamespaceConnectionString();
ConnectionString = connectionString;
TopicName = topicName;
SubscriptionName = topicName;
SubscriptionClient = new SubscriptionClient(connectionString, topicName, subscriberName, mode);
}
You can check #george chen's answer from this post How to create service bus trigger webjob?
where instead of creating a receiver and registering a message handler, you can use the in built queue trigger and and write your message handler logic inside it.
Related
I've implemented a listener service using rabbitMQ. I used a log for show the message body received but i dont know the main purpose for "_channel.BasicAck" and "_channel.BasicConsume" methods and why is in that order.
using RabbitMQ.Client;
using RabbitMQ.Client.Events;
using Serilog;
using System.Text;
using System.Text.Json;
public class ListenerBackgroundService : BackgroundService
{
private readonly ILogger _logger = Log.Logger.ForContext<ListenerBackgroundService();
private readonly IConfiguration _configuration;
private IConnection connection;
private IModel _channel;
public ListenerBackgroundService(IConfiguration configuration)
{
_configuration = configuration;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
stoppingToken.ThrowIfCancellationRequested();
var consumer = new AsyncEventingBasicConsumer(_channel);
consumer.Received += async (bc, ea) =>
{
var message = Encoding.UTF8.GetString(ea.Body.ToArray());
var mensaje = JsonSerializer.Deserialize<IntegrationMessage>(message);
try
{
//Add message parsing and handling logic here
_logger.Information(mensaje.Message);
_channel.BasicAck(ea.DeliveryTag, false);
}
catch (Exception ex)
{
_channel.BasicNack(ea.DeliveryTag, false, false);
}
};
_channel.BasicConsume(queue: "hello", autoAck: false, consumer: consumer);
await Task.CompletedTask;
}
RabbitMQ is meesage queue which is based on publisher and consumer. So
"channel.BasicAck" is used to send acknowledgement.
API methods used for delivery acknowledgement are usually exposed as operations on a channel in client libraries. Java client users will use Channel#basicAck and Channel#basicNack to perform a basic.ack and basic.nack, respectively.
MoreDetail
If Not use Basicack then Messages will be redelivered when your client quits (which may look like random redelivery), but RabbitMQ will eat more and more memory as it won't be able to release any unacked messages.
More Details
"_channel.BasicConsume" is used to recieve message as consumer and do needful operation on queue as consumer.
For more detail, Please refer the above links.
I'm writing a WinUI3 (Project Reunion 0.5) application with .NET 5 and would like to use the .NET Generic Host. I'm using the default host with a custom IHostedService:
public App() {
_host = Host.CreateDefaultBuilder()
.ConfigureServices((context, services) =>
{
services.AddHostedService<MyHostedService>();
}).Build();
InitializeComponent();
}
The hosted service performs some asynchronous operations in StopAsync. For demonstration purposes, let's say it delays for 1 second (this code still produces the issue):
public override async Task StopAsync(CancellationToken cancellationToken)
{
await Task.Delay(1000);
}
I start the host in OnLaunched:
protected override async void OnLaunched(Microsoft.UI.Xaml.LaunchActivatedEventArgs args)
{
await _host.StartAsync();
m_window = new MainWindow();
m_window.Activate();
}
I let the default ConsoleLifetime implementation stop the host before the process exits.
The Task returned by my IHostedService.StopAsync implementation completes, but IHost.StopAsync never returns and the process hangs with this message in the output:
Microsoft.Hosting.Lifetime: Information: Application is shutting down...
Microsoft.Hosting.Lifetime: Information: Waiting for the host to be disposed. Ensure all 'IHost' instances are wrapped in 'using' blocks.
If I step through with the debugger, sometimes the IHost.StopAsync method will time out and an exception will be thrown. This never happens outside of the debugger. I have tried explicitly stopping and disposing the host when the MainWindow is closed, but it didn't make any difference.
I thought perhaps the DispatcherQueueSynchronizationContext was being shut down before the host could stop and tasks were not being serviced, but the DispatcherQueue.ShutdownStarting event is never fired.
Any other ideas?
I took #Dai's advice from the comments and investigated running WinUI on a separate thread and running the host on the main thread.
I created an IHostedService to manage the WinUI application:
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Options;
using Microsoft.System;
using Microsoft.UI.Xaml;
using System;
using System.Threading;
using System.Threading.Tasks;
namespace MyApp.Hosting
{
public class WinUIHostedService<TApplication> : IHostedService, IDisposable
where TApplication : Application, new()
{
private readonly IHostApplicationLifetime HostApplicationLifetime;
private readonly IServiceProvider ServiceProvider;
public WinUIHostedService(
IHostApplicationLifetime hostApplicationLifetime,
IServiceProvider serviceProvider)
{
HostApplicationLifetime = hostApplicationLifetime;
ServiceProvider = serviceProvider;
}
public void Dispose()
{
}
public Task StartAsync(CancellationToken cancellationToken)
{
var thread = new Thread(Main);
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
private void Main()
{
WinRT.ComWrappersSupport.InitializeComWrappers();
Application.Start((p) => {
var context = new DispatcherQueueSynchronizationContext(DispatcherQueue.GetForCurrentThread());
SynchronizationContext.SetSynchronizationContext(context);
new TApplication();
});
HostApplicationLifetime.StopApplication();
}
}
}
I defined DISABLE_XAML_GENERATED_MAIN in the build settings and added my own Main:
public class Program
{
public static void Main(string[] args)
{
Host.CreateDefaultBuilder()
.ConfigureServices(services =>
{
services.AddHostedService<WinUIHostedService<App>>();
})
.Build().Run();
}
}
Voila! The WinUI application still runs fine and the host stops cleanly when the main window closes, even when IHostedService.StopAsync runs asynchronous code.
Note that this code is just the first thing that worked. It could probably be improved and I don't fully understand the Generic Host lifetime semantics.
I have a .net core 3.0 web application. In Startup.cs, I register an EventProcessor (from Microsoft.Azure.EventHubs.Processor) that listens to an Azure EventHub for events. I do it like this:
await eventProcessorHost.RegisterEventProcessorAsync<TwinChangesEventHandler>();
I'm interested in device twin changes in an IoT Hub that's connected to the EventHub.
So, in the EventProcessor, I want to access the SignalR IHubContext interface (from Microsoft.AspNetCore.SignalR - the new version of SignalR) to be able to notify connected browsers of a device twin property change. My problem is that the EventProcessor can't get a handle to IHubContext. How can I get it?
I see online that people are using dependency injection but because my EventProcessor is created by RegisterEventProcessorAsync() like I showed above, its default constructor is ALWAYS called and NOT the one with IHubContext as a parameter! Even if I use a factory to create the EventProcessor and call RegisterEventProcessorFactoryAsync() in Startup.cs, I can't get the IHubContext handle in the factory, because the call is not originating from a controller. It either originates from Startup.ConfigureServices() or a callback from whenever something happens in the EventHub, which is not a controller method. I'm really stuck, so any help would be much appreciated. Does anyone know the answer to this?
You can add your Factory and processor to services
.ConfigureServices((hostContext, services) =>
{
...
services.AddSingleton<IEventProcessorFactory, EventProcessorFactory>();
services.AddSingleton<IEventProcessor, TwinChangesEventHandler>();
...
});
public class EventProcessorFactory : IEventProcessorFactory
{
private readonly IEventProcessor _fluxEventProcessor;
public EventProcessorFactory(IEventProcessor fluxEventProcessor)
{
_fluxEventProcessor = fluxEventProcessor;
}
public IEventProcessor CreateEventProcessor(PartitionContext context)
{
return _fluxEventProcessor;
}
}
Then in your handler you can have access to the injected hub
public class TwinChangesEventHandler : IEventProcessor
{
private readonly IHubContext<MyHub> _myHubContext;
public TwinChangesEventHandler(IHubContext<MyHub> myHubContext)
{
_myHubContext= myHubContext;
}
...
async Task IEventProcessor.ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
{
foreach (var eventData in messages)
{
await _myHubContext.Clients.All.SendAsync("Update", eventData);
}
//Call checkpoint every 5 minutes, so that worker can resume processing from 5 minutes back if it restarts.
if (_checkpointStopWatch.Elapsed > TimeSpan.FromMinutes(5))
{
await context.CheckpointAsync();
_checkpointStopWatch.Restart();
}
}
}
Edit: I will accept Azure configuration related changes as an answer to this question.
I am attempting to setup a retry policy to prevent instantly retrying a message when a 3rd party service is temporarily unavailable.
Currently the job is retried immediately multiple times and fails each time due to the temporary outage of the 3rd party service.
How do I set a retry delay for these messages?
I have the following code for Main:
class Program
{
static void Main()
{
var config = new JobHostConfiguration();
if (config.IsDevelopment)
config.UseDevelopmentSettings();
config.UseCore();
config.UseServiceBus(new ServiceBusConfiguration()
{
ConnectionString = Configuration.GetAppSetting("Microsoft.ServiceBus.ConnectionString"),
MessageOptions = new OnMessageOptions()
{
}
});
var host = new JobHost(config);
LogManager.GetCurrentClassLogger().Information("F1.Birst.Automation web job starting.");
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}
}
I have an ErrorMonitor setup which properly logs errors:
public class ExceptionHandler
{
private static readonly ILogger Log = LogManager.GetCurrentClassLogger();
public void Handle([ErrorTrigger] TraceFilter message, TextWriter log)
{
foreach (var exception in message.GetEvents())
Log.Error(exception.Exception.InnerException, exception.Message);
}
}
And my message handler looks like this:
public class ChurchCodeChangedEventHandler : ChurchSpaceHandler
{
private static readonly ILogger Log = LogManager.GetCurrentClassLogger();
public void Handle([ServiceBusTrigger(nameof(ChurchCodeChangedEvent), "F1.Birst.Automation.ChurchCodeChangedEvent")] ChurchCodeChangedEvent message, TextWriter log)
{
Log.Information(LogTemplates.ChurchCodeChanged, message.ChurchId);
// snip
}
}
How do I set a retry delay for these messages?
Webjobs do not support the concept of delayed retries. You can only control a few things using ServiceBusConfiguration, but those are not retries looking at the source code.
You could use frameworks like NServiceBus or MassTransit to get delayed retries. There's an example of how to use NServiceBus with WebJobs and you can run it locally to see how delayed retries would work.
I have an application which receive messages through a Azure ServiceBus using the Rebus library.
I observe a different behaviour when the application runs on Azure as a Web app, compared to when I run it locally.
When it runs in Azure, my message handlers are not being invoked.
Everything works as expected when the application runs local.
This is my setup.
In my web project Portal, Global.asax.cs:
protected void Application_Start()
{
_webContainer = new WindsorContainer();
_queueContainer = new WindsorContainer();
ConfigureWebContainer();
_queueContainer.Install(FromAssembly.Containing<ServiceBusInstaller>());
ApplicationInsightsConfig.Configure();
}
In the project ServiceBus (where ServiceBusInstaller lies)
public void Install(IWindsorContainer container, IConfigurationStore store)
{
var adapter = new CastleWindsorContainerAdapter(container);
var connectionString = CloudConfigurationManager.GetSetting("QueueUrl");
var configurer = Configure
.With(adapter)
.Options(o =>
{
o.SimpleRetryStrategy(maxDeliveryAttempts: 0);
})
.Logging(l => l.Log4Net())
.Routing(r => r.TypeBased()
.MapAssemblyOf<ResetPasswordMessage>("Email"))
Transport(t => t.UseAzureServiceBus(connectionString, "Email"));
// Create and starts the bus
configurer.Start();
}
My ResetPasswordHandler looks like this:
public class ResetPasswordHandler : IHandleMessages<ResetPasswordMessage>
{
private readonly IContactAndEmailService _contactAndEmailService;
public ResetPasswordHandler(IContactAndEmailService contactAndEmailService)
{
_contactAndEmailService = contactAndEmailService;
}
public async Task Handle(ResetPasswordMessage message)
{
_contactAndEmailService.SendEmail(message);
}
}
I am using Service Bus Explorer to connect to the service bus, and I can see that the message is in the "email" queue. It does however stay there, and thus is not consumed.
Any pointers to either setup or azure subscribtion limitations or something else that can help me move forward is highly appreciated.
I think it might have something to do with the casing of the queue name, Email.
Could you try and make it lowercase throughout?