We Keep Coding

Azure Function Event Hub Output Binding not working when deployed

I am using an Azure Function to get messages from a Rabbit MQ Broker to an Event Hub.
The function works perfect when I run it locally.
Here is the code of the function:
using System.Text;
using System.Dynamic;
using System.Threading.Tasks;
using CaseOnline.Azure.WebJobs.Extensions.Mqtt;
using CaseOnline.Azure.WebJobs.Extensions.Mqtt.Messaging;
using Microsoft.Azure.WebJobs;
using Newtonsoft.Json;
public static class Test
{
[FunctionName("EventHubOutput")]
public static async Task Run(
[MqttTrigger("topic/#")] IMqttMessage message,
[EventHub("eventhubname", Connection = "EventHubConnectionAppSetting")] IAsyncCollector<string> outputEvents,
ILogger log)
{
var body = message.GetMessage();
var bodyString = Encoding.UTF8.GetString(body);
dynamic obj = JsonConvert.DeserializeObject<ExpandoObject>(bodyString);
obj.Topic = message.Topic;
await outputEvents.AddAsync(JsonConvert.SerializeObject(obj));
}
}
When deployed and run in the Azure portal, I get the following error messages:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: EventHubOutput
---> System.InvalidOperationException: Error while handling parameter outputEvents after function returned:
---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known
at (...)
Any idea what the problem might be?
Thank you.

You are using the bindings incorrectly. Check out RabbitMQ bindings for Azure Functions overview.
The following example shows a C# function that reads and logs the RabbitMQ message as a RabbitMQ Event:
[FunctionName("RabbitMQTriggerCSharp")]
public static void RabbitMQTrigger_BasicDeliverEventArgs(
[RabbitMQTrigger("queue", ConnectionStringSetting = "rabbitMQConnectionAppSetting")] BasicDeliverEventArgs args,
ILogger logger
)
{
logger.LogInformation($"C# RabbitMQ queue trigger function processed message: {Encoding.UTF8.GetString(args.Body)}");
}
The following example shows how to read the message as a POCO:
namespace Company.Function
{
public class TestClass
{
public string x { get; set; }
}
public class RabbitMQTriggerCSharp{
[FunctionName("RabbitMQTriggerCSharp")]
public static void RabbitMQTrigger_BasicDeliverEventArgs(
[RabbitMQTrigger("queue", ConnectionStringSetting = "rabbitMQConnectionAppSetting")] TestClass pocObj,
ILogger logger
)
{
logger.LogInformation($"C# RabbitMQ queue trigger function processed message: {pocObj}");
}
}
}
I recommend you to check out this complete guide to setup Rabbit MQ Trigger in Azure Functions: RabbitMQ trigger for Azure Functions overview

Why is the WebApplicationFactory not resetting between tests?

I have two tests of an ASP.NET core webapp within a single test class. The class uses an IClassFixture<WebApplicationFactory<Startup>>.
The first test is a happy path test: when a message arrives on a Kafka topic, some file appears in a particular output directory.
The second test is to verify that if something goes wrong with outputting said file, the webapp should then be in an unhealthy status.
For brevity, I'm omitting the code that publishes to Kafka and that generates the output folder the application saves files to - neither of these are relevant.
[CollectionDefinition("e2e", DisableParallelization = true)]
[Trait("Category", "IntegrationTest")]
public class EndToEndTest : IClassFixture<WebApplicationFactory<Startup>>, IClassFixture<KafkaFixture>
{
private readonly WebApplicationFactory<Startup> _factory;
public EndToEndTest(WebApplicationFactory<Startup> factory)
{
_factory = factory;
KafkaUtils.Publish(SomeMessage()).GetAwaiter().GetResult();
}
[Fact]
public void WhenXReceived_ThenFileIsOutput()
{
var options = OutputFolder();
Directory.CreateDirectory(options.Path);
RunService(services => services.AddSingleton(Options.Create(options))).CreateClient();
Thread.Sleep(10_000);
var outputDirectory = Directory.GetFiles(options.Path);
Assert.Single(outputDirectory);
var file = outputDirectory.Single();
Assert.NotEmpty(File.ReadAllLinesAsync(file).GetAwaiter().GetResult());
Directory.Delete(options.Path, true);
}
[Fact]
public void WhenFileTransferFails_ThenAppShouldBeUnhealthy()
{
var options = OutputFolder();
// only try once, so we don't have to wait
var retry = new RetryPolicyConfiguration {OnErrorRetryCount = 1};
var factory = RunService(services =>
{
services.AddSingleton(Options.Create(options));
services.AddSingleton(Options.Create(retry));
services.RemoveAll<IFileWriter>();
services.AddScoped<IFileWriter, ThrowingFileWriter>();
});
var client = factory.CreateClient();
Thread.Sleep(10_000);
Assert.False(Directory.Exists(options.Path));
var response = client.GetAsync("/health/ready").GetAwaiter().GetResult();
var stream = response.Content.ReadAsStreamAsync().GetAwaiter().GetResult();
using var reader = new StreamReader(stream);
var body = reader.ReadToEndAsync().GetAwaiter().GetResult();
var health = JsonConvert.DeserializeObject<HealthCheckResult>(body);
Assert.NotEqual(HealthStatus.Healthy, health.Status);
}
protected WebApplicationFactory<Startup> RunService(Action<IServiceCollection> serviceConfig)
{
return _factory
.WithWebHostBuilder(builder => builder
.ConfigureTestServices(services =>
{
serviceConfig(services);
services.AddSubscribeBus(new ConfigurationBuilder()
.AddJsonFile(ConfigMapFileProvider.FromRelativePath("config"),
"appsettings.e2e.json", true, true)
.Build());
}));
}
}
public class ThrowingFileWriter : IFileWriter
{
public void Write(string fileName, envelope envelope)
{
throw new Exception("foo");
}
public void Delete(string fileName)
{
}
}
If I run WhenFileTransferFails_ThenAppShouldBeUnhealthy() (unhealthyTest from here on, for brevity) on its own, then the test passes. But if I run the entire test class, then WhenXReceived_ThenFileIsOutput() (healthyTest, for brevity) runs before it, and for some reason unhealthyTest seems to use the service context that was created for healthyTest.
I've tried my best to work out what's going on, but it doesn't make sense to me. When I debug through it, the order of events is roughly as follows:
healthyTest starts
Startup invoked for healthytest
Constructor of service that uses IFileWriter is invoked with real instance of service
healthyTest completes
unhealthyTest starts
Constructor of service that uses IFileWriter is invoked with real instance of service (???)
Startup invoked for unhealthyTest
serviceConfig action invoked for unhealthyTest: all implementations of IFileWriter replaced with ThrowingFileWriter - but this is too late, it's already been constructed with the real instance in step 6
Consequently, the real service is used instead of the throwing service, and the exception handler that sets the app status to Unhealthy is never invoked.
Originally this was all running asynchronously, but I thought that perhaps the async nature of the tests meant that the two were conflicting with the webapp created by the factory - hence all the GetAwaiter().GetResult().
What am I doing wrong in my setup of the webapp for the test?
Note: it's unfortunately absolutely not an option to move these tests into their own test classes.

How to implement a triggerless .NET Core Console App as a continuous Azure WebJob?

All the code samples I've seen so far for Azure WebJobs rely on some kind of trigger (e.g. TimerTrigger or QueueTrigger).
I am looking specifically at WebJobs SDK 3.x, by the way.
So. For a triggerless WebJob (Windows Service-alike one), am I expected to use NoAutomaticTrigger and find a way to kickoff my "main" code manually?
Or should I resort to implementing and registering a class that implements the IHostedService interface?
So far that's the approach I'm taking but it feels more of a hack than a recommended way.
I have not even tried to deploy this code and only ran it on my local machine, so I am afraid that the publishing process will confirm my code is not suitable for Azure WebJobs in its current form.
EntryPoint.cs
This is how the application is being bootstrap when the process is starting.
using Microsoft.Azure.ServiceBus;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
namespace AbcCorp.Jobs
{
public static class Program
{
static async Task Main(string[] args)
{
var config = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json", false)
.Build();
var hostBuilder = new HostBuilder()
.ConfigureWebJobs(builder => { builder.AddAzureStorageCoreServices(); })
.ConfigureServices(serviceCollection =>
{
ConfigureServices(serviceCollection, config);
serviceCollection.AddHostedService<ConsoleApplication>();
});
using (var host = hostBuilder.Build())
await host.RunAsync();
}
private static IServiceCollection ConfigureServices(IServiceCollection services, IConfigurationRoot configuration)
{
services.AddTransient<ConsoleApplication>();
// ... more DI registrations
return services;
}
}
}
ConsoleApplication.cs
This would normally be implemented as a function with a trigger.
The thing is, I want this code to only run once on the process startup.
It will start listening on the service bus events using the regular Microsoft.Azure.ServiceBus SDK package.
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;
using AbcCorp.Internal.Microsoft.Azure.ServiceBus;
using AbcCorp.Api.Messaging;
namespace AbcCorp.Jobs
{
public sealed class ConsoleApplication: IHostedService
{
private readonly IReceiver<SubmissionNotification> _messageReceiver;
private readonly MessageHandler _messageHandler;
public ConsoleApplication(IReceiver<SubmissionNotification> messageReceiver, MessageHandler messageHandler)
{
_messageReceiver = messageReceiver;
_messageHandler = messageHandler;
}
public Task StartAsync(CancellationToken cancellationToken)
{
_messageReceiver.StartListening(_messageHandler.HandleMessage, _messageHandler.HandleException);
return Task.Delay(Timeout.Infinite);
}
public Task StopAsync(CancellationToken cancellationToken)
{
_messageReceiver.Dispose();
return Task.CompletedTask;
}
}
}

So you want a console application to run in a WebJob and listen to messages. You don't really care about WebJob magic like triggers, it's just a place to run your console app. I've done the exact same thing before.
I found the IHostedService abstraction to be very helpful, but I didn't like their SDK. I found it bloated and hard to use. I didn't want to take a large dependency in order use a large array of special magic Azure stuff, when all I wanted to do was run a console application in a WebJob for now, and maybe move it elsewhere later.
So I ended just deleting that dependency, stealing the Shutdown code from the SDK and writing my own Service Host. The result is on my Github Repo azure-webjob-host. Feel free to use it or raid it for ideas. I don't know, maybe if I did it again I'd have another attempt at getting the SDK to work, but I present this is a bit of an alternative to the SDK.
Basically I wrote an IServiceHost not too different from yours (except that StartAsync exited when stuff started instead of just hanging). Then I wrote my own service host, which is basically just a loop:
await _service.StartAsync(cancellationToken);
while (!token.IsCancellationRequested){await Task.Delay(1000);}
await _service.StopAsync(default);
Then I stole the WebJobsShutdownWatcher code from their repo.
Then I created an IServiceHost that started my message handler. (I was using Rabbit, which has nothing to do with triggers or azure stuff)
public class MessagingService : IHostedService, IDisposable
{
public MessagingService(ConnectionSettings connectionSettings,
AppSubscriberSettings subscriberSettings,
MessageHandlerTypeMapping[] messageHandlerTypeMappings,
ILogger<MessagingService> logger)
{
....
}
public async Task StartAsync(CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
await Task.WhenAll(subscribers.Value.Select(s => s.StartSubscriptionAsync()));
}
public async Task StopAsync(CancellationToken cancellationToken)
{
...
}
public void Dispose()
{
...
}
}
Then I put that all together into something like this:
IHostedService myService = new MyService();
using (var host = new ServiceHostBuilder().HostService(myService))
{
await host.RunAsync(default);
}

I have some workers attached to service bus topics and what we do is the following (ServiceBusClient is a custom Class that contains our Subscription Client):
public override Task StartAsync(CancellationToken cancellationToken)
{
_serviceBusClient.RegisterOnMessageHandlerAndReceiveMessages(MessageReceivedAsync);
_logger.LogDebug($"Started successfully the Import Client. Listening for messages...");
return base.StartAsync(cancellationToken);
}
public void RegisterOnMessageHandlerAndReceiveMessages(Func<Message, CancellationToken, Task> ProcessMessagesAsync)
{
// Configure the message handler options in terms of exception handling, number of concurrent messages to deliver, etc.
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
// Maximum number of concurrent calls to the callback ProcessMessagesAsync(), set to 1 for simplicity.
// Set it according to how many messages the application wants to process in parallel.
MaxConcurrentCalls = 1,
// Indicates whether MessagePump should automatically complete the messages after returning from User Callback.
// False below indicates the Complete will be handled by the User Callback as in `ProcessMessagesAsync` below.
AutoComplete = false
};
// Register the function that processes messages.
SubscriptionClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
And then you can use DI to instantiate your service bus client and inject on the constructor of your Worker class.
Here i have the initialization of the singleton instance of my custom class Service Bus Client
services.AddSingleton<IServiceBusClient, ServiceBusClient>((p) =>
{
var diagnostics = p.GetService<EventHandling>();
var sbc = new ServiceBusClient(
programOptions.Endpoint,
programOptions.TopicName,
programOptions.Subscriber,
programOptions.SubscriberKey);
sbc.Exception += exception => diagnostics.HandleException(exception);
return sbc;
});
Then on this custom class, i initialize my subscription client
public ServiceBusClient(
string endpoint,
string topicName,
string subscriberName,
string subscriberKey, ReceiveMode mode = ReceiveMode.PeekLock)
{
var connBuilder = new ServiceBusConnectionStringBuilder(endpoint, topicName, subscriberName, subscriberKey);
var connectionString = connBuilder.GetNamespaceConnectionString();
ConnectionString = connectionString;
TopicName = topicName;
SubscriptionName = topicName;
SubscriptionClient = new SubscriptionClient(connectionString, topicName, subscriberName, mode);
}

You can check #george chen's answer from this post How to create service bus trigger webjob?
where instead of creating a receiver and registering a message handler, you can use the in built queue trigger and and write your message handler logic inside it.

Obtaining SignalR Hub without losing its clients from "other place in code"

I have app for one person that has to display data on webpage from external device connected via usb port
Before, I gave user button "Start listening" that sent http request to backend which started listening on port (and blocked app, but it was fine because it is supposed to be used by exactly 1 person at time) until it received not-error response (SerialDataReceivedEvent/SerialErrorReceivedEvent)
and SerialData was returned from that request and displayed on page
I have to rewrite this using SignalR, so I quickly came with naive solution like this:
public class DeviceReaderHub : Hub
{
private readonly IConfiguration _config;
// this is static because listening that port (SerialPort) has to stay open
private static DeviceReader_Helper _service;
public DeviceReaderHub(IConfiguration config)
{
_config = config;
if (_service == null)
{
_service = new DeviceReader_Helper();
_service.Open(_config["DeviceInfo:Port"]);
}
_service.DataReceived_Delegate = SendMessage;
_service.ErrorReceived_Delegate = SendErrorMessage;
}
public async Task SendMessage(string message)
{
await Clients.All.SendAsync("onRead", message);
}
public async Task SendErrorMessage(string message)
{
await Clients.All.SendAsync("onRead", $"error = {message}");
}
public async override Task OnConnectedAsync()
{
await Clients.All.SendAsync("onConnected");
await base.OnConnectedAsync();
}
}
but I received reality-check very quickly - I cannot do it like that because DeviceReaderHub is disposed.
Exception thrown: 'System.ObjectDisposedException' in Microsoft.AspNetCore.SignalR.Core.dll
I thought about obtaining new DeviceReaderHub hub whenever SerialDataReceivedEvent or SerialErrorReceivedEvent is being Invoked
but I have not idea how can I get instance of the hub without losing connected clients

SignalR hubs are transient objects, which means that a new hub instance is used for each method call on the hub from the client. According to this Microsoft Docs:
Don't store state in a property on the hub class. Every hub method call is executed on a new hub instance.
So you should not use the hub to do anything other than receiving and handling requests from the client. To send messages to the client outside of the hub, SignalR provides the HubContext<T> class which is available from Dependency Injection. For example:
public class DeviceReader_Helper {
private readonly IHubContext<DeviceReaderHub> _hubContext;
// you can obtain hubContext either from constructor DI, or service locator pattern with an IServiceProvider
public DeviceReader_Helper(IHubContext<DeviceReaderHub> hubContext) {
_hubContext = hubContext;
}
public async Task SendMessage(string message) {
await _hubContext.Clients.All.SendAsync("onRead", message);
}
}
For more information see this. As for your concern that
but I have not idea how can I get instance of the hub without losing connected clients
Clients can be connected without a hub instance. Hubs are only used to receive messages from the client, and are not necessary to keep a client connected to the server.

How to intercept an Azure WebJob failure / exception

Currently in Azure when a a WebJob throws an exception, the exception gets caught and handled by the JobHost (somehow) and then logs the exception to the dashboard that's available through the blade of the Web App in which the webjob is hosted. Is there any way to intercept the error handling or override it so that I can plug in my Application Insights instance ?

You can use the Azure WebJobs SDK Extensions : there is an ErrorTrigger so that you can use to intercept unhandled exceptions :
public class UnhandledErrorTrigger : IDisposable
{
private readonly TelemetryClient _telemetryClient;
public UnhandledErrorTrigger(TelemetryClient telemetryClient)
{
_telemetryClient = telemetryClient;
}
public void UnHandledException([ErrorTrigger("0:01:00", 1)] TraceFilter filter, TextWriter log)
{
foreach (var traceEvent in filter.Events)
{
_telemetryClient.TrackException(traceEvent.Exception);
}
// log the last detailed errors to the Dashboard
log.WriteLine(filter.GetDetailedMessage(1));
}
public void Dispose()
{
_telemetryClient.Flush();
}
}
To register the Error extensions, call config.UseCore() in your startup code :
private static void Main()
{
var config = new JobHostConfiguration();
config.UseCore();
...
new JobHost(config).RunAndBlock();
}
So if you are using an IoC container, you can easily inject your TelemetryClient. To configure a job activator for the webjob you can look at this post:
Dependency injection using Azure WebJobs SDK?

Have a look at some azure docs here. You can attach a handler to the AppDomain handling unknown exceptions (taken from the link above):
AppDomain.CurrentDomain.UnhandledException += CurrentDomain_UnhandledException;
// ...
private void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
ExceptionTelemetry excTelemetry = new ExceptionTelemetry((Exception)e.ExceptionObject);
excTelemetry.SeverityLevel = SeverityLevel.Critical;
excTelemetry.HandledAt = ExceptionHandledAt.Unhandled;
telemetryClient.TrackException(excTelemetry);
telemetryClient.Flush();
}