Now i'm working at writing unit test on azure service bus trigger function
It's highly needed to mock somehow BrokeredMessage object that pass around into function. Function declaration is given below:
public static void Run(
[ServiceBusTrigger("saas01.queue.dbmigration", AccessRights.Manage, Connection = "connection")]BrokeredMessage message)
Unfortunately, i can't find any applicable way to mock it. It hardly mocking du to this class is sealed and i can't event create wrapper around it. Do you have some ideas about it?
Thanks for helping
,
One solution is to create a wrapper around BrokeredMessage you can test, as is done here. Here's also a MSDN post to the ServiceBus team that talks about using a wrapper too.
Note that Azure Functions V2 uses the Message class, which is public and not sealed.
[FunctionName("ServiceBusFunc")]
public static void Run([ServiceBusTrigger("myqueue", AccessRights.Manage, Connection = "ServiceBus")]BrokeredMessage myQueueItem, TraceWriter log)
{
var message = new MyBrokeredMessage(myQueueItem);
BusinessLogic(message, log);
}
public static void BusinessLogic(MyBrokeredMessage myMessage, TraceWriter log)
{
var stream = myMessage.GetBody<Stream>();
var reader = new StreamReader(stream);
log.Info($"C# ServiceBus queue trigger function processed message: '{reader.ReadToEnd() }'");
}
public class MyBrokeredMessage
{
private BrokeredMessage _msg;
public MyBrokeredMessage(BrokeredMessage msg) => _msg = msg;
public T GetBody<T>()
{
return _msg.GetBody<T>();
}
}
Related
Is it possible that 2 azure functions can get triggered by one eventhub? One azure function will write its data to database1 and the other azure function writes its data to database2
[FunctionName("EventToDB1")]
public async System.Threading.Tasks.Task Run([EventHubTrigger("eventhub", Connection = "Debezium")]
EventData[] events, ILogger log)
{
{
[FunctionName("EventToDB2")]
public async System.Threading.Tasks.Task Run([EventHubTrigger("eventhub", Connection = "Debezium")]
EventData[] events, ILogger log)
{
{
answer on the possibility of having 2 azure functions get triggered by one eventhub
Yes that is possible by using different consumer groups. Because you specified the same connection to the Event Hub, being "Debezium", I Assume you want both funtions to process the same message. You have to create a new consumer group and specify the name using the ConsumerGroup property of the EventHubTrigger attribute (The default consumergroup is $Default):
public class EventToDB1
{
[FunctionName("EventToDB1")]
public async System.Threading.Tasks.Task Run(
[EventHubTrigger("eventhub",
Connection = "Debezium",
ConsumerGroup = "CG1")]
EventData[] events, ILogger log)
{
}
}
public class EventToDB2
{
[FunctionName("EventToDB2")]
public async System.Threading.Tasks.Task Run(
[EventHubTrigger("eventhub",
Connection = "Debezium",
ConsumerGroup = "CG2")]
EventData[] events, ILogger log)
{
}
}
Each consumer group receives the same messages from the Event Hub.
I do agree with #peter bons, you need to create two consumer groups for that and you can create two consumers by below process:
You can also use logic apps to work with event hubs.
I am using an Azure Function to get messages from a Rabbit MQ Broker to an Event Hub.
The function works perfect when I run it locally.
Here is the code of the function:
using System.Text;
using System.Dynamic;
using System.Threading.Tasks;
using CaseOnline.Azure.WebJobs.Extensions.Mqtt;
using CaseOnline.Azure.WebJobs.Extensions.Mqtt.Messaging;
using Microsoft.Azure.WebJobs;
using Newtonsoft.Json;
public static class Test
{
[FunctionName("EventHubOutput")]
public static async Task Run(
[MqttTrigger("topic/#")] IMqttMessage message,
[EventHub("eventhubname", Connection = "EventHubConnectionAppSetting")] IAsyncCollector<string> outputEvents,
ILogger log)
{
var body = message.GetMessage();
var bodyString = Encoding.UTF8.GetString(body);
dynamic obj = JsonConvert.DeserializeObject<ExpandoObject>(bodyString);
obj.Topic = message.Topic;
await outputEvents.AddAsync(JsonConvert.SerializeObject(obj));
}
}
When deployed and run in the Azure portal, I get the following error messages:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: EventHubOutput
---> System.InvalidOperationException: Error while handling parameter outputEvents after function returned:
---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known
at (...)
Any idea what the problem might be?
Thank you.
You are using the bindings incorrectly. Check out RabbitMQ bindings for Azure Functions overview.
The following example shows a C# function that reads and logs the RabbitMQ message as a RabbitMQ Event:
[FunctionName("RabbitMQTriggerCSharp")]
public static void RabbitMQTrigger_BasicDeliverEventArgs(
[RabbitMQTrigger("queue", ConnectionStringSetting = "rabbitMQConnectionAppSetting")] BasicDeliverEventArgs args,
ILogger logger
)
{
logger.LogInformation($"C# RabbitMQ queue trigger function processed message: {Encoding.UTF8.GetString(args.Body)}");
}
The following example shows how to read the message as a POCO:
namespace Company.Function
{
public class TestClass
{
public string x { get; set; }
}
public class RabbitMQTriggerCSharp{
[FunctionName("RabbitMQTriggerCSharp")]
public static void RabbitMQTrigger_BasicDeliverEventArgs(
[RabbitMQTrigger("queue", ConnectionStringSetting = "rabbitMQConnectionAppSetting")] TestClass pocObj,
ILogger logger
)
{
logger.LogInformation($"C# RabbitMQ queue trigger function processed message: {pocObj}");
}
}
}
I recommend you to check out this complete guide to setup Rabbit MQ Trigger in Azure Functions: RabbitMQ trigger for Azure Functions overview
All the code samples I've seen so far for Azure WebJobs rely on some kind of trigger (e.g. TimerTrigger or QueueTrigger).
I am looking specifically at WebJobs SDK 3.x, by the way.
So. For a triggerless WebJob (Windows Service-alike one), am I expected to use NoAutomaticTrigger and find a way to kickoff my "main" code manually?
Or should I resort to implementing and registering a class that implements the IHostedService interface?
So far that's the approach I'm taking but it feels more of a hack than a recommended way.
I have not even tried to deploy this code and only ran it on my local machine, so I am afraid that the publishing process will confirm my code is not suitable for Azure WebJobs in its current form.
EntryPoint.cs
This is how the application is being bootstrap when the process is starting.
using Microsoft.Azure.ServiceBus;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
namespace AbcCorp.Jobs
{
public static class Program
{
static async Task Main(string[] args)
{
var config = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json", false)
.Build();
var hostBuilder = new HostBuilder()
.ConfigureWebJobs(builder => { builder.AddAzureStorageCoreServices(); })
.ConfigureServices(serviceCollection =>
{
ConfigureServices(serviceCollection, config);
serviceCollection.AddHostedService<ConsoleApplication>();
});
using (var host = hostBuilder.Build())
await host.RunAsync();
}
private static IServiceCollection ConfigureServices(IServiceCollection services, IConfigurationRoot configuration)
{
services.AddTransient<ConsoleApplication>();
// ... more DI registrations
return services;
}
}
}
ConsoleApplication.cs
This would normally be implemented as a function with a trigger.
The thing is, I want this code to only run once on the process startup.
It will start listening on the service bus events using the regular Microsoft.Azure.ServiceBus SDK package.
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;
using AbcCorp.Internal.Microsoft.Azure.ServiceBus;
using AbcCorp.Api.Messaging;
namespace AbcCorp.Jobs
{
public sealed class ConsoleApplication: IHostedService
{
private readonly IReceiver<SubmissionNotification> _messageReceiver;
private readonly MessageHandler _messageHandler;
public ConsoleApplication(IReceiver<SubmissionNotification> messageReceiver, MessageHandler messageHandler)
{
_messageReceiver = messageReceiver;
_messageHandler = messageHandler;
}
public Task StartAsync(CancellationToken cancellationToken)
{
_messageReceiver.StartListening(_messageHandler.HandleMessage, _messageHandler.HandleException);
return Task.Delay(Timeout.Infinite);
}
public Task StopAsync(CancellationToken cancellationToken)
{
_messageReceiver.Dispose();
return Task.CompletedTask;
}
}
}
So you want a console application to run in a WebJob and listen to messages. You don't really care about WebJob magic like triggers, it's just a place to run your console app. I've done the exact same thing before.
I found the IHostedService abstraction to be very helpful, but I didn't like their SDK. I found it bloated and hard to use. I didn't want to take a large dependency in order use a large array of special magic Azure stuff, when all I wanted to do was run a console application in a WebJob for now, and maybe move it elsewhere later.
So I ended just deleting that dependency, stealing the Shutdown code from the SDK and writing my own Service Host. The result is on my Github Repo azure-webjob-host. Feel free to use it or raid it for ideas. I don't know, maybe if I did it again I'd have another attempt at getting the SDK to work, but I present this is a bit of an alternative to the SDK.
Basically I wrote an IServiceHost not too different from yours (except that StartAsync exited when stuff started instead of just hanging). Then I wrote my own service host, which is basically just a loop:
await _service.StartAsync(cancellationToken);
while (!token.IsCancellationRequested){await Task.Delay(1000);}
await _service.StopAsync(default);
Then I stole the WebJobsShutdownWatcher code from their repo.
Then I created an IServiceHost that started my message handler. (I was using Rabbit, which has nothing to do with triggers or azure stuff)
public class MessagingService : IHostedService, IDisposable
{
public MessagingService(ConnectionSettings connectionSettings,
AppSubscriberSettings subscriberSettings,
MessageHandlerTypeMapping[] messageHandlerTypeMappings,
ILogger<MessagingService> logger)
{
....
}
public async Task StartAsync(CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
await Task.WhenAll(subscribers.Value.Select(s => s.StartSubscriptionAsync()));
}
public async Task StopAsync(CancellationToken cancellationToken)
{
...
}
public void Dispose()
{
...
}
}
Then I put that all together into something like this:
IHostedService myService = new MyService();
using (var host = new ServiceHostBuilder().HostService(myService))
{
await host.RunAsync(default);
}
I have some workers attached to service bus topics and what we do is the following (ServiceBusClient is a custom Class that contains our Subscription Client):
public override Task StartAsync(CancellationToken cancellationToken)
{
_serviceBusClient.RegisterOnMessageHandlerAndReceiveMessages(MessageReceivedAsync);
_logger.LogDebug($"Started successfully the Import Client. Listening for messages...");
return base.StartAsync(cancellationToken);
}
public void RegisterOnMessageHandlerAndReceiveMessages(Func<Message, CancellationToken, Task> ProcessMessagesAsync)
{
// Configure the message handler options in terms of exception handling, number of concurrent messages to deliver, etc.
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
// Maximum number of concurrent calls to the callback ProcessMessagesAsync(), set to 1 for simplicity.
// Set it according to how many messages the application wants to process in parallel.
MaxConcurrentCalls = 1,
// Indicates whether MessagePump should automatically complete the messages after returning from User Callback.
// False below indicates the Complete will be handled by the User Callback as in `ProcessMessagesAsync` below.
AutoComplete = false
};
// Register the function that processes messages.
SubscriptionClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
And then you can use DI to instantiate your service bus client and inject on the constructor of your Worker class.
Here i have the initialization of the singleton instance of my custom class Service Bus Client
services.AddSingleton<IServiceBusClient, ServiceBusClient>((p) =>
{
var diagnostics = p.GetService<EventHandling>();
var sbc = new ServiceBusClient(
programOptions.Endpoint,
programOptions.TopicName,
programOptions.Subscriber,
programOptions.SubscriberKey);
sbc.Exception += exception => diagnostics.HandleException(exception);
return sbc;
});
Then on this custom class, i initialize my subscription client
public ServiceBusClient(
string endpoint,
string topicName,
string subscriberName,
string subscriberKey, ReceiveMode mode = ReceiveMode.PeekLock)
{
var connBuilder = new ServiceBusConnectionStringBuilder(endpoint, topicName, subscriberName, subscriberKey);
var connectionString = connBuilder.GetNamespaceConnectionString();
ConnectionString = connectionString;
TopicName = topicName;
SubscriptionName = topicName;
SubscriptionClient = new SubscriptionClient(connectionString, topicName, subscriberName, mode);
}
You can check #george chen's answer from this post How to create service bus trigger webjob?
where instead of creating a receiver and registering a message handler, you can use the in built queue trigger and and write your message handler logic inside it.
Assume the following typical Queue Trigger function:
public void Run([QueueTrigger("queue1")]object data, ILogger log)
{
// Do something with data
}
My problem is that "queue1" has to be a constant field, so it has to be defined at compile time.
Also, I'd want to have a base class for Queue Triggers, that could work like this:
public abstract class QueueBase<TModel>
{
public void Run([QueueTrigger("queueName")]TModel data, ILogger log)
{
// Do something with data, log something etc.
OnRunExecuted(data);
// Do something with data, log something etc.
}
public abstract void OnRunExecuted(TModel data);
}
with this, I could write own classes which inherit from QueueBase but can even live inside a library which doesn't have Microsoft.Azure.WebJobs dependency:
public class MyQueueHandler : QueueBase<MyModel>
{
public void OnRunExecuted(MyModel data) => ...;
}
But it's impossible to pass in a Queue name... is it?
See binding expressions:
In short, you can pass in a variable queue name as "%queue-name-variable%"
[FunctionName("QueueTrigger")]
public static void Run(
[QueueTrigger("%queue-name-variable%")]string myQueueItem,
ILogger log)
{
log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
}
Where input-queue-name is defined in your configuration like
{"queue-name-variable": "queue-name-in-current-env"}
As i remember attribute QueueTrigger accept only const string, so you can try make some tricks using environment variables like in post how to pass dynamic queue name
I am attempting to convert my v1 function to a v2 function, but I cannot find a replacement for deferring a message.
In V1 of Azure Functions it was a method on the BrokeredMesage called .DeferAsync(). In V2 there is no longer a BrokeredMessage but just a Microsoft.Azure.ServiceBus.Message and this does not contain the method of .DeferAsync().
According to the docs:
The API is BrokeredMessage.Defer or BrokeredMessage.DeferAsync in the .NET Framework client, MessageReceiver.DeferAsync in the .NET Standard client, and mesageReceiver.defer or messageReceiver.deferSync in the Java client.
But how can I get access to the MessageReciever?
Here is an example of my function:
[FunctionName("MyFunction")]
public static void Run([ServiceBusTrigger("topic", "subscription", Connection = "AzureServiceBusPrimary")]Message message, ILogger log)
{
//Code
}
So does anyone know how to defer a V2 Message that is triggered from the Azure Service Bus?
As you mention, the new message receiver offers an async defer method and you can add this to your function by using the following code:
[FunctionName("MyFunction")]
public static async Task Run([ServiceBusTrigger("topic", "subscription", Connection = "AzureServiceBusPrimary")]Message message, string lockToken, MessageReceiver messageReceiver, ILogger log)
{
//Your function logic
await messageReceiver.DeferAsync(lockToken);
}