Any way to verify a service bus trigger to SendGridMessage succeeded before executing code? - c#

I've got an azure function right now that runs on a service bus trigger (queue trigger) and outputs a SendGridMessage. The trick is I need to do some cleanup in my blob storage after the function has successfully sent a sendgrid message but it seems like I have no way of identifying whether or not the function was successful until after it goes out of scope.
I'm currently attempting to push the message that needs to be cleaned up to a cleanup queue and take care of it after the try catch but I think I'm still running into the same problem. The function could succeed and then fail on the SendGrid output and the message would be cleaned up but thrown back into the queue to be reprocessed on this function and fail. Bleh.
Queue Trigger and Sendgrid Output
[FunctionName("ProcessEmail")]
public static void Run([ServiceBusTrigger("email-queue-jobs", AccessRights.Manage,
Connection = "MicroServicesServiceBus")]OutgoingEmail outgoingEmail, TraceWriter log,
[ServiceBus("email-queue-cleanup", Connection = "MicroServicesServiceBus",
EntityType = Microsoft.Azure.WebJobs.ServiceBus.EntityType.Queue)] IAsyncCollector<OutgoingEmail> cleanupEmailQueue,
[SendGrid] out SendGridMessage message)
{
try
{
log.Info($"Attempting to send the email {outgoingEmail.Id}");
message = SendgridHelper.ConvertToSendgridMessage(outgoingEmail);
log.Info("Successfully sent email:");
log.Info(JsonConvert.SerializeObject(outgoingEmail));
}
catch (Exception ex)
{
message = null;
throw ex;
}
// Add email to the cleanup queue
log.Info("Sending email to the cleanup queue.");
cleanupEmailQueue.AddAsync(outgoingEmail).Wait();
}

You should be able to achieve this by using ICollector or IAsyncCollector
[SendGrid] ICollector<SendGridMessage> messageCollector)
and then
var message = SendgridHelper.ConvertToSendgridMessage(outgoingEmail);
messageCollector.Add(message);
should call SendGrid synchronously and throw exception in case of failure.
If you want to use IAsyncCollector (as you already do for another binding), be sure to call FlushAsync method too:
[SendGrid] IAsyncCollector<SendGridMessage> messageCollector)
and then
var message = SendgridHelper.ConvertToSendgridMessage(outgoingEmail);
await messageCollector.AddAsync(message);
await messageCollector.FlushAsync();

Related

How to manually handle message completion in C# azure function with azure service bus 5.0.0

I'm writing an Azure Function to get the messages in an Azure Service Bus.
I want to handle any exceptions manually ("autoCompleteMessages": false)
Can't figure how to send the complete or abandon back to the service queue.
Tried Option 1:
[FunctionName("SBQ_F1_VC")]
public static async Task Run([ServiceBusTrigger("sbqfn1", Connection = "BrnlTest1_SERVICEBUS")]
ServiceBusReceivedMessage msg, ILogger log)
{
//.....
if(!Int32.TryParse(msg.ApplicationProperties.GetValueOrDefault("vid").ToString(), out vid))
{ await using ServiceBusClient client = new ServiceBusClient(Environment.GetEnvironmentVariable("BrnlTest1_SERVICEBUS"));
ServiceBusReceiver msgRcvr = client.CreateReceiver(Environment.GetEnvironmentVariable("queueName"), new ServiceBusReceiverOptions());
//await msgRcvr.RenewMessageLockAsync(msg);
await msgRcvr.AbandonMessageAsync(msg); //vid = 0;
}
//.....
}
Errors Option 1
System.Private.CoreLib: Exception while executing function: SBQ_F1_VC. Azure.Messaging.ServiceBus: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance. (MessageLockLost).
Tried Option 2:
[FunctionName("SBQ_F1_VC")]
public static async Task Run([ServiceBusTrigger("sbqfn1", Connection = "BrnlTest1_SERVICEBUS")]
ServiceBusReceivedMessage[] msgs,
ServiceBusMessageActions msgActions)
{
//.....
await msgActions.DeadLetterMessageAsync(msg);
}
Errors Option 2:
Microsoft.Azure.WebJobs.Host: Error indexing method 'SBQ_F1_VC'. Microsoft.Azure.WebJobs.Host: Cannot bind parameter 'msgActions' to type ServiceBusMessageActions. Make sure the parameter Type is supported by the binding. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).
Notes:
Service Bus is 5.0.0 so I have to use Azure.Messaging.ServiceBus namespace
This implies the message object is ServiceBusReceivedMessage
What worked (based on Jesse's and Josh's answers)
The Parameter Name for ServiceBusMessageActions MUST be messageActions.
Changing this name is not tolerated for some reason...
public static async Task Run(
[ServiceBusTrigger("sbqfn1", Connection = "BrnlTest1_SERVICEBUS")]
ServiceBusReceivedMessage[] msgs,
ServiceBusMessageActions messageActions)
{
You will need to use "messageActions" as the parameter name rather than "msgActions".
Locks are only valid on the AMQP link that the message was received from. In order to manually settle messages, you'll need to bind your trigger to the ServiceBusMessageActions. Those actions can be used for message settlement.
For example:
[FunctionName("BindingToMessageActions")]
public static async Task Run(
[ServiceBusTrigger("<queue_name>", Connection = "<connection_name>")]
ServiceBusReceivedMessage[] messages,
ServiceBusMessageActions messageActions)
{
foreach (ServiceBusReceivedMessage message in messages)
{
if (message.MessageId == "1")
{
await messageActions.DeadLetterMessageAsync(message);
}
else
{
await messageActions.CompleteMessageAsync(message);
}
}
}

Getting the error "Please define a receive (response) queue for the current resource before calling this method"

I am trying to read messages from the queue using C#. I am able to to publish the message to the queue but not able to retrieve message from the queue. Below is the sample code snippet.
Messages are already there in some queue, some one publishing messages to the queue i want to read those messages from the same queue.
using (var broker = Current.Resources.Allocate(resourceConfigUri) as IMessageSource)
{
if (broker != null)
{
Message messageResponse = null;
// now get the message
messageResponse = broker.Receive(); // getting the exception as mentioned.
// SerializationUtility.ToByteArray(this);
string messageRequestString = SerializationUtility.ToXml(messageRequest.Body);
string messageResponseString = SerializationUtility.ToXml(messageResponse.Body);
}
}
Here is the error:
Please define a receive (response) queue for the current resource before calling this method: resource:Tibco:Ems:Queue:EventBus.

c# sharp proejct .net 4.5 dependency, should interact with a azure message queue

I am currently working on a project that depends on .net 4.5, which I am why forced to use Microsoft.Servicebus - and thus message factory
Creating a messagefactory object, doesn't seem to be problem, but creating the message sender seem to be the issue here.
I am not sure what the entity path for my azure message queue is, and don't know how I should look it up?
I initialized my message factory as such;
public Microsoft.ServiceBus.Messaging.MessagingFactory client = Microsoft.ServiceBus.Messaging.MessagingFactory.CreateFromConnectionString(ServiceBusConnectString);
Where servicebusconnectionstring, is the primary conenctionstring extracted from the azure queue, or by following this guide: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues#1-create-a-namespace-using-the-azure-portal
The message sender is created as such:
Microsoft.ServiceBus.Messaging.MessageSender sender = client.CreateMessageSender(destinationQueue,queueName);
Again destionationQueue, and queueName are strings, which are defined from azure?
The message is to be send is created as such:
Microsoft.ServiceBus.Messaging.BrokeredMessage message = new Microsoft.ServiceBus.Messaging.BrokeredMessage(e.ToXml());
e is just an object, which has publich function which can convert it to an xml string.
Here is where I send the message:
try
{
IntegrationLogger.Write(LogLevel.Verbose, "Sending message to azure");
sender.Send(message);
}
catch(System.Exception ex)
{
if(ex is System.TimeoutException)
{
IntegrationLogger.Write(LogLevel.Error, "Timeout exeption");
}
if (ex is System.ArgumentException)
{
IntegrationLogger.Write(LogLevel.Error, "Message is empty");
}
}
IntegrationLogger is a log function I use - and the last message being logged is "Sending message to azure", meaning somethng goes wrong when I send it.
even catch does not catch the exception?..
What could be the problem here?
Am I using the sender incorrectly?
I am not sure what the enity path for my azure message queue is, and don't know how i should look it up?
If we want to create queue MessageSender, we could use following code.
var sender = client.CreateMessageSender(queueName);
We also could use queueclient to send and receive the broker messsage.
var client = QueueClient.CreateFromConnectionString("servicebus connection string", queueName);
// how to send message
var sendMessage = new BrokeredMessage(data);
client.Send(sendMessage);
//how to receive message
client.OnMessage(message =>
{
var dataInfo = message.GetBody<T>();
});
IntegrationLogger is a log function I use - and the last message being logged is "Sending message to azure", meaning somethng goes wrong when I send it. even catch does not catch the exepetion?..
In your case I recommand that you could add the log info to check whether there is any exception during send message.
try
{
IntegrationLogger.Write(LogLevel.Verbose, "Sending message to azure");
sender.Send(message);
//add log
IntegrationLogger.Write(LogLevel.Verbose, "Send message successfully");
}

Azure Service Bus SubscriptionClient.OnMessage always completes message when it shouldnt

I am trying to receive all messages for a given subscription to a Service Bus Topic, but for the context of this app I do not want them dead lettered at this time, I just want to view them and leave them on the subscription. Despite instantiating the Client as
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
and making sure that I am using message.Abandon() rather than message.Complete() the message always gets Dead-lettered after accessing the message. I also have options.AutoComplete set to false
full method code below:
public List<ServiceBusMessage> RetrieveSubscriptionMessages(Subscription sub) {
ServiceBusMessage sbm;
List<ServiceBusMessage> list = new List<ServiceBusMessage>();
String connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"].ToString();
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
sc.OnMessage((message) => {
try {
sbm = new ServiceBusMessage() {
topicName = sub.topicName,
messageText = message.GetBody<String>()
};
list.Add(sbm);
message.Abandon();
}
catch (Exception) {
message.Abandon();
throw;
}
}, options);
return list;
}
Am I missing something ? Or is there an issue with auto dead-lettering with the onMessage() method?
Thanks !
When a message is abandoned the service bus will immediately make it available for re-delivery to any subscriber of the topic.
If you are trying to configure a multicast mechanism in which multiple listeners all receive the same message, then understand that all listeners on a given subscription will be competing for the same message. In order for every listener to receive its own copy of the message, then simply create a unique subscription to the topic for each listener.
If your intent is to delay re-delivery of the abandoned message, you might look at the SO question: What's the proper way to abandon an Azure SB Message so that it becomes visible again in the future in a way I can control?

MSMQ receive with transaction - rollback not making message available again

I have this in a class called "MessageQueueReceive".
public MessageQueueTransaction BlockingReceive(out Message message)
{
MessageQueueTransaction tran = null;
message = null;
tran = new MessageQueueTransaction();
tran.Begin();
try
{
message = Queue.Receive(new TimeSpan(0, 0, 5), tran);
}
catch (MessageQueueException ex)
{
// If the exception was a timeout, then just continue
// otherwise re-raise it.
if (ex.MessageQueueErrorCode != MessageQueueErrorCode.IOTimeout)
throw ex;
}
return tran;
}
Then my processing loop has this:-
while (!Abort)
{
try
{
tran = this.Queue.BlockingReceive(out msg);
if (msg != null)
{
// Process message here
if (tran != null)
tran.Commit();
}
}
catch (Exception ex)
{
if (tran != null)
tran.Abort();
}
}
The control panel tool shows that the message queues I'm using are transactional. Journal queue is not enabled.
This code creates the queue:-
private static MessageQueue CreateMessageQueue(string queueName, bool transactional = false)
{
MessageQueue messageQueue = MessageQueue.Create(queueName, transactional);
messageQueue.SetPermissions("Administrators", MessageQueueAccessRights.FullControl,
AccessControlEntryType.Allow);
return messageQueue;
}
The transactional parameter is set as "true" when this is called.
What I find is that when an exception occurs during the processing of the message, tran.Abort is called but at that point I'd expect the message to be returned to the queue. However, this is not happening and the messages are lost.
Am I missing something obvious? Can anyone see what I'm doing wrong?
Thanks for all the comments. I did re-organise my code as Russell McClure suggested, and I tried to create simple test cases but could not reproduce the problem.
In the end, the problem was not at all where I was looking (how often does that happen?).
In my pipeline, I had a duplicate message checker. The "messages" my system deals with are from remote devices on a WAN, and occasionally messages on the wire are duplicated.
When a message was pulled from the MSMQ, it would pass via the duplicate checker the database writer. If the database writer failed, the duplicate checked did not remove the hash from its table. When the process tried to loop again, it would get the same message from the queue agan because the MSMQ transaction had been rolled back when the database writer failed. However, on the second attempt, the duplicate checker would spot that it had seen the message before, and swallow it silently.
The fix was to make the duplicate checker spot the exception coming from the next link in the chain, and roll-back anything it had done too.
Your queue needs to be created as a transactional queue to get what you want.
EDIT:
Well, if your queue is transactional then that points to the fact that you are mishandling your transaction, although I can't see specifically how it is happening. I would change your BlockingReceive method to return the message. I would move the creation of the MessageQueueTransaction to the outer method. Your code will be much more maintainable if you have the Begin, Commit and Abort method calls in the same method.

Categories