I found two ways to send messages into service bus topic from azure function.
one is using output -
[FunctionName("ServiceBusOutput")]
[return: ServiceBus("myqueue", Connection = "ServiceBusConnection")]
public static string ServiceBusOutput([HttpTrigger] dynamic input, ILogger log)
{
log.LogInformation($"C# function processed: {input.Text}");
return input.Text;
}
Another is using code -
const string ServiceBusConnectionString = "string";
const string TopicName = "topicName";
static ITopicClient topicClient;
topicClient = new TopicClient(ServiceBusConnectionString, TopicName);
string messageBody = "Test";
var message = new Message(Encoding.UTF8.GetBytes(messageBody));
await topicClient.SendAsync(message);
I'm not getting which one we should use and when?
if we use Output how to pass queue name myqueue as a variable
so that in code we can assign it.
if i have array how can we return one by one message to output
return which will send one by one message to queue ?
Full examples from here. For instance, how to write multiple messages, using the ICollector
public static void Run(TimerInfo myTimer, ILogger log, [ServiceBus("myqueue", Connection = "ServiceBusConnection")] ICollector<string> outputSbQueue)
{
string message = $"Service Bus queue messages created at: {DateTime.Now}";
log.LogInformation(message);
outputSbQueue.Add("1 " + message);
outputSbQueue.Add("2 " + message);
}
As far as I know the first version, using return does not work if you have any async calls inside your Function. The version using the collector can also work in async Functions but simply using an IAsyncCollector instead.
if we use Output how to pass queue name myqueue as a variable
so that in code we can assign it.
For this you can use Imperative Binding.Imperative binding is useful when binding parameters need to be computed at runtime rather than design time. More reference here
Example:
public static async Task ServiceBusBinderTest(
string message,
int numMessages,
Binder binder) {
var attribute = new ServiceBusAttribute(BinderQueueName) {
EntityType = EntityType.Queue
};
var collector = await binder.BindAsync < IAsyncCollector < string >> (attribute);
for (int i = 0; i < numMessages; i++) {
await collector.AddAsync(message + i);
}
await collector.FlushAsync();
}
if i have array how can we return one by one message to output return which will send one by one message to queue ?
You can configure the OnMessageOptions instance with decreasing your MaxConcurrentCalls
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.MaxConcurrentCalls = 5;
Related
I want to use MassTransit to send messages that may have different structures in terms of message.Data, to different Azure Service Bus queues. As long as the router.Name keeps the initial value, it works welll. But, whenever the destination Uri of EndpointConvention.Map<ManyToOneTransferMessage> changes, an exception is thrown by MassTransit as "The endpoint convention has already been created and can no longer be modified". Is there any way to remap the message type with another destination to use MassTransit with multiple queues?
public class AzureServiceBusManager
{
string ServiceBusConnectionString = string.Empty;
public AzureServiceBusManager()
{
ServiceBusConnectionString = ConfigurationManager.AppSettings["AppSettings:ServiceBusConnectionString"];
}
public async Task SendMessageAsyncN1(TransferMessage transferMessage, Router router)
{
var message = new ManyToOneTransferMessage
{
BlobFileName = transferMessage.BlobFileName,
Compressed = transferMessage.Compressed,
Data = transferMessage.Data,
MessageId = transferMessage.MessageId,
TransferId = transferMessage.TransferId,
TransferType = transferMessage.TransferType
};
var queueBusControl = Bus.Factory.CreateUsingAzureServiceBus(
cfg =>
{
cfg.Host(ServiceBusConnectionString);
EndpointConvention.Map<ManyToOneTransferMessage>(new Uri("queue:" + router.Name));
cfg.ReceiveEndpoint(router.Name, e =>
{
e.RequiresSession = true;
e.MaxConcurrentCalls = 500;
});
});
await queueBusControl.Send(message);
}
}
So, first of all, do not use EndpointConvention.Map<ManyToOneTransferMessage>(new Uri("queue:" + router.Name));. It isn't useful, and only adds to the confusion.
You can resolve the endpoint from the bus, but you have to realize that creating a bus for each call is a bad idea. It is best to start the bus at startup (you aren't even starting it in the code above), and stop it at application shutdown.
Then, for each call, you can use that bus to resolve the send endpoint and send the message.
var endpoint = await bus.GetSendEndpoint(new Uri("queue:" + router.Name));
await endpoint.Send(message);
Also, you should remove this since it will cause all messages to be moved to the _skipped queue:
cfg.ReceiveEndpoint(router.Name, e =>
{
e.RequiresSession = true;
e.MaxConcurrentCalls = 500;
});
You'll likely need to configure the queues separately, in advance, if you requireSession, although I don't see you setting a SessionId on the message so it likely will not work anyway without one.
I have a c# WebApi project in which the users can make orders in a website, each order after payment complete will execute a function called ConfirmOrder which will update the order status from STB to COMPLETED.
In the following function that looks like this:
public static void ConfirmOrder(string piva, string orderID, double importo = 0, string transazione = "", string paymentID = "", string tipo = "MENU")
{
string connectionString = getConnectionString(piva);
using var connection = new MySqlConnection(connectionString);
string query_menu = "QUERY";
string query_pagamenti = "QUERY";
using var cmd = new MySqlCommand(query_pagamenti, connection);
connection.Open();
cmd.Parameters.AddWithValue("#tipo", tipo.ToUpper());
cmd.Parameters.AddWithValue("#importo", importo);
cmd.Parameters.AddWithValue("#transazione", transazione);
cmd.Parameters.AddWithValue("#dataOra", DateTime.Now);
cmd.Parameters.AddWithValue("#orderID", orderID);
cmd.Parameters.AddWithValue("#paymentID", paymentID);
cmd.Prepare();
cmd.ExecuteNonQuery();
cmd.CommandText = query_menu;
cmd.ExecuteNonQuery();
if (!tipo.Equals("MENU"))
{
EmailHelper.SendRiepilogo(piva, int.Parse(orderID)); // SENDING SUMMARY MAIL
}
}
I'm calling another function SendRiepilogo which sends a summary to the user and the shop, but in this case i can't wait for that function response but it have to be executed for it's own without stucking ConfirmOrder callback.. so i can't wait for SendRiepilogo to be executed, at this point i've read about IHostingService, but i can't figure out on how i could migrate my SendRiepilogo to a IHostingService and run it from ConfirmOrder...
My SendRiepilogo looks like this:
public static async void SendRiepilogo(string piva, int idOrdine)
{
var order = GetOrdine(piva, idOrdine);
if (order == null)
{
return;
}
try
{
var negozio = getNegozio(order.idNegozio);
var from = new MailAddress("ordini#visualorder.it", "VisualOrder");
var to = new MailAddress(order.cliente.FirstOrDefault().email);
using MemoryStream ms = new MemoryStream();
QRCodeGenerator qrGenerator = new QRCodeGenerator();
QRCodeData qrCodeData = qrGenerator.CreateQrCode("vo/" + idOrdine, QRCodeGenerator.ECCLevel.Q);
Base64QRCode qrCode = new Base64QRCode(qrCodeData);
byte[] byteQr = Convert.FromBase64String(qrCode.GetGraphic(20));
MemoryStream streamQr = new MemoryStream(byteQr);
var qrImage = new LinkedResource(streamQr, MediaTypeNames.Image.Jpeg)
{
ContentId = "qrImage"
};
string nome = order.cliente.FirstOrDefault().nome;
var orderEmail = new { idOrdine, order, nome, negozio };
byte[] byteLogo = Convert.FromBase64String(System.Text.Encoding.UTF8.GetString(negozio.logo));
MemoryStream streamLogo = new MemoryStream(byteLogo);
var logoImage = new LinkedResource(streamLogo, MediaTypeNames.Image.Jpeg)
{
ContentId = "logoImage"
};
string template = File.ReadAllText("Views/Emails/EmailRiepilogo.cshtml");
var htmlBody = Engine.Razor.RunCompile(template, "riepilogo", null, orderEmail);
AlternateView alternateView = AlternateView.CreateAlternateViewFromString(htmlBody, null, MediaTypeNames.Text.Html);
alternateView.LinkedResources.Add(qrImage);
alternateView.LinkedResources.Add(logoImage);
var message = new MailMessage(from, to)
{
Subject = "Riepilogo ordine",
Body = htmlBody
};
message.IsBodyHtml = true;
message.AlternateViews.Add(alternateView);
using var smtp = new SmtpClient("smtps.aruba.it", 587)
{
EnableSsl = true,
Credentials = new NetworkCredential("XXX", "XXX")
};
await smtp.SendMailAsync(message); // sending email to user
await smtp.SendMailAsync(MessageNegozio(order, idOrdine, negozio)); // sending email to shop
}
catch (Exception e)
{
return;
}
ConfirmEmail(piva, idOrdine); // setting "EMAIL SENT" flag in DB to true
return;
}
A background (hosted) service is a completely different service, using its own thread to do its job. You can't have your controller "run" something on that service, you have to tell it what to do, and have it do it.
The Background tasks with hosted services section in the docs shows two different ways a long running background service can work :
A timed service can run each time a timer fires and do a periodic job, as long as the application is running
A queued service waits for messages in a queue and performs a job when a message arrives
Sending an email fits into the second case. You could use the documentation example almost as-is. You can create an IBackgroundTaskQueue interface that clients like your controller can use to submit jobs to run in the background:
public interface IBackgroundTaskQueue
{
void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem);
Task<Func<CancellationToken, Task>> DequeueAsync(
CancellationToken cancellationToken);
}
This interface can be added as a dependency in your container's constructor.
Assuming the injected service is called myJobQueue, the controller can enqueue a job to run in the background with :
IBackgroundTaskQueue _myJobQueue
public MyController(IBackgroundTaskQueue myJobQueue)
{
_myJobQueue=myJobQueue;
}
public void ConfirmOrder(...)
{
...
if (!tipo.Equals("MENU"))
{
var ordId=int.Parse(orderID);
_myJobQueue.QueueBackgroundWorkItem(ct=>EmailHelper.SendRiepilogoAsync(piva,ordId ));
}
async void should only be used for asynchronous event handlers. That's not what SendRiepilogo is. async void methods can't be awaited, they are essentially fire-and-forget methods that may never run, as the application doesn't know it has to await them. The correct syntax should be :
public static async Task SendRiepilogoAsync(string piva, int idOrdine)
{
...
}
The rest of the documentation example can be used as-is.
Simplifying the service
Instead of a generic queued service that runs any available job, you could create a queue that accepts specific message classes only, only an address and order ID, and have the service do the job of retrieving any data and sending the email. Essentially, SendRiepilogoAsync becomes part of the background service. This allows creating services that could eg batch emails, send several emails concurrently, apply throttling etc.
This would allow reusing expensive resources or perform expensive operations just once, eg create the SmptClient and authenticate before starting to process queue messages
My client is attempting to send messages to the receiver. However I noticed that the receiver sometimes does not receive all the messages sent by the client thus missing a few messages (not sure where the problem is ? Client or the receiver).
Any suggestions on why that might be happening. This is what I am currently doing
On the receiver side this is what I am doing.
This is the Event Processor
async Task IEventProcessor.ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
{
foreach (var eventData in messages)
{
var data = Encoding.UTF8.GetString(eventData.Body.Array, eventData.Body.Offset, eventData.Body.Count);
}
}
This is how the client connects to the event hub
var StrBuilder = new EventHubsConnectionStringBuilder(eventHubConnectionString)
{
EntityPath = eventHubName,
};
this.eventHubClient = EventHubClient.CreateFromConnectionString(StrBuilder.ToString());
How do I direct my messages to specific consumers
I'm using this sample code from eventhub official doc, for sending and receiving.
And I have 2 consumer groups: $Default and newcg. Suppose you have 2 clients, the client_1 are using the default consumer group($Default), and client_2 are using the other consumer group(newcg)
First, after create the send client, in the SendMessagesToEventHub method, we need to add a property with value. The value should be the consumer group name. Sample code like below:
private static async Task SendMessagesToEventHub(int numMessagesToSend)
{
for (var i = 0; i < numMessagesToSend; i++)
{
try
{
var message = "444 Message";
Console.WriteLine($"Sending message: {message}");
EventData mydata = new EventData(Encoding.UTF8.GetBytes(message));
//here, we add a property named "cg", it's value is the consumer group. By setting this property, then we can read this message via this specified consumer group.
mydata.Properties.Add("cg", "newcg");
await eventHubClient.SendAsync(mydata);
}
catch (Exception exception)
{
Console.WriteLine($"{DateTime.Now} > Exception: {exception.Message}");
}
await Task.Delay(10);
}
Console.WriteLine($"{numMessagesToSend} messages sent.");
}
Then in the client_1, after create the receiver project, which use the default consumer group($Default)
-> in the SimpleEventProcessor class -> ProcessEventsAsync method, we can filter out the unnecessary event data. Sample code for ProcessEventsAsync method:
public Task ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
{
foreach (var eventData in messages)
{
//filter the data here
if (eventData.Properties["cg"].ToString() == "$Default")
{
var data = Encoding.UTF8.GetString(eventData.Body.Array, eventData.Body.Offset, eventData.Body.Count);
Console.WriteLine($"Message received. Partition: '{context.PartitionId}', Data: '{data}'");
Console.WriteLine(context.ConsumerGroupName);
}
}
return context.CheckpointAsync();
}
And in another client, like client_2, which use another consumer group, like it's name is newcg, we can follow the steps in client_1, just a little changes in ProcessEventsAsync method, like below:
public Task ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
{
foreach (var eventData in messages)
{
//filter the data here, using another consumer group name
if (eventData.Properties["cg"].ToString() == "newcg")
{
//other code
}
}
return context.CheckpointAsync();
}
This happens only when there are 2 or more Event Processor Host reading from same consumer group.
If you have event hub with 32 partitions and 2 event processor host reading from same consumer group. Then each event processor host will read from 16 partition and so on.
Similarly if 4 Event processor host parallelly reading from same consumer group then each will read from 8 partitions.
Check if you have 2 or more event processor host running on same consumer group.
I have tested your code and slightly modified it(different overload of EventProcessorHost constructor, and added CheckpointAsync after consuming the messages), and then did some tests.
By using the default implementation and default EventProcessorOptions(EventProcessorOptions.DefaultOptions) I can say that I did experience some latency when it comes to consuming messages, but all messages were processed successfully.
So, sometimes it seems like I am not getting the messages from the certain partition, but after a certain period of time, all messages arrive:
Here you can find the actual modified code that worked for me. It is a simple console app that prints to the console if something arrives.
string processorHostName = Guid.NewGuid().ToString();
var Options = new EventProcessorOptions()
{
MaxBatchSize = 1, //not required to make it working, just for testing
};
Options.SetExceptionHandler((ex) =>
{
System.Diagnostics.Debug.WriteLine($"Exception : {ex}");
});
var eventHubCS = "event hub connection string";
var storageCS = "storage connection string";
var containerName = "test";
var eventHubname = "test2";
EventProcessorHost eventProcessorHost = new EventProcessorHost(eventHubname, "$Default", eventHubCS, storageCS, containerName);
eventProcessorHost.RegisterEventProcessorAsync<MyEventProcessor>(Options).Wait();
For sending the messages to the event hub and testing I used this message publisher app.
I am developing a chatbot using the Microsoft bot framework in C#. We have a functionality where it queries the database and returns the result, but it might take up to 25-30 secs for the result to return.
By that time bot says "cannot send,please retry". Is there a way to increase this timeout? Or can we have something like "please wait" message for the user so that user will know that the request is processing?
It's hard coded in SDK, we're not able to override the message like "Couldn't send, retry". As Nicolas said, a workaround is to send a proactive message to user.
For example you can firstly create a ConversationStarter.cs class like this:
public class ConversationStarter
{
//Note: Of course you don't want these here. Eventually you will need to save these in some table
//Having them here as static variables means we can only remember one user :)
public static string fromId;
public static string fromName;
public static string toId;
public static string toName;
public static string serviceUrl;
public static string channelId;
public static string conversationId;
//This will send an adhoc message to the user
public static async Task Resume(string conversationId, string channelId)
{
var userAccount = new ChannelAccount(toId, toName);
var botAccount = new ChannelAccount(fromId, fromName);
var connector = new ConnectorClient(new Uri(serviceUrl));
IMessageActivity message = Activity.CreateMessageActivity();
if (!string.IsNullOrEmpty(conversationId) && !string.IsNullOrEmpty(channelId))
{
message.ChannelId = channelId;
}
else
{
conversationId = (await connector.Conversations.CreateDirectConversationAsync(botAccount, userAccount)).Id;
}
message.From = botAccount;
message.Recipient = userAccount;
message.Conversation = new ConversationAccount(id: conversationId);
message.Text = "Hello, work is done!";
message.Locale = "en-Us";
await connector.Conversations.SendToConversationAsync((Activity)message);
}
}
Then in your dialog, you can code like this:
public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> result)
{
var message = await result;
//We need to keep this data so we know who to send the message to. Assume this would be stored somewhere, e.g. an Azure Table
ConversationStarter.toId = message.From.Id;
ConversationStarter.toName = message.From.Name;
ConversationStarter.fromId = message.Recipient.Id;
ConversationStarter.fromName = message.Recipient.Name;
ConversationStarter.serviceUrl = message.ServiceUrl;
ConversationStarter.channelId = message.ChannelId;
ConversationStarter.conversationId = message.Conversation.Id;
await context.PostAsync("Please wait, we're processing...");
Processing();
}
public async Task Processing()
{
//replace the task.delay() method with your task.
await Task.Delay(30000).ContinueWith((t) =>
{
ConversationStarter.Resume(ConversationStarter.conversationId, ConversationStarter.channelId);
});
}
Then Task.Delay(30000) method is used for a 30s task testing, you should be able to replace it with your task for retrieving data from your database.
You should do the following:
acknowledge the request of the user with a basic text reply
save the message information and process the request
make a proactive message once you got the reply of your system
I am trying to write an integration / acceptance test to test some code in azure, the code in the question ATM simply subscribes to one topic and publishes to another.
I have written the test but it doesn't always pass, seems as though there could be a race condition in place. I've tried writing it a couple of ways including using OnMessage and also using Receive (example I show here).
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
My query concerning my example specifically, I assumed that once I created the subscription to the target topic, that any message sent to it I would be able to pickup using Receive(), whatever point in time that message arrived meaning, if the message arrives at the target topic before I call Receive(), I would still be able to read the message afterward by calling Receive(). Could anyone please shed any light on this?
namespace somenamespace {
[TestClass]
public class SampleTopicTest
{
private static TopicClient topicClient;
private static SubscriptionClient subClientKoEligible;
private static SubscriptionClient subClientKoIneligible;
private static OnMessageOptions options;
public const string TEST_MESSAGE_SUB = "TestMessageSub";
private static NamespaceManager namespaceManager;
private static string topicFleKoEligible;
private static string topicFleKoIneligible;
private BrokeredMessage message;
[ClassInitialize]
public static void BeforeClass(TestContext testContext)
{
//client for publishing messages
string connectionString = ConfigurationManager.AppSettings["ServiceBusConnectionString"];
string topicDataReady = ConfigurationManager.AppSettings["DataReadyTopicName"];
topicClient = TopicClient.CreateFromConnectionString(connectionString, topicDataReady);
topicFleKoEligible = ConfigurationManager.AppSettings["KnockOutEligibleTopicName"];
topicFleKoIneligible = ConfigurationManager.AppSettings["KnockOutIneligibleTopicName"];
//create test subscription to receive messages
namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.SubscriptionExists(topicFleKoEligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoEligible, TEST_MESSAGE_SUB);
}
if (!namespaceManager.SubscriptionExists(topicFleKoIneligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoIneligible, TEST_MESSAGE_SUB);
}
//subscriber client koeligible
subClientKoEligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoEligible, TEST_MESSAGE_SUB);
subClientKoIneligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoIneligible, TEST_MESSAGE_SUB);
options = new OnMessageOptions()
{
AutoComplete = false,
AutoRenewTimeout = TimeSpan.FromMinutes(1),
};
}
[TestMethod]
public void E2EPOCTopicTestLT50()
{
Random rnd = new Random();
string customerId = rnd.Next(1, 49).ToString();
FurtherLendingCustomer sentCustomer = new FurtherLendingCustomer { CustomerId = customerId };
BrokeredMessage sentMessage = new BrokeredMessage(sentCustomer.ToJson());
sentMessage.CorrelationId = Guid.NewGuid().ToString();
string messageId = sentMessage.MessageId;
topicClient.Send(sentMessage);
Boolean messageRead = false;
//wait for message to arrive on the ko eligible queue
while((message = subClientKoEligible.Receive(TimeSpan.FromMinutes(2))) != null){
//read message
string messageString = message.GetBody<String>();
//Serialize
FurtherLendingCustomer receivedCustomer = JsonConvert.DeserializeObject<FurtherLendingCustomer>(messageString.Substring(messageString.IndexOf("{")));
//assertion
Assert.AreEqual(sentCustomer.CustomerId, receivedCustomer.CustomerId,"verify customer id");
//pop message
message.Complete();
messageRead = true;
//leave loop after processing one message
break;
}
if (!messageRead)
Assert.Fail("Didn't receive any message after 2 mins");
}
}
}
As the official document states about SubscriptionClient.Receive(TimeSpan):
Parameters
serverWaitTime
TimeSpan
The time span the server waits for receiving a message before it times out.
A Null can be return by this API if operation exceeded the timeout specified, or the operations succeeded but there are no more messages to be received.
Per my test, if a message sent to the topic and then delivered to your subscription within your specific serverWaitTime, then you could receive a message no matter whether the message arrives at the target topic before or after you call Receive.
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
[TestMethod]
public void ReceiveMessages()
{
subClient.OnMessage(msg => {
System.Diagnostics.Trace.TraceInformation($"{DateTime.Now}：{msg.GetBody<string>()}");
msg.Complete();
});
Task.Delay(TimeSpan.FromMinutes(5)).Wait();
}
For SubscriptionClient.OnMessage, I assumed that it basically a loop invoking Receive. After calling OnMessage, you need to wait for a while and stop this method to exit. Here is a blog about the Event-Driven message programming for windows Azure Service Bus, you could refer to here.
Additionally, I found that your topicClient for sending messages and the subClientKoEligible for receiving a message are not targeted at the same topic path.