We Keep Coding

How to map message type to a different Azure ServiceBus queue with MassTransit EndpointConvention.Map<T>

I want to use MassTransit to send messages that may have different structures in terms of message.Data, to different Azure Service Bus queues. As long as the router.Name keeps the initial value, it works welll. But, whenever the destination Uri of EndpointConvention.Map<ManyToOneTransferMessage> changes, an exception is thrown by MassTransit as "The endpoint convention has already been created and can no longer be modified". Is there any way to remap the message type with another destination to use MassTransit with multiple queues?
public class AzureServiceBusManager
{
string ServiceBusConnectionString = string.Empty;
public AzureServiceBusManager()
{
ServiceBusConnectionString = ConfigurationManager.AppSettings["AppSettings:ServiceBusConnectionString"];
}
public async Task SendMessageAsyncN1(TransferMessage transferMessage, Router router)
{
var message = new ManyToOneTransferMessage
{
BlobFileName = transferMessage.BlobFileName,
Compressed = transferMessage.Compressed,
Data = transferMessage.Data,
MessageId = transferMessage.MessageId,
TransferId = transferMessage.TransferId,
TransferType = transferMessage.TransferType
};
var queueBusControl = Bus.Factory.CreateUsingAzureServiceBus(
cfg =>
{
cfg.Host(ServiceBusConnectionString);
EndpointConvention.Map<ManyToOneTransferMessage>(new Uri("queue:" + router.Name));
cfg.ReceiveEndpoint(router.Name, e =>
{
e.RequiresSession = true;
e.MaxConcurrentCalls = 500;
});
});
await queueBusControl.Send(message);
}
}

So, first of all, do not use EndpointConvention.Map<ManyToOneTransferMessage>(new Uri("queue:" + router.Name));. It isn't useful, and only adds to the confusion.
You can resolve the endpoint from the bus, but you have to realize that creating a bus for each call is a bad idea. It is best to start the bus at startup (you aren't even starting it in the code above), and stop it at application shutdown.
Then, for each call, you can use that bus to resolve the send endpoint and send the message.
var endpoint = await bus.GetSendEndpoint(new Uri("queue:" + router.Name));
await endpoint.Send(message);
Also, you should remove this since it will cause all messages to be moved to the _skipped queue:
cfg.ReceiveEndpoint(router.Name, e =>
{
e.RequiresSession = true;
e.MaxConcurrentCalls = 500;
});
You'll likely need to configure the queues separately, in advance, if you requireSession, although I don't see you setting a SessionId on the message so it likely will not work anyway without one.

Blocking connection to Azure storage account

I have an application developed with c# which the first functionality is a method that connect to a storage account in order to be able to manage blobs.
My problem is that I want to block connection after 3 essaies of trying to connect.
this is the method that represent the connection to the storage account
public bool Connect(out String strerror)
{
strerror = "";
try
{
storageAccount = new CloudStorageAccount(new StorageCredentials(AccountName, AccountConnectionString), true);
MSAzureBlobStorageGUILogger.TraceLog(MessageType.Control,CommonMessages.ConnectionSuccessful);
return true;
}
catch (Exception ex01)
{
Console.WriteLine(CommonMessages.ConnectionFailed + ex01.Message);
strerror =CommonMessages.ConnectionFailed +ex01.Message;
return false;
}
}

At the moment you create the CloudStorageAccount variable there's still no connection made to the Storage Account, which you can easily test out by adding random credentials. In the background all the library does is fire a REST call to the Storage API and therefore doesn't make any connection until you actually retrieve or send data.
The library also already has its own mechanism implemented to retry requests in case of failures, which defaults to 3 retries but you can change manually like this:
var options = new BlobRequestOptions()
{
RetryPolicy = new ExponentialRetry(deltaBackoff, maxAttempts),
};
cloudBlobClient.DefaultRequestOptions = options;

What about wrapping it in a while loop and continuing to retry until either success or hitting the 3 attempt maximum?
string strError;
const int maxConnectionAttempts = 3;
var connectionAttempts = 0;
var connected = false;
while (!connected && connectionAttempts < maxConnectionAttempts)
{
connected = Connect(out strError);
connectionAttempts++;
}

Integration Test to publish to a topic and subscribe to another in Azure Service Bus is unreliable is there a race condition?

I am trying to write an integration / acceptance test to test some code in azure, the code in the question ATM simply subscribes to one topic and publishes to another.
I have written the test but it doesn't always pass, seems as though there could be a race condition in place. I've tried writing it a couple of ways including using OnMessage and also using Receive (example I show here).
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
My query concerning my example specifically, I assumed that once I created the subscription to the target topic, that any message sent to it I would be able to pickup using Receive(), whatever point in time that message arrived meaning, if the message arrives at the target topic before I call Receive(), I would still be able to read the message afterward by calling Receive(). Could anyone please shed any light on this?
namespace somenamespace {
[TestClass]
public class SampleTopicTest
{
private static TopicClient topicClient;
private static SubscriptionClient subClientKoEligible;
private static SubscriptionClient subClientKoIneligible;
private static OnMessageOptions options;
public const string TEST_MESSAGE_SUB = "TestMessageSub";
private static NamespaceManager namespaceManager;
private static string topicFleKoEligible;
private static string topicFleKoIneligible;
private BrokeredMessage message;
[ClassInitialize]
public static void BeforeClass(TestContext testContext)
{
//client for publishing messages
string connectionString = ConfigurationManager.AppSettings["ServiceBusConnectionString"];
string topicDataReady = ConfigurationManager.AppSettings["DataReadyTopicName"];
topicClient = TopicClient.CreateFromConnectionString(connectionString, topicDataReady);
topicFleKoEligible = ConfigurationManager.AppSettings["KnockOutEligibleTopicName"];
topicFleKoIneligible = ConfigurationManager.AppSettings["KnockOutIneligibleTopicName"];
//create test subscription to receive messages
namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.SubscriptionExists(topicFleKoEligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoEligible, TEST_MESSAGE_SUB);
}
if (!namespaceManager.SubscriptionExists(topicFleKoIneligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoIneligible, TEST_MESSAGE_SUB);
}
//subscriber client koeligible
subClientKoEligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoEligible, TEST_MESSAGE_SUB);
subClientKoIneligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoIneligible, TEST_MESSAGE_SUB);
options = new OnMessageOptions()
{
AutoComplete = false,
AutoRenewTimeout = TimeSpan.FromMinutes(1),
};
}
[TestMethod]
public void E2EPOCTopicTestLT50()
{
Random rnd = new Random();
string customerId = rnd.Next(1, 49).ToString();
FurtherLendingCustomer sentCustomer = new FurtherLendingCustomer { CustomerId = customerId };
BrokeredMessage sentMessage = new BrokeredMessage(sentCustomer.ToJson());
sentMessage.CorrelationId = Guid.NewGuid().ToString();
string messageId = sentMessage.MessageId;
topicClient.Send(sentMessage);
Boolean messageRead = false;
//wait for message to arrive on the ko eligible queue
while((message = subClientKoEligible.Receive(TimeSpan.FromMinutes(2))) != null){
//read message
string messageString = message.GetBody<String>();
//Serialize
FurtherLendingCustomer receivedCustomer = JsonConvert.DeserializeObject<FurtherLendingCustomer>(messageString.Substring(messageString.IndexOf("{")));
//assertion
Assert.AreEqual(sentCustomer.CustomerId, receivedCustomer.CustomerId,"verify customer id");
//pop message
message.Complete();
messageRead = true;
//leave loop after processing one message
break;
}
if (!messageRead)
Assert.Fail("Didn't receive any message after 2 mins");
}
}
}

As the official document states about SubscriptionClient.Receive(TimeSpan):
Parameters
serverWaitTime
TimeSpan
The time span the server waits for receiving a message before it times out.
A Null can be return by this API if operation exceeded the timeout specified, or the operations succeeded but there are no more messages to be received.
Per my test, if a message sent to the topic and then delivered to your subscription within your specific serverWaitTime, then you could receive a message no matter whether the message arrives at the target topic before or after you call Receive.
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
[TestMethod]
public void ReceiveMessages()
{
subClient.OnMessage(msg => {
System.Diagnostics.Trace.TraceInformation($"{DateTime.Now}：{msg.GetBody<string>()}");
msg.Complete();
});
Task.Delay(TimeSpan.FromMinutes(5)).Wait();
}
For Subscription​Client.​On​Message, I assumed that it basically a loop invoking Receive. After calling OnMessage, you need to wait for a while and stop this method to exit. Here is a blog about the Event-Driven message programming for windows Azure Service Bus, you could refer to here.
Additionally, I found that your topicClient for sending messages and the subClientKoEligible for receiving a message are not targeted at the same topic path.

Azure Service Bus SubscriptionClient.OnMessage always completes message when it shouldnt

I am trying to receive all messages for a given subscription to a Service Bus Topic, but for the context of this app I do not want them dead lettered at this time, I just want to view them and leave them on the subscription. Despite instantiating the Client as
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
and making sure that I am using message.Abandon() rather than message.Complete() the message always gets Dead-lettered after accessing the message. I also have options.AutoComplete set to false
full method code below:
public List<ServiceBusMessage> RetrieveSubscriptionMessages(Subscription sub) {
ServiceBusMessage sbm;
List<ServiceBusMessage> list = new List<ServiceBusMessage>();
String connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"].ToString();
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
sc.OnMessage((message) => {
try {
sbm = new ServiceBusMessage() {
topicName = sub.topicName,
messageText = message.GetBody<String>()
};
list.Add(sbm);
message.Abandon();
}
catch (Exception) {
message.Abandon();
throw;
}
}, options);
return list;
}
Am I missing something ? Or is there an issue with auto dead-lettering with the onMessage() method?
Thanks !

When a message is abandoned the service bus will immediately make it available for re-delivery to any subscriber of the topic.
If you are trying to configure a multicast mechanism in which multiple listeners all receive the same message, then understand that all listeners on a given subscription will be competing for the same message. In order for every listener to receive its own copy of the message, then simply create a unique subscription to the topic for each listener.
If your intent is to delay re-delivery of the abandoned message, you might look at the SO question: What's the proper way to abandon an Azure SB Message so that it becomes visible again in the future in a way I can control?

EasyNetQ. Advanced API - Publish does not produce response on RabbitServer

Looking at EasyNetQ as replacement for our current library for MQ communication.
For Testing im trying to simply publish a number of messages to an exchange, using a custom naming strategy.
My method for publishing is in t he small test method below>
public void PublishTest()
{
var advancedBus = RabbitHutch.CreateBus("host=localhost;virtualHost=Test;username=guest;password=guest;").Advanced;
var routingKey = "SimpleMessage";
// declare some objects
var queue = advancedBus.QueueDeclare("Q.TestQueue.SimpleMessage");
var exchange = advancedBus.ExchangeDeclare("E.TestExchange.SimpleMessage", ExchangeType.Direct);
var binding = advancedBus.Bind(exchange, queue, routingKey);
var message = new SimpleMessage() {Test = "HELLO"};
for (int i = 0; i < 100; i++)
{
advancedBus.Publish(exchange, routingKey, true, true, new Message<SimpleMessage>(message));
}
advancedBus.Dispose();
}
The problem is that even thou the Exchange and Queue is created, and bound proper, publishing does not produce anything.
No messages hit the queue.
The graph in the Rabbit MQ management interface does not even show any activity on the exchange.
Am i missing something here? The code is basically taken straight from the documentation.
If im using the simple bus and simply just publish, an exchange is created and i can see via the management interface, that messages are being published.
Since the simple bus uses the advanced API to publish i assume that it is a setup issue that i am missing.
I hope someone can bring some insight:-)
/Thomas

I finally tracked down what was causing the problem.
It turns out that setting the parameter: immediate to true will cause the systems to throw exceptions.
the paramters is apparently not supported any more in the RabbitMQ client, see the discussion here: https://github.com/mikehadlow/EasyNetQ/issues/112
So the code below works just fine, mark the change from true to false in the publish method:
public void PublishTest()
{
var advancedBus = RabbitHutch.CreateBus("host=localhost;virtualHost=Test;username=guest;password=guest;").Advanced;
var routingKey = "SimpleMessage";
// declare some objects
var queue = advancedBus.QueueDeclare("Q.TestQueue.SimpleMessage");
var exchange = advancedBus.ExchangeDeclare("E.TestExchange.SimpleMessage", ExchangeType.Direct);
var binding = advancedBus.Bind(exchange, queue, routingKey);
var message = new SimpleMessage() {Test = "HELLO"};
for (int i = 0; i < 100; i++)
{
advancedBus.Publish(exchange, routingKey, true, false, new Message<SimpleMessage>(message));
}
advancedBus.Dispose();
}