I created a small demo to show the RabbitMQ basics. Unfortunatly it doesn't work as expected an has two issues. I am using .NET Core 3.1 and RabbitMQ.Client 6.2.2
I created the Employee class which receives messages from the task queue. The first employee is working nice but if I start more employees they don't work (don't receive messages). And I can't figure out why that would be.
And if I have a lot of messages in the queue (before starting the second employee) I see that all messages in the tasks queue get ACKed when the second starts and then after a short time they become UNACKed again. Somehow weird.
But mainly: why do the other employees not work?
using RabbitMQ.Client;
using RabbitMQ.Client.Events;
using System;
using System.Text;
using System.Threading;
namespace DemoTasks.Employee
{
class Employee
{
static void Main(string[] args)
{
string clientName = "Employee-" + Guid.NewGuid().ToString();
Console.Title = clientName;
Console.WriteLine("Moin moin");
IConnectionFactory connectionFactory = new ConnectionFactory
{
HostName = "localhost",
Port = 5672,
VirtualHost = "/",
UserName = "user",
Password = "password",
ClientProvidedName = clientName
};
using (IConnection connection = connectionFactory.CreateConnection(clientName))
{
using (IModel model = connection.CreateModel())
{
model.ExchangeDeclare("jobs", "fanout", false, false, null);
model.QueueDeclare("tasks", true, false, false);
model.QueueBind("tasks", "jobs", "", null);
EventingBasicConsumer consumer = new EventingBasicConsumer(model);
consumer.Received += OnWorkReceived;
model.BasicConsume("tasks", false, clientName + ":OnWorkReceived", consumer);
Console.ReadLine();
model.Close();
}
connection.Close();
}
Console.WriteLine("Wochenende ... woooh !!!");
}
private static void OnWorkReceived(object sender, BasicDeliverEventArgs e)
{
EventingBasicConsumer consumer = (EventingBasicConsumer)sender;
IModel model = consumer.Model;
string task = Encoding.UTF8.GetString(e.Body.ToArray());
Console.Write("working on: " + task + " ... ");
Thread.Sleep(5000);
Console.WriteLine("done!");
model.BasicAck(e.DeliveryTag, false);
}
}
}
I think your problem is about setting PrefetchCount on your channel. It's about how many messages that one consumer can get from rabbit and cache them on itself to process them.
If don't set it, one consumer can consume all messages on queue and no time to get messages by other consumers, so you can set it by using channel.basicQos(1) or basicqos(0,1,false). By this setting every consumer can get one message after send ack to rabbit then can get another one.
When set prefetch count to lower number, can affect on performance because your consumer must ask rabbit more to get messages .
For detail information see this: https://www.rabbitmq.com/consumer-prefetch.html
Related
I am connecting to Apache Active MQ which is hosted on AWS to integrate my app to a custom service. I need to keep this running always, not one time like it's right now. The code below works, but only for one message, I need to maintain the connection active all the time listening in order to receive all the messages.
Here is the code.
using Apache.NMS;
using Apache.NMS.Util;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace ApacheMQAsync
{
class Program
{
protected static ITextMessage message = null;
public static void Main(string[] args)
{
Uri connecturi = new Uri("URL:61617");
Console.WriteLine("About to connect to " + connecturi);
// NOTE: ensure the nmsprovider-activemq.config file exists in the executable folder.
IConnectionFactory factory = new Apache.NMS.ActiveMQ.ConnectionFactory(connecturi);
IConnection connection = factory.CreateConnection("username", "password");
ISession session = connection.CreateSession();
IDestination destination = SessionUtil.GetDestination(session, "queue://FOO.BAR");
Console.WriteLine("Using destination: " + destination);
// Create a consumer and producer
IMessageConsumer consumer = session.CreateConsumer(destination);
consumer.Listener += new MessageListener(OnMessage);
connection.Start();
// Wait for the message
if (message == null)
{
Console.WriteLine("No message received!");
}
else
{
Console.WriteLine("Received message with ID: " + message.NMSMessageId);
Console.WriteLine("Received message with text: " + message.Text);
}
}
protected static void OnMessage(IMessage receivedMsg)
{
message = receivedMsg as ITextMessage;
message.Acknowledge();
}
}
}
On the console it displays following
No message received!
and after few seconds the console exist?
There's no real magic there, you need to do something to keep your application running such as pausing on console input or looping on a sleep or other wait type call and then checking something to see if your application should continue. The JMS client isn't guaranteed to keep your application open and running and you should never rely on it to.
I'm trying to handle request authorization in a microservice based architecture using a message queue (RabbitMQ).
I've got a receiver and sender configured fine as a console application in .NET Core per these instructions. However, when using this in a real world example, my application receiving project isn't collecting messages as a consumer.
I'm assuming I have to register the consumer in the Startup.cs, but I can't seem to get this working.
My consumer/responder code:
public class RabbitMqHandler
{
private readonly IJWTFactory _jwtFactory;
public RabbitMqHandler(IJWTFactory jWTFactory)
{
_jwtFactory = jWTFactory;
}
public void Register()
{
var mqFactory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = mqFactory.CreateConnection())
{
Console.WriteLine("Listening on Rabbit MQ");
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: "Authorize", durable: false, exclusive: false, autoDelete: false, arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var jwtToken = Encoding.UTF8.GetString(body);
Console.WriteLine("Rceived Message");
var validatedToken = _jwtFactory.ValidateTokenSignature(jwtToken);
SendResponse(validatedToken);
};
channel.BasicConsume(queue: "Authorize", autoAck: true, consumer: consumer);
}
}
}
public void Deregister()
{
}
Startup.cs to register the
.AddSingleton()
Edit: I've added some additional listening code, this is definitely running on startup, but RabbitMQ is not showing the app as a consumer or a channel:
public static class ApplicationBuilderExtentions
{
public static RabbitMqHandler Listener { get; set; }
public static IApplicationBuilder UseRabbitListener(this IApplicationBuilder app)
{
Listener = app.ApplicationServices.GetService<RabbitMqHandler>();
var life = app.ApplicationServices.GetService<IApplicationLifetime>();
life.ApplicationStarted.Register(OnStarted);
//press Ctrl+C to reproduce if your app runs in Kestrel as a console app
life.ApplicationStopping.Register(OnStopping);
return app;
}
private static void OnStarted()
{
Listener.Register();
}
private static void OnStopping()
{
Listener.Deregister();
}
}
To summarise:
How do I correctly configure a consumer in .NET Core to consume messages?
Is this just the wrong to expect a Message Queue to manage request/response style communication?
Should I just be using an API call to authenticate and authorize users?
Answer as provided by #Evk (in the comments):
"using is designed to dispose things, it calls Dispose when you reach the end of the using block. BasicConsume is not a blocking call, so it starts consumption and returns immediately.
Right after that the end of the using blocks is reached for both channel and connection, disposing them (and disposing them is the same as closing)."
I'd like to add the following:
You can quickly try if removing the using brings the desired result very easily. Change the following code line:
using (var connection = mqFactory.CreateConnection())
to:
var connection = mqFactory.CreateConnection();
This will instantly do the trick. But be aware it also removes proper disposal - so you need to add that - here is an article from Microsoft describing how IDisposable is to be implemented correctly.
I have a scenerio,where a executable is the producer and WCF service is the consumer.
WCF service WorkFlow is as follows:
1)Service invokes the executable (producer), this executable is another process which produces the messages into RabbitMQ Queue.
2)Service has to consume the messages from the RabbitMQ Queue
3)Returns the data to client.
using RabbitMQ.Client;
using RabbitMQ.Client.Events;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.Text;
namespace ConnectionServices
{
public class Connection : IConnection
{
public string ConnectSite(string provider, string server, string siteName)
{
InvokeProducer(provider, server, siteName);
string activeInstance = RunRabbitMQ();
return activeInstance;
}
public void InvokeProducer(string provider, string server, string siteName)
{
string siteManagerExePath = #"C:\Users\mbmercha\Documents\Visual Studio 2015\Projects\Producer\Producer\bin\Debug\Producer.exe";
try
{
ProcessStartInfo startInfo = new ProcessStartInfo();
Process siteManagerProcess = new Process();
startInfo.FileName = siteManagerExePath;
startInfo.Arguments = string.Format("{0} {1} {2} {3}", "-b ", provider, server, siteName);
siteManagerProcess.StartInfo = startInfo;
siteManagerProcess.Start();
siteManagerProcess.WaitForExit();
}
catch (Exception e)
{
}
}
public string RunRabbitMQ()
{
var factory = new ConnectionFactory() { HostName = "localhost" };
string activeInstance = null;
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("DurableQueue", true, false, false, null);
channel.ExchangeDeclare("DurableExchange", ExchangeType.Topic, true);
channel.QueueBind("DurableQueue", "DurableExchange", "durable");
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
activeInstance = message;
};
channel.BasicConsume(queue: "DurableQueue",
autoAck: false,
consumer: consumer);
}
return activeInstance;
}
}
}
So far service is able to invoke executable and messages are produced in the queue.
But service fails from step 2, it is returning null instead of actual message.
Can anybody suggest me what I am missing here?
Thanks in Advance.
You're never setting activeInstance to anything except null.
You appear to be using the asynchronous API, which means that you're retrieving the message from RabbitMQ long after the RunRabbitMQ method call has completed... or you would be if you didn't immediately dispose of all the consumer machinery when returning.
If you want to retrieve messages synchronously - in this case, within a synchronous method call - you'll need to wait for a message to become available. For this, you'd want to use the 'pull API', which is channel.BasicGet(...).
I am trying to write an integration / acceptance test to test some code in azure, the code in the question ATM simply subscribes to one topic and publishes to another.
I have written the test but it doesn't always pass, seems as though there could be a race condition in place. I've tried writing it a couple of ways including using OnMessage and also using Receive (example I show here).
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
My query concerning my example specifically, I assumed that once I created the subscription to the target topic, that any message sent to it I would be able to pickup using Receive(), whatever point in time that message arrived meaning, if the message arrives at the target topic before I call Receive(), I would still be able to read the message afterward by calling Receive(). Could anyone please shed any light on this?
namespace somenamespace {
[TestClass]
public class SampleTopicTest
{
private static TopicClient topicClient;
private static SubscriptionClient subClientKoEligible;
private static SubscriptionClient subClientKoIneligible;
private static OnMessageOptions options;
public const string TEST_MESSAGE_SUB = "TestMessageSub";
private static NamespaceManager namespaceManager;
private static string topicFleKoEligible;
private static string topicFleKoIneligible;
private BrokeredMessage message;
[ClassInitialize]
public static void BeforeClass(TestContext testContext)
{
//client for publishing messages
string connectionString = ConfigurationManager.AppSettings["ServiceBusConnectionString"];
string topicDataReady = ConfigurationManager.AppSettings["DataReadyTopicName"];
topicClient = TopicClient.CreateFromConnectionString(connectionString, topicDataReady);
topicFleKoEligible = ConfigurationManager.AppSettings["KnockOutEligibleTopicName"];
topicFleKoIneligible = ConfigurationManager.AppSettings["KnockOutIneligibleTopicName"];
//create test subscription to receive messages
namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.SubscriptionExists(topicFleKoEligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoEligible, TEST_MESSAGE_SUB);
}
if (!namespaceManager.SubscriptionExists(topicFleKoIneligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoIneligible, TEST_MESSAGE_SUB);
}
//subscriber client koeligible
subClientKoEligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoEligible, TEST_MESSAGE_SUB);
subClientKoIneligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoIneligible, TEST_MESSAGE_SUB);
options = new OnMessageOptions()
{
AutoComplete = false,
AutoRenewTimeout = TimeSpan.FromMinutes(1),
};
}
[TestMethod]
public void E2EPOCTopicTestLT50()
{
Random rnd = new Random();
string customerId = rnd.Next(1, 49).ToString();
FurtherLendingCustomer sentCustomer = new FurtherLendingCustomer { CustomerId = customerId };
BrokeredMessage sentMessage = new BrokeredMessage(sentCustomer.ToJson());
sentMessage.CorrelationId = Guid.NewGuid().ToString();
string messageId = sentMessage.MessageId;
topicClient.Send(sentMessage);
Boolean messageRead = false;
//wait for message to arrive on the ko eligible queue
while((message = subClientKoEligible.Receive(TimeSpan.FromMinutes(2))) != null){
//read message
string messageString = message.GetBody<String>();
//Serialize
FurtherLendingCustomer receivedCustomer = JsonConvert.DeserializeObject<FurtherLendingCustomer>(messageString.Substring(messageString.IndexOf("{")));
//assertion
Assert.AreEqual(sentCustomer.CustomerId, receivedCustomer.CustomerId,"verify customer id");
//pop message
message.Complete();
messageRead = true;
//leave loop after processing one message
break;
}
if (!messageRead)
Assert.Fail("Didn't receive any message after 2 mins");
}
}
}
As the official document states about SubscriptionClient.Receive(TimeSpan):
Parameters
serverWaitTime
TimeSpan
The time span the server waits for receiving a message before it times out.
A Null can be return by this API if operation exceeded the timeout specified, or the operations succeeded but there are no more messages to be received.
Per my test, if a message sent to the topic and then delivered to your subscription within your specific serverWaitTime, then you could receive a message no matter whether the message arrives at the target topic before or after you call Receive.
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
[TestMethod]
public void ReceiveMessages()
{
subClient.OnMessage(msg => {
System.Diagnostics.Trace.TraceInformation($"{DateTime.Now}：{msg.GetBody<string>()}");
msg.Complete();
});
Task.Delay(TimeSpan.FromMinutes(5)).Wait();
}
For SubscriptionClient.OnMessage, I assumed that it basically a loop invoking Receive. After calling OnMessage, you need to wait for a while and stop this method to exit. Here is a blog about the Event-Driven message programming for windows Azure Service Bus, you could refer to here.
Additionally, I found that your topicClient for sending messages and the subClientKoEligible for receiving a message are not targeted at the same topic path.
We are using RabbitMQ for queuing messages in C# .Net (EasyNetQ Client).
i want one consumer app (C# Console App) listen to one queue and provide multiple handlers for each message type.
I implemented this scenario and my code is here :
using (var advancedBus = RabbitHutch.CreateBus("host=localhost;prefetchcount=100")
.Advanced)
{
var queue = advancedBus.QueueDeclare("MyQueue");
advancedBus.Consume(queue, x => x
.Add<MessageType1>((message, info) =>
{
Console.WriteLine("MessageType1 Body : " + message.Body.Body);
})
.Add<MessageType2>((message, info) =>
{
Console.WriteLine(" MessageType2 Body: " + message.Body.Body);
}).ThrowOnNoMatchingHandler = false);
}
My Problem :
But when i execute this consumer it does nothing. do not any thing happen.
i publish messages to that queue like this :
using (var advancedBus = RabbitHutch.CreateBus("host=localhost").Advanced)
{
var queue = advancedBus.QueueDeclare("MyQueue");
if (advancedBus.IsConnected)
advancedBus.Publish(Exchange.GetDefault(), queue.Name, false, false,
new Message<MessageType1>(change));
else
result = false;
}
What is the problem.
Ok, after testing this code, these are the problems:
First of all, you're disposing your advancedBus right after you register for consumption. You need to remember that when you invoke IAdvanceBus.Consume, you're only registering a callback for each message. If you dispose the bus immediately after registration, your delegate can't be invoked since the connection was already closed. So, you'll to remove the using statement around the rabbit declaration (don't forget to dispose it when you're done):
var advancedBus = RabbitHutch.CreateBus("host=localhost;prefetchcount=100").Advanced
Second, the immediate flag has been deprecated and shouldn't be used, the message doesn't seem to be getting to the queue. Change Publish to:
advancedBus.Publish(Exchange.GetDefault(), queue.Name, true, false,
new Message<MessageType1>(change));
Also, if you're running this from a console application, don't forget to use Console.ReadKey so your main thread doesn't terminate.
Here's a working code sample:
static void Main()
{
var change = new MessageType1();
var advancedBus = RabbitHutch.CreateBus("host=localhost").Advanced;
ConsumeMessage(advancedBus);
var queue = advancedBus.QueueDeclare("MyQueue");
if (advancedBus.IsConnected)
{
advancedBus.Publish(Exchange.GetDefault(), queue.Name, true, false,
new Message<MessageType1>(change));
}
else
{
Console.WriteLine("Can't connect");
}
Console.ReadKey();
}
private static void ConsumeMessage(IAdvancedBus advancedBus)
{
var queue = advancedBus.QueueDeclare("MyQueue");
advancedBus.Consume(queue, registration =>
{
registration.Add<MessageType1>((message, info) =>
{
Console.WriteLine("Body: {0}", message.Body);
});
});
}