Hello I'm Using RabbitMQ in .NET core app and my problem is when I use basic consume it fetches all of the elements in the queue but I just want one of them in advance.
var factory = new ConnectionFactory { HostName = "localhost" };
var connection = factory.CreateConnection();
var channel = connection.CreateModel();
channel.QueueDeclare("inserted_datas", exclusive: false);
var status = channel.MessageCount("inserted_datas");
if (status == 0)
{
return Problem("مقدار قبلی موجود نیست");
}
var consumer = new EventingBasicConsumer(channel);
consumer.Received += async (model, eventArgs) =>
{
var body = eventArgs.Body.ToArray();
var message = Encoding.UTF8.GetString(body);
await Task.CompletedTask;
flag = true;
_state = JsonConvert.DeserializeObject<Current_State>(message);
};
channel.BasicConsume(queue: "inserted_datas", autoAck: false, consumer: consumer);
channel.BasicAck(1, false);
await Task.CompletedTask;
the picture of the initially inserted queue(example has 2 messages)
and the picture after consuming one item
I wanted a leftover message to go to the ready list and just pop one message from the queue
Related
In my program example I have 100 tasks and 5 computers to work on them. My question is about how I can initiate my queue once with these tasks and then keep them permanent (as in a computer finishes a task and instead of a normal acknowledgment after the task is complete where it gets removed, it gets moved back to the queue instead). I want to keep these 100 tasks in my example in a never ending queue that keeps getting worked on by whatever computer is available. I have a prefetch count of 1 meaning a computer can only work on one task at a time.
My current pseudo code is below attempting to do what I'm trying to do but in the end, tasks get removed from the queue and don't seem to be getting added back into the queue all of the time. I started this test last night with the 100 tasks in the queue but by the morning I only had 56 tasks in the queue.
public static async Task Dequeue(CancellationToken token)
{
try
{
using var channel = await GetQueue().ConfigureAwait(false);
if (channel != null)
{
var consumer = new AsyncEventingBasicConsumer(channel);
consumer.Received += async (ch, ea) =>
{
var body = ea.Body.ToArray();
var task = Encoding.UTF8.GetString(body);
var properties = ea.BasicProperties;
// do long running task here
await Task.Delay(600000).ConfigureAwait(false);
// when finished with work then send ack
channel.BasicAck(ea.DeliveryTag, false);
// now add to end of queue when work is complete
channel.BasicPublish(exchange: "",
routingKey: "task_queue",
basicProperties: properties,
body: body);
await Task.Yield();
};
// this consumer tag identifies the subscription
// when it has to be cancelled
string consumerTag = channel.BasicConsume("task_queue", false, consumer);
channel.WaitForConfirmsOrDie(new TimeSpan(0, 0, 10000));
while (!token.IsCancellationRequested)
{
// just wait until cancellation is requested
await Task.Delay(1000, token).ConfigureAwait(false);
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
public static async Task<IModel?> GetQueue(bool isPublisherMode = true)
{
try
{
var factory = new ConnectionFactory()
{
HostName = "localhost",
Port = 5672,
UserName = "guest",
Password = "*******",
DispatchConsumersAsync = true,
ContinuationTimeout = new TimeSpan(2, 0, 0),
HandshakeContinuationTimeout = new TimeSpan(2, 0, 0),
RequestedConnectionTimeout = new TimeSpan(2, 0, 0)
};
var connection = factory.CreateConnection();
var channel = connection.CreateModel();
if (isPublisherMode)
{
channel.ConfirmSelect();
}
var args = new Dictionary<string, object>();
channel.QueueDeclare(queue: "task_queue",
durable: true,
exclusive: false,
autoDelete: false,
arguments: args);
channel.BasicQos(prefetchSize: 0, prefetchCount: 1, global: false);
Console.WriteLine("[*] Waiting for tasks.");
return channel;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return null;
}
I just started learning RabbitMQ yesterday and I don't understand how it can be implemented.
Here is my producer
public class RabbitMQProducerService: IMessageSender
{
public void SendMessage<T>(T message)
{
var factory = new ConnectionFactory { HostName = "localhost" };
var connection = factory.CreateConnection();
var channel = connection.CreateModel();
channel.QueueDeclare("Input", exclusive: false);
var json = JsonConvert.SerializeObject(message);
var body = Encoding.UTF8.GetBytes(json);
channel.BasicPublish(exchange: "", routingKey: "Input", basicProperties: null, body: body);
}
}
And my subscriber
string exepction = "";
StatisticOfTest statistic;
var factory = new ConnectionFactory
{
HostName = "localhost"
};
var connection = factory.CreateConnection();
using var channel = connection.CreateModel();
channel.QueueDeclare("Input", exclusive: false);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, eventArgs) =>
{
SolutionService solution = new SolutionService();
var body = eventArgs.Body.ToArray();
var jsonStr = Encoding.UTF8.GetString(body);
SolutionAddModel solutionAdd = JsonConvert.DeserializeObject<SolutionAddModel>(jsonStr);
if (String.IsNullOrEmpty(solution.Validating(solutionAdd)))
{
statistic = solution.Create(solutionAdd); //I want to return statistic
}
else
{
exepction = solution.Validating(solutionAdd); // Or Exception
}
Console.WriteLine($"Message received: {jsonStr}");
};
channel.BasicConsume(queue: "Input", autoAck: true, consumer: consumer);
Console.ReadKey();
Based on the result, I want to return statistic or exception. But to another queue, for example, to some Output queue
consumer.Received += (model, eventArgs) => { ... }
The ... is the place when you declare what will happen, when a new message is received. The code there would be an event handler for a Received event.
In your case you want each new message to be published to a queue on some condition. Easy to do, just publish a message in your event handler code:
...
// Create a separate channel for sending messages to the output queue
var outputChannel = connection.CreateModel();
// Declare the output queue, you want to send messages to
outputChannel.QueueDeclare("Output", exclusive: false);
// Specify how the actual processing is going to happen
consumer.Received += (model, eventArgs) =>
{
var body = GetMessageBody(eventArgs.Body); // This is where you get new message content based on the received message.
outputChannel.BasicPublish(exchange: "", routingKey: "Input", basicProperties: null, body: body);
};
...
Also, looking at your producer code: you don't need to create a new connection, a new channel, and declare a queue every time you want to send a message. You could do it beforehand just once. It would really save RabbitMQ's resources.
RabbitMQ also has a connection limit, so sometime in the future your program would stop working. It is better if an application actually uses a single connection for everything, if possible, just spawning channels where needed.
Also, don;t forget, that IConnection and IModel are implementing IDisposable, so you should dispose of them properly.
This is how you should do it:
// 'using' tells the compiler to generate a Dispose call for this object at the end of the current block.
using var connection = connectionFactory.CreateConnection();
uisng var channel = connection.CreateModel();
channel.QueueDeclare("Input", exclusive: false);
for (var i = 0; i < 100l i++)
producer.SendMessage(i);
Following is the method to receive the message on RabbitMq Queue. Method works fine but the unit testing this method has problem mentioned below
public void GetMessage(Action<string> action)
{
//create a connection factory
var factory = new ConnectionFactory();
if (_connectionString != null)
{
//assign connection string of rabbitmq
factory.Uri = new Uri(_connectionString);
}
//create connection and channel from connectionfactory
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
//declare the queque on the channel
channel.QueueDeclare(queue: _queque,
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
//create consumer using the channel
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
//When message is recieved on the consumer this will be triggered
var body = ea.Body.ToArray();
_message = Encoding.UTF8.GetString(body);
//call the method passed by the caller method
action(_message);
};
//add the consumer on this channel
channel.BasicConsume(queue: _queque,
autoAck: true,
consumer: consumer);
Console.ReadLine();
}
}
How to write a unit test test above message. Below unit test loading forever inside the call back method and the execution never goes to Assert.Equal line.
[Fact]
public async Task AddMessageAsync_ShouldAddToQueue()
{
//Arrange
string messageToSend = "Test";
string recievedMessage = null;
var queue = GetQueueManager();
queue.Message = messageToSend;
await queue.AddMessageAsync();
//Act
queue.GetMessage((string message) =>
{
//Hit here and recieved the message but does not exist from this method
recievedMessage = message;
});
//Assert
Assert.Equal(messageToSend, recievedMessage);
}
One way to adress the issue would be a synchronization barrier.
Here is an example: (There may be more efficient ways, though.)
... // Your code with some changes:
//create connection and channel from connectionfactory
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
//declare the queque on the channel
channel.QueueDeclare(queue: _queque,
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
// Set up a synchronization barrier.
using (var barrier = new ManualResetEventSlim(false))
{
//create consumer using the channel
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
//When message is recieved on the consumer this will be triggered
var body = ea.Body.ToArray();
_message = Encoding.UTF8.GetString(body);
try // you don't want "injected" code to break yours.
{
//call the method passed by the caller method
action(_message);
}
catch(Exception)
{
// at least log it!
}
finally
{
barrier.Set(); // Signal Event fired.
}
};
//add the consumer on this channel
channel.BasicConsume(queue: _queque,
autoAck: true,
consumer: consumer);
barrier.Wait(); // Wait for Event to fire.
}
}
I would imagine that the RabbitMQ API has had an update to support async/await instead of event-based? If so, then using that would of course be preferable.
If not: You may also want to explore Tasks and the Event-based Asynchronous Pattern (EAP)
I'm working through my first steps with RabbitMQ and have a question on why this didn't work.
The basic tutorial (https://www.rabbitmq.com/tutorials/tutorial-one-dotnet.html) has one executable to send to the broker and another to receive it.
I ran this code through a single console app in Visual Studio and couldn't receive any messages.
If I take the "Receive" code and put it into a separate console app and open that, I get the message (no other code changes).
Can someone explain why I'm not able to have both in the same process? I had figured that the connection factory would handle independent connections accordingly regardless of whether it was the same process or not.
For the sake of completeness (though I doubt it's required), here's the code that didn't work until I pulled out the "Receiver" code and put it into it's own console app:
class Program
{
static void Main(string[] args) {
Receiver.Receive();
Console.WriteLine("receiver set up");
System.Threading.Thread.Sleep(5000);
Console.WriteLine("sending...");
Test.Send();
// can also reverse order of send/receive methods, same result
Console.ReadKey();
}
}
public class Receiver
{
public static void Receive() {
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection()) {
using (var channel = connection.CreateModel()) {
channel.QueueDeclare("hello", false, false, false, null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) => {
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
System.Diagnostics.Debug.WriteLine("=====================");
System.Diagnostics.Debug.WriteLine(message);
System.Diagnostics.Debug.WriteLine("=====================");
};
channel.BasicConsume("hello", true, consumer);
}
}
}
}
public class Test
{
public static void Send() {
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection()) {
using (var channel = connection.CreateModel()) {
channel.QueueDeclare("hello", false, false, false, null);
string message = "Check it!";
var body = Encoding.UTF8.GetBytes(message);
channel.BasicPublish("", "hello", null, body);
}
}
}
}
I think code has a problem. When you send message then Receive connection and channel already dead (Dispose called).
Standard code from RabbitMQ site:
using(var connection = factory.CreateConnection())
using(var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: "hello",
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
};
channel.BasicConsume(queue: "hello",
noAck: true,
consumer: consumer);
Console.WriteLine(" Press [enter] to exit.");
Console.ReadLine();
}
connection and channel are yet alive when client received message.
I am working on a project with RabbitMQ. My code is below.
Producer:
public static void Main()
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("hello", false, false, false, null);
string message = "Hello World!";
var body = Encoding.UTF8.GetBytes(message);
channel.BasicPublish("", "hello", null, body);
Console.WriteLine(" [x] Sent {0}", message);
}
}
}
Consumer with Exchange declared:
public static void Main()
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.ExchangeDeclare("hello", "direct",false, false, false, null);
channel.QueueDeclare("hello", false, false, false, null);
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume("hello", true, consumer);
Console.WriteLine(" [*] Waiting for messages." +
"To exit press CTRL+C");
while (true)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
}
}
}
Consumer without Exchange declared:
public static void Main()
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("hello", false, false, false, null);
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume("hello", true, consumer);
Console.WriteLine(" [*] Waiting for messages." +
"To exit press CTRL+C");
while (true)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
}
}
}
Both consumer code works well, so what's the main use of declaring exchange? I am confused. Can anyone clarify?
Publishing to queues lets you only implement basic publish-subscribe scenarios, where the producer and consumer use the exact queue. In case of multiple consumers a single queue of messages is distributed between multiple consumers.
Publishing to exchanges lets you create complicated scenarios, because of routing between exchanges and queues.
For example, a fanout exchange routes messages to all bound queues. This way, you can have one producer and multiple consumers and each message is copied to all bound queues independently and received independently.
Another example of exchange, a topic exchange routes messages to bound queues based on routing key in a message and a pattern on a queue. This introduces an interesting possibility of tagging messages and delivering them conditionally.
For a complete reference of exchange types and their profiles refer to the documentation:
https://www.rabbitmq.com/tutorials/amqp-concepts.html
https://www.rabbitmq.com/getstarted.html