Is it possible to use kafka like api?
It means to send a message and wait to receive the answer.
I have just started programming with Kafka and I have a question about sending messages and receiving processing results.
Producers send events, and you can wait for an acknowledgement from the broker server, yes.
No, you cannot await receiving the consumer in the same action, because that is a separate Kafka function, and consumers poll from topics, rather than listen to the client-local producer events.
Related
I'm using rabbitmq in my microservices project and I saw these two methods
what are these and when do we use each
I suspect you're using something on top of RMQ, like EasyNetQ, because these aren't rabbit terms specifically, but in essence:
Publish publishes messages to a queue
Subscribe subscribes to a queue and defines the code that will act on the received message
A producer is a user application that sends messages while a consumer is a user application that receives messages.A queue is a buffer that stores those messages.
https://dev.to/mashaa/introduction-to-rabbitmq-49n8
In many pub/sub systems, publishers post messages to an intermediary message broker or event bus, and subscribers register subscriptions with that broker, letting the broker perform the filtering. The broker normally performs a store and forward function to route messages from publishers to subscribers. In addition, the broker may prioritize messages in a queue before routing.
https://en.m.wikipedia.org/wiki/Publish–subscribe_pattern
The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue. Actually, quite often the producer doesn’t even know if a message will be delivered to any queue at all.
https://www.rabbitmq.com/tutorials/tutorial-three-python.html
My requirement is not to consume the messages by the consumer if the condition not met, post the consumed message back to rabbitmq and stop the Masstransit bus (I believe consumer can't be stopped). I can post the consumed message back to rabbitmq and stop the bus but the problem is, as soon as I post the message back, my consumer consumes the same message again before stopping the bus and post the same message back again and it repeats until bus is stopped. In result, I'm getting duplicate messages on the queue.
if(!valid)
{
await ep.Send(message);
bus.StopAsync();
}
I think enabling rabbitmq_message_deduplication plugin may help in this situation but just wonder if there is any other ways to achieve.
I am using RabbitMQ to send data from a server to several consumers. I have to decide when a message will be deleted from the queue so I want to implement that the consumers send a message to the server. If all the consumers send a response then I will remove the message.
I already found an RPC call but I think that is to complex for the problem I have. Is there a faster way to send a quick response to the server? I am using C#.
UPDATE: I think I already found a solution. I will notify the broker so it can be removed from the queue.
You can use prefetch to send multiple messages. The prefetch value is used to specify how many messages that are being sent to the consumer at the same time.
The client can send an ack back to the server, which will delete the message from the broker.
I think this is a fairly common problem.
I'm using C# and Rabbit MQ client. I have a publisher sending a message to an exchange. The publisher queue and subscriber queues are all bound to the exchange via a routing key. When the publisher sends a message to the Exchange, all the subscriber queues receive the message as expected, however the publisher queue also receives the message.
is there a good pattern that can be employed to prevent this?
When you send a message, you give the message an Id. Normally a GUID. You could track which messages you send, and if a message you receive is one you know you have sent you could discard it.
However, as Derick said, the best bet is just not to do this.
I have a winforms client application that sends messages to an asp.net web service, I need to be able to queue these messages on the client and then then send them in order, waiting for a response from the webservice before sending the next message.
I did look at some examples of queueing using WCF but they seemed to have the queue on the server and not the client.
Any advice abotu what technology to use and on how to implement a solution would be very much appreciated.
Why wait for the response of the server before sending the next message? there is no good reason to do that. Just mark the messages with a sequence number and process them in order at the server.
MSMQ has a queue both on the client and the server and moves the message when a connection is available.
There's also good ol' MSMQ, but that queues things on the server as well.
You could use middleware for the queue (MSMQ etc).
An alternative would be a thread-safe producer/consumer queue at the client. Your "main" code just adds to the queue (ConcurrentQueue in 4.0 might work nicely here, although even in 4.0 I tend to use a utility queue I wrote a while ago instead); and you have a dedicated worker thread that dequeues messages, does the WCF work, and processes the response.
If you need reliable delivery, why not use AMQP with a message broker like RabbitMQ?