I have a message queue where i add some emails.
I want to extract all emails, examine the date when they were added and their priority and send only one of them.
I read messages like this:
private IList<Email> GetEmailsFromQueue(MessageQueue queue)
{
queue.Formatter = new XmlMessageFormatter(new Type[] { typeof(Email) });
IList<Email> emails = new List<Email>();
Cursor cursor = queue.CreateCursor();
Message m = PeekWithoutTimeout(queue, cursor, PeekAction.Current);
if (m != null)
{
emails.Add((Email)m.Body);
while ((m = PeekWithoutTimeout(queue, cursor, PeekAction.Next)) != null)
{
emails.Add((Email)m.Body);
}
}
return emails;
}
private Message PeekWithoutTimeout(MessageQueue q, Cursor cursor, PeekAction action)
{
Message msgFromQueue = null;
try
{
msgFromQueue = q.Peek(new TimeSpan(1), cursor, action);
}
catch(MessageQueueException ex)
{
if (!ex.Message.ToLower().Contains("timeout"))
{
throw;
}
}
return msgFromQueue;
}
Receive method will remove the message from the queue.
Is there any way to read and remove only some messages?
LE: One solution i mighty think of is to add an id to each message, and use ReceiveById
Any other tips?
Is there any way to read and remove only some messages?
I've never used MSMQ (well not in anger anyways) but I think you should be able to combine both the Peek and ReceiveById methods.
So you would continue to Peek at the queue to see what messages are available and then once you have decided to remove a message make use of the ReceiveById, to process/remove from the queue.
Aside from this perhaps the other option would be to make use of 2 queues. The first would be for all inbound messages and the second would be used to add back messages you want to keep.
So going back to your example (assuming I understand what you're trying to achieve)
the first would be for inbound emails which you process as described extract all emails, examine the date when they were added and their priority
the second would be for your outbound emails i.e. once you have the email you want to send, push it on to the outbound queue
Related
I am making a program that starts a child process and communicates via anonymous pipes. When I read from a pipe, the program hangs at the first ReadLine() call as seen in the following code method:
// Reads messages sent from module's process via anonymous pipe
internal string[] ReadPipe() {
try {
Log.Verbose("Checking read pipe");
// Check for sync message and return null if there is no message to receive
string pipeMessage = _pipeInReader.ReadLine(); // HANGS ON THIS LINE
if(pipeMessage == null || !pipeMessage.StartsWith("SYNC")) {
Log.Verbose("No message found in pipe");
return null;
}
// Return array of message lines
Log.Verbose("Received message from module {ModuleName}", _fileInfo.Name);
List<string> pipeMessageLines = new();
do {
pipeMessage = _pipeInReader.ReadLine();
pipeMessageLines.Add(pipeMessage);
Log.Debug(pipeMessage);
} while(pipeMessage != null && !pipeMessage.StartsWith("END"));
return pipeMessageLines.ToArray();
} catch(Exception e) {
Log.Error(e.ToString());
return null;
}
}
The code I am using to write to the pipe is the following:
// Sends a message to module's process via anonymous pipe
public static void WritePipe(string message) {
try {
Log.Verbose("Sending \"{Message}\" to kit pipe", message);
// Send sync message and wait for module process to receive it
Log.Verbose("Waiting for pipe drain");
_pipeOutWriter.Write("SYNC");
_pipeOut.WaitForPipeDrain();
// Send the specified message
Log.Verbose("Pipe drained. Sending message");
_pipeOutWriter.Write(message);
_pipeOutWriter.Write("END");
} catch(Exception e) {
Log.Error(e.ToString());
}
}
Why does it hang at that ReadLine() line?
Thanks in advance!
Without a proper minimal, reproducible example, it's impossible to say for sure. However, one glaring problem with your code is that when you write to the _pipeOutWriter object, you don't call Flush(). Assuming that's a TextWriter, by default it's going to buffer the data until the internal buffer is full, and not send anything to the underlying Stream until then.
By calling Flush(), you force it to flush its internal buffer and send the data right away.
If that does not address your question, please edit the question to improve it, making sure to provide a minimal, reproducible example, and any additional details about what you've tried so far to fix the problem and what specifically you need help with.
I have an MSMQ which would be used almost as some form of "Schedule queue" for sending out tweets at specific time (Think of it as setting a time to send a specific tweet out).
I have sort of ran into a brick wall in how I would retrieve items from the queue. I am aware of the Peek() and Remove() methods, and they work, but they wouldn't remove the item which is next scheduled to be processed, they would simply only retrieve the the oldest item on the queue.
I have also had a look at using the TimeToBeReceived property, but it doesn't seem to make much difference.
If I was to add an message to the queue, with the TimeToBeReceived property of the next hour, I shouldn't expect to be able to retrieve it for the next hour, it the RetrieveNextItem() method I have written should either return nothing at all or the next item where the time has passed. Could anyone explain to me how I would accomplish this?
Please see my code below to get an general idea on what I already have.
public void AddItemToQueue(IScheduledTweet scheduledTweet)
{
MessageQueue messageQueue = new MessageQueue(#".\Private$\TwitterBot");
Message message = new Message();
message.Body = scheduledTweet;
message.TimeToBeReceived = scheduledTweet.Time.TimeOfDay;
messageQueue.Send(scheduledTweet);
}
public IScheduledTweet RetrieveNextItem()
{
IScheduledTweet returnValue;
MessageQueue messageQueue;
messageQueue = new MessageQueue(#".\Private$\TwitterBot");
messageQueue.MessageReadPropertyFilter.TimeToBeReceived = true;
messageQueue.Formatter = new XmlMessageFormatter(new Type[] { typeof(ScheduledTweet) });
// I expect the message variable to be a message where its TimeToBeReceived property is either equal to the current time, or past the time.
Message message = messageQueue.Peek();
returnValue = (ScheduledTweet)message.Body;
return returnValue;
}
I'm looking for a solid way of re-queuing messages that couldn't be handled properly - at this time.
I've been looking at http://dotnetcodr.com/2014/06/16/rabbitmq-in-net-c-basic-error-handling-in-receiver/ and it seems that it's supported to requeue messages in the RabbitMQ API.
else //reject the message but push back to queue for later re-try
{
Console.WriteLine("Rejecting message and putting it back to the queue: {0}", message);
model.BasicReject(deliveryArguments.DeliveryTag, true);
}
However I'm using EasyNetQ.
So wondering how I would do something similar here.
bus.Subscribe<MyMessage>("my_subscription_id", msg => {
try
{
// do work... could be long running
}
catch ()
{
// something went wrong - requeue message
}
});
Is this even a good approach? Not ACK the message could cause problems if do work exceeds the wait for ACK timeout by the RabbitMQ server.
So I came up with this solution. Which replaces the default error strategy by EasyNetQ.
public class DeadLetterStrategy : DefaultConsumerErrorStrategy
{
public DeadLetterStrategy(IConnectionFactory connectionFactory, ISerializer serializer, IEasyNetQLogger logger, IConventions conventions, ITypeNameSerializer typeNameSerializer)
: base(connectionFactory, serializer, logger, conventions, typeNameSerializer)
{
}
public override AckStrategy HandleConsumerError(ConsumerExecutionContext context, Exception exception)
{
object deathHeaderObject;
if (!context.Properties.Headers.TryGetValue("x-death", out deathHeaderObject))
return AckStrategies.NackWithoutRequeue;
var deathHeaders = deathHeaderObject as IList;
if (deathHeaders == null)
return AckStrategies.NackWithoutRequeue;
var retries = 0;
foreach (IDictionary header in deathHeaders)
{
var count = int.Parse(header["count"].ToString());
retries += count;
}
if (retries < 3)
return AckStrategies.NackWithoutRequeue;
return base.HandleConsumerError(context, exception);
}
}
You replace it like this:
RabbitHutch.CreateBus("host=localhost", serviceRegister => serviceRegister.Register<IConsumerErrorStrategy, DeadLetterStrategy>())
You have to use the AdvancedBus so you have to setup everything up manually.
using (var bus = RabbitHutch.CreateBus("host=localhost", serviceRegister => serviceRegister.Register<IConsumerErrorStrategy, DeadLetterStrategy>()))
{
var deadExchange = bus.Advanced.ExchangeDeclare("exchange.text.dead", ExchangeType.Direct);
var textExchange = bus.Advanced.ExchangeDeclare("exchange.text", ExchangeType.Direct);
var queue = bus.Advanced.QueueDeclare("queue.text", deadLetterExchange: deadExchange.Name);
bus.Advanced.Bind(deadExchange, queue, "");
bus.Advanced.Bind(textExchange, queue, "");
bus.Advanced.Consume<TextMessage>(queue, (message, info) => HandleTextMessage(message, info));
}
This will dead letter a failed message 3 times. After that it'll go to the default error queue provided by EasyNetQ for error handling. You can subscribe to that queue.
A message is dead lettered when an exception propagates out of your consumer method. So this would trigger a dead letter.
static void HandleTextMessage(IMessage<TextMessage> textMessage, MessageReceivedInfo info)
{
throw new Exception("This is a test!");
}
to the best of my knowledge, there is no way to manually ack, nack or reject a message with EasyNetQ.
I see you have opened an issue ticket with the EasyNetQ team, regarding this... but no answer, yet.
FWIW, this is a very appropriate thing to do. All of the libraries that I use support this feature set (in NodeJS) and it is common. I'm surprised EasyNetQ doesn't support this.
I am trying to receive all messages for a given subscription to a Service Bus Topic, but for the context of this app I do not want them dead lettered at this time, I just want to view them and leave them on the subscription. Despite instantiating the Client as
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
and making sure that I am using message.Abandon() rather than message.Complete() the message always gets Dead-lettered after accessing the message. I also have options.AutoComplete set to false
full method code below:
public List<ServiceBusMessage> RetrieveSubscriptionMessages(Subscription sub) {
ServiceBusMessage sbm;
List<ServiceBusMessage> list = new List<ServiceBusMessage>();
String connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"].ToString();
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
sc.OnMessage((message) => {
try {
sbm = new ServiceBusMessage() {
topicName = sub.topicName,
messageText = message.GetBody<String>()
};
list.Add(sbm);
message.Abandon();
}
catch (Exception) {
message.Abandon();
throw;
}
}, options);
return list;
}
Am I missing something ? Or is there an issue with auto dead-lettering with the onMessage() method?
Thanks !
When a message is abandoned the service bus will immediately make it available for re-delivery to any subscriber of the topic.
If you are trying to configure a multicast mechanism in which multiple listeners all receive the same message, then understand that all listeners on a given subscription will be competing for the same message. In order for every listener to receive its own copy of the message, then simply create a unique subscription to the topic for each listener.
If your intent is to delay re-delivery of the abandoned message, you might look at the SO question: What's the proper way to abandon an Azure SB Message so that it becomes visible again in the future in a way I can control?
As part of an effort to automate starting/stopping some of our NServiceBus services, I'd like to know when a service has finished processing all the messages in it's input queue.
The problem is that, while the NServiceBus service is running, my C# code is reporting one less message than is actually there. So it thinks that the queue is empty when there is still one message left. If the service is stopped, it reports the "correct" number of messages. This is confusing because, when I inspect the queues myself using the Private Queues view in the Computer Management application, it displays the "correct" number.
I'm using a variant of the following C# code to find the message count:
var queue = new MessageQueue(path);
return queue.GetAllMessages().Length;
I know this will perform horribly when there are many messages. The queues I'm inspecting should only ever have a handful of messages at a time.
I have looked at
other
related
questions,
but haven't found the help I need.
Any insight or suggestions would be appreciated!
Update: I should have mentioned that this service is behind a Distributor, which is shut down before trying to shut down this service. So I have confidence that new messages will not be added to the service's input queue.
The thing is that it's not actually "one less message", but rather dependent on the number of messages currently being processed by the endpoint which, in a multi-threaded process, can be as high as the number of threads.
There's also the issue of client processes that continue to send messages to that same queue.
Probably the only "sure" way of handling this is by counting the messages multiple times with a delay in between and if the number stay zero over a certain number of attempts that you can assume the queue is empty.
WMI was the answer! Here's a first pass at the code. It could doubtless be improved.
public int GetMessageCount(string queuePath)
{
const string query = "select * from Win32_PerfRawData_MSMQ_MSMQQueue";
var query = new WqlObjectQuery(query);
var searcher = new ManagementObjectSearcher(query);
var queues = searcher.Get();
foreach (ManagementObject queue in queues)
{
var name = queue["Name"].ToString();
if (AreTheSameQueue(queuePath, name))
{
// Depending on the machine (32/64-bit), this value is a different type.
// Casting directly to UInt64 or UInt32 only works on the relative CPU architecture.
// To work around this run-time unknown, convert to string and then parse to int.
var countAsString = queue["MessagesInQueue"].ToString();
var messageCount = int.Parse(countAsString);
return messageCount;
}
}
return 0;
}
private static bool AreTheSameQueue(string path1, string path2)
{
// Tests whether two queue paths are equivalent, accounting for differences
// in case and length (if one path was truncated, for example by WMI).
string sanitizedPath1 = Sanitize(path1);
string sanitizedPath2 = Sanitize(path2);
if (sanitizedPath1.Length > sanitizedPath2.Length)
{
return sanitizedPath1.StartsWith(sanitizedPath2);
}
if (sanitizedPath1.Length < sanitizedPath2.Length)
{
return sanitizedPath2.StartsWith(sanitizedPath1);
}
return sanitizedPath1 == sanitizedPath2;
}
private static string Sanitize(string queueName)
{
var machineName = Environment.MachineName.ToLowerInvariant();
return queueName.ToLowerInvariant().Replace(machineName, ".");
}