We Keep Coding

Apache NMS using ActiveMQ: How do I use transactional acknowledge mode but still acknowledging/rolling back a single message every time?

I use Apache NMS (in c#) to receive messages from ActiveMQ.
I want to be able to acknowledge every message I received, or roll back a message in case I had an error.
I solved the first part by using the CreateSession(AcknowledgementMode.IndividualAcknowledge), and then for every received message I use message.Acknowledge().
The problem is that in this mode there is no Rollback option. if the message is not acknowledged - I can never receive it again for another trial. It can only be sent to another consumer, but there isn't another consumer so it is just stucked in queue.
So I tried to use AcknowledgementMode.Transactional instead, but here there is another problem: I can only use session.Commit() or session.Rollback(), but there is no way to know which specific message I commit or role back.
What is the correct way to do this?

Stay with INDIVIDUAL_ACKNOWLEDGE and then try session.recover() and session.close(). Both of those should signal to the broker that the messages are not going to be acknowledged.

My solution to this was to throw an exception if (for any reason (exception from db savechanges event for example)) I did not want to acknowledge the message with message.Acknowledge().
When you throw an exception inside your extended method of IMessageConsumer Listener then the message will be sent again to your consumer for about 5 times (it will then moved to default DLQ queue for investigation).
However you can change this using the RedeliveryPolicy in connection object.
Example of Redelivery
Policy redeliveryPolicy = new RedeliveryPolicy
{
InitialRedeliveryDelay = 5000, // every 5 secs
MaximumRedeliveries = 10, // the message will be redelivered 10 times
UseCollisionAvoidance = true, // use this to random calculate the 5 secs
CollisionAvoidancePercent = 50,// used along with above option
UseExponentialBackOff = false
};
If message fails again (after 10 times) then it will be moved to a default DLQ queue. (this queue will be created automatically)
You can use this queue to investigate the messages that have not been acknowledged using an other consumer.

In discord.net, how do I make all messages automatically deleted every minute?

So I have a verify channel on my discord server which verifies you if you type in //verify , but I want to make it so all messages are automatically deleted every minute on that channel. How do I do that?

One minute delay
Well if you really need that 1 Minute delay you can have a List of Message Id's (ulongs). On your MessageReceived method you can do something like if(message.Channel.Id == YOURCHANNELID) YOURMESSAGELIST.Add(message.Id); On your Client Ready event you would start an async Timer (from System.Threading) with a delay on 1 Minute. So always on your Timer Tick:
var guild = MainClass.DiscordClient.GetGuild(YOURCHANNELID);
var channel = guild.GetTextChannel(YOURCHANNELID);
await channel.DeleteMessagesAsync(YOURMESSAGELIST);
YOURMESSAGELIST.Clear();
Delete message when receiving
What would be way easier and probaply cleaner is deleting every message from that channel as soon as you receive it which mean that you have an if in your MessageReceived method similar to this: if(message.Channel.Id == YOURCHANNELID) await message.DeleteAsync();
I hope that helps you.

How to Determine Azure ServiceBus PrefetchCount and ReceiveBatch Size

I have a queue processor that is retrieving all messages from a ServiceBus Queue. I am wondering how I should determine the MessageReceiver PrefetchCount and the ReceiveBatch messageCount to optimize performance. I am currently setting these to the arbitrary number 500, as seen below:
var receiverFactory = MessagingFactory.CreateFromConnectionString("ConnectionString");
var receiver = await receiverFactory.CreateMessageReceiverAsync("QueueName", ReceiveMode.PeekLock);
receiver.PrefetchCount = 500;
bool loopBatch = true;
while (loopBatch)
{
var tempMessages = await receiver.ReceiveBatchAsync(500, TimeSpan.FromSeconds(1));
// Do some message processing...
loopBatch = tempMessages.Any();
}
When running, I see that my batches often take time to "warm up," retrieving counts such as "1, 1, 1, 1, 1, 1, 1, 1, 125, 125, 125, 125..." where the batch retrieval number suddenly jumps much higher.
From the Prefetching optimization docs:
When using the default lock expiration of 60 seconds, a good value for SubscriptionClient.PrefetchCount is 20 times the maximum processing rates of all receivers of the factory. For example, a factory creates 3 receivers, and each receiver can process up to 10 messages per second. The prefetch count should not exceed 20 X 3 X 10 = 600. By default, QueueClient.PrefetchCount is set to 0, which means that no additional messages are fetched from the service.
I don't really understand how to determine the receiver's "messages per second" when the batch retrieval seems to retrieve widely-varying numbers of messages at a time. Any assistance would be greatly appreciated.

I don't really understand how to determine the receiver's "messages per second" when the batch retrieval seems to retrieve widely-varying numbers of messages at a time.
Prefetch makes more sense in the scenario when OnMessage API is used. In that scenario a callback is registered that takes a single message for processing and you can estimate an average processing time of that message. OnMessage API allows to define how many concurrent callback will be running. It would be extremely innefficient to retrieve messages one by one knowing there is a constant flow of incoming messages. Hence, PrefetchCount is used to specify how many mesasges should be retrieved in a "batch" by clients in the background to save the roundtrips back to the server.

Windows Azure Service Bus Billing

I have a question about the azure service bus billing.
If I have the following code, and a message isn't sent for a long time say 5 hours.
Assume I only have one subscription and the code is as below.
In this scenario over that 5 hour period what do I get charged (is it once for sending and once for downloading, or do I incur charges for the polling keep alive that azure implements in the background)?
enter code here
var subscriptionClient = SubscriptionClient.CreateFromConnectionString(ConnString, topic, subscriptionName);
while (true)
{
var message = subscriptionClient.Receive();
if (message != null)
{
try
{
message.Complete();
}
catch (Exception)
{
// Indicate a problem, unlock message in subscription
message.Abandon();
}
}
else
{
Console.WriteLine("null message received");
}
Thread.Sleep(25);
}

From the code above you will get charged for a single message every time the Receive call returns (even if the result is null). The default timeout for the Receive call is 60 seconds so in the case there is no message for 5 hours, your code will return every one minute and then sleep for 25 seconds so assume that per hour you will get charged for 48 messages (1 min timeout and 25 second wait). You can call the overload of Receive that takes a timeout and pass in 5 hour timeout there. Here the connection will be kept alive for 5 hours before it returns and thus no charges will occur during that time.
From a back of the envelope calculation: A single receiver, running with one minute timeout with no wait and no real messages will get a message charged every minute. That is less than 5cents for the entire month. See billing calculator here

Only Message Transaction will be counted( Send,Receive)... Azure not charging for KeepAlive Messages...
Refer MSDN topic: http://msdn.microsoft.com/en-us/library/hh667438.aspx#BKMK_SBv2FAQ2_1

MSMQ Receive() method timeout

My original question from a while ago is MSMQ Slow Queue Reading, however I have advanced from that and now think I know the problem a bit more clearer.
My code (well actually part of an open source library I am using) looks like this:
queue.Receive(TimeSpan.FromSeconds(10), MessageQueueTransactionType.Automatic);
Which is using the Messaging.MessageQueue.Receive function and queue is a MessageQueue. The problem is as follows.
The above line of code will be called with the specified timeout (10 seconds). The Receive(...) function is a blocking function, and is supposed to block until a message arrives in the queue at which time it will return. If no message is received before the timeout is hit, it will return at the timeout. If a message is in the queue when the function is called, it will return that message immediately.
However, what is happening is the Receive(...) function is being called, seeing that there is no message in the queue, and hence waiting for a new message to come in. When a new message comes in (before the timeout), it isn't detecting this new message and continues waiting. The timeout is eventually hit, at which point the code continues and calls Receive(...) again, where it picks up the message and processes it.
Now, this problem only occurs after a number of days/weeks. I can make it work normally again by deleting & recreating the queue. It happens on different computers, and different queues. So it seems like something is building up, until some point when it breaks the triggering/notification ability that the Receive(...) function uses.
I've checked a lot of different things, and everything seems normal & isn't different from a queue that is working normally. There is plenty of disk space (13gig free) and RAM (about 350MB free out of 1GB from what I can tell). I have checked registry entries which all appear the same as other queues, and the performance monitor doesn't show anything out of the normal. I have also run the TMQ tool and can't see anything noticably wrong from that.
I am using Windows XP on all the machines and they all have service pack 3 installed. I am not sending a large amount of messages to the queues, at most it would be 1 every 2 seconds but generally a lot less frequent than that. The messages are only small too and nowhere near the 4MB limit.
The only thing I have just noticed is the p0000001.mq and r0000067.mq files in C:\WINDOWS\system32\msmq\storage are both 4,096KB however they are that size on other computers also which are not currently experiencing the problem. The problem does not happen to every queue on the computer at once, as I can recreate 1 problem queue on the computer and the other queues still experience the problem.
I am not very experienced with MSMQ so if you post possible things to check can you please explain how to check them or where I can find more details on what you are talking about.
Currently the situation is:
ComputerA - 4 queues normal
ComputerB - 2 queues experiencing problem, 1 queue normal
ComputerC - 2 queues experiencing problem
ComputerD - 1 queue normal
ComputerE - 2 queues normal
So I have a large number of computers/queues to compare and test against.

Any particular reason you aren't using an event handler to listen to the queues? The System.Messaging library allows you to attach a handler to a queue instead of, if I understand what you are doing correctly, looping Receive every 10 seconds. Try something like this:
class MSMQListener
{
public void StartListening(string queuePath)
{
MessageQueue msQueue = new MessageQueue(queuePath);
msQueue.ReceiveCompleted += QueueMessageReceived;
msQueue.BeginReceive();
}
private void QueueMessageReceived(object source, ReceiveCompletedEventArgs args)
{
MessageQueue msQueue = (MessageQueue)source;
//once a message is received, stop receiving
Message msMessage = null;
msMessage = msQueue.EndReceive(args.AsyncResult);
//do something with the message
//begin receiving again
msQueue.BeginReceive();
}
}

We are also using NServiceBus and had a similar problem inside our network.
Basically, MSMQ is using UDP with two-phase commits. After a message is received, it has to be acknowledged. Until it is acknowledged, it cannot be received on the client side as the receive transaction hasn't been finalized.
This was caused by different things in different times for us:
once, this was due to the Distributed Transaction Coordinator unable to communicate between machines as firewall misconfiguration
another time, we were using cloned virtual machines without sysprep which made internal MSMQ ids non-unique and made it receive a message to one machine and ack to another. Eventually, MSMQ figures things out but it takes quite a while.

Try this
public Message Receive( TimeSpan timeout, Cursor cursor )
overloaded function.
To get a cursor for a MessageQueue, call the CreateCursor method for that queue.
A Cursor is used with such methods as Peek(TimeSpan, Cursor, PeekAction) and Receive(TimeSpan, Cursor) when you need to read messages that are not at the front of the queue. This includes reading messages synchronously or asynchronously. Cursors do not need to be used to read only the first message in a queue.
When reading messages within a transaction, Message Queuing does not roll back cursor movement if the transaction is aborted. For example, suppose there is a queue with two messages, A1 and A2. If you remove message A1 while in a transaction, Message Queuing moves the cursor to message A2. However, if the transaction is aborted for any reason, message A1 is inserted back into the queue but the cursor remains pointing at message A2.
To close the cursor, call Close.

If you want to use something completely synchronous and without event you can test this method
public object Receive(string path, int millisecondsTimeout)
{
var mq = new System.Messaging.MessageQueue(path);
var asyncResult = mq.BeginReceive();
var handles = new System.Threading.WaitHandle[] { asyncResult.AsyncWaitHandle };
var index = System.Threading.WaitHandle.WaitAny(handles, millisecondsTimeout);
if (index == 258) // Timeout
{
mq.Close();
return null;
}
var result = mq.EndReceive(asyncResult);
return result;
}