We Keep Coding

Apache NMS using ActiveMQ: How do I use transactional acknowledge mode but still acknowledging/rolling back a single message every time?

I use Apache NMS (in c#) to receive messages from ActiveMQ.
I want to be able to acknowledge every message I received, or roll back a message in case I had an error.
I solved the first part by using the CreateSession(AcknowledgementMode.IndividualAcknowledge), and then for every received message I use message.Acknowledge().
The problem is that in this mode there is no Rollback option. if the message is not acknowledged - I can never receive it again for another trial. It can only be sent to another consumer, but there isn't another consumer so it is just stucked in queue.
So I tried to use AcknowledgementMode.Transactional instead, but here there is another problem: I can only use session.Commit() or session.Rollback(), but there is no way to know which specific message I commit or role back.
What is the correct way to do this?

Stay with INDIVIDUAL_ACKNOWLEDGE and then try session.recover() and session.close(). Both of those should signal to the broker that the messages are not going to be acknowledged.

My solution to this was to throw an exception if (for any reason (exception from db savechanges event for example)) I did not want to acknowledge the message with message.Acknowledge().
When you throw an exception inside your extended method of IMessageConsumer Listener then the message will be sent again to your consumer for about 5 times (it will then moved to default DLQ queue for investigation).
However you can change this using the RedeliveryPolicy in connection object.
Example of Redelivery
Policy redeliveryPolicy = new RedeliveryPolicy
{
InitialRedeliveryDelay = 5000, // every 5 secs
MaximumRedeliveries = 10, // the message will be redelivered 10 times
UseCollisionAvoidance = true, // use this to random calculate the 5 secs
CollisionAvoidancePercent = 50,// used along with above option
UseExponentialBackOff = false
};
If message fails again (after 10 times) then it will be moved to a default DLQ queue. (this queue will be created automatically)
You can use this queue to investigate the messages that have not been acknowledged using an other consumer.

ActiveMQ - Do I need to re-subscribe to a queue after the Listener event fires?

I am integrating with an ActiveMQ JMS system using the Apache.NMS library. For the response queue listener, it's not clear whether the consumer is disposed after a message is received.
Here are excerpts from the solution:
var destination = getDestination(session, Settings.Instance.OutboundQueue);
// Create a consumer and producer
using (var consumer = session.CreateConsumer(destination))
{
// Start the connection so that messages will be processed.
connection.Start();
consumer.Listener += OnMessage;
var receiveTimeout = TimeSpan.FromSeconds(Settings.Instance.Timeout);
// Wait for the message
semaphore.WaitOne((int)receiveTimeout.TotalMilliseconds, true);
}
// The Listener event
protected void OnMessage(IMessage receivedMsg)
{
var message = receivedMsg as ITextMessage;
semaphore.Set();
// process the message
}
Is the consumer durable?
Do you have to resubscribe after receiving a message?
Is this similar to other queuing/listener implementations (SQL Server service broker or the TCP/IP listener) where you need a while(true) loop to just keep the listener active?

Because of the way you've coded this, I believe (my .NET is rusty) you would need to create a new listener on each message as the using block will dispose of the consumer.
If you coded things such that the consumer was a member variable where it was saved away and only closed when you wanted to stop listening then you would not have this issue. The using block by it's nature will dispose of the resources that you ask it to manage.

Rabbit MQ - Recovery of connection/channel/consumer

I am creating a consumer that runs in an infinite loop to read messages from the queue. I am looking for advice/sample code on how to recover abd continue within my infinite loop even if there are network disruptions. The consumer has to stay running as it will be installed as a WindowsService.
1) Can someone please explain how to properly use these settings? What is the difference between them?
NetworkRecoveryInterval
AutomaticRecoveryEnabled
RequestedHeartbeat
2) Please see my current sample code for the consumer. I am using the .Net RabbitMQ Client v3.5.6.
How will the above settings do the "recovery" for me?
e.g. will consumer.Queue.Dequeue block until it is recovered?
That doesn't seem right
so...
Do I have to code for this manually? e.g. will consumer.Queue.Dequeue throw an exception for which I have to detect and manually re-create my connection, channel, and consumer? Or just the consumer, as "AutomaticRecovery" will recover the channel for me?
Does this mean I should move the consumer creation inside the while loop? what about the channel creation? and the connection creation?
3) Assuming I have to do some of this recovery code manually, are there event callbacks (and how do I register for them) to tell me that there are network problems?
Thanks!
public void StartConsumer(string queue)
{
using (IModel channel = this.Connection.CreateModel())
{
var consumer = new QueueingBasicConsumer(channel);
const bool noAck = false;
channel.BasicConsume(queue, noAck, consumer);
// do I need these conditions? or should I just do while(true)???
while (channel.IsOpen &&
Connection.IsOpen &&
consumer.IsRunning)
{
try
{
BasicDeliverEventArgs item;
if (consumer.Queue.Dequeue(Timeout, out item))
{
string message = System.Text.Encoding.UTF8.GetString(item.Body);
DoSomethingMethod(message);
channel.BasicAck(item.DeliveryTag, false);
}
}
catch (EndOfStreamException ex)
{
// this is likely due to some connection issue -- what am I to do?
}
catch (Exception ex)
{
// should never happen, but lets say my DoSomethingMethod(message); throws an exception
// presumably, I'll just log the error and keep on going
}
}
}
}
public IConnection Connection
{
get
{
if (_connection == null) // _connection defined in class -- private static IConnection _connection;
{
_connection = CreateConnection();
}
return _connection;
}
}
private IConnection CreateConnection()
{
ConnectionFactory factory = new ConnectionFactory()
{
HostName = "RabbitMqHostName",
UserName = "RabbitMqUserName",
Password = "RabbitMqPassword",
};
// why do we need to set this explicitly? shouldn't this be the default?
factory.AutomaticRecoveryEnabled = true;
// what is a good value to use?
factory.NetworkRecoveryInterval = TimeSpan.FromSeconds(5);
// what is a good value to use? How is this different from NetworkRecoveryInterval?
factory.RequestedHeartbeat = 5;
IConnection connection = factory.CreateConnection();
return connection;
}

RabbitMQ features
The documentation on RabbitMQ's site is actually really good. If you want to recover queues, exchanges and consumers, you're looking for topology recovery, which is enabled by default. Automatic Recovery (which is enabled by default) includes:
Reconnect
Restore connection listeners
Re-open channels
Restore channel listeners
Restore channel basic.qos setting, publisher confirms and transaction settings
The NetworkRecoveryInterval is the amount of time before a retry on an automatic recovery is performed (defaults to 5s).
Heartbeat has another purpose, namely to identify dead TCP connections. There are more to read about that at RabbitMQ's site.
Code sample
Writing reliable code for recovery is tricky. The EndOfStreamException is (as you suspect) most likely due to network problems. If you use the management plugin, you can reproduce this by closing the connection from there and see that the exception is triggered. For production-like applications, you might want to have a set of brokers that you alternate between in case of connection failure. If you have several RabbitMQ brokers, you might also want to guard yourself against long-term server failure on one or more of the servers. You might want to implement error strategies, like requeuing the message, or using a dead letter exchange.
I've been thinking a bit of these things and written a thin client, RawRabbit, that handles some of these things. Maybe it could be something for you? If not, I would suggest that you change the QueueingBasicConsumer to an EventingBasicConsumer. It is event driven, rather than thread blocking.
var eventConsumer = new EventingBasicConsumer(channel);
eventConsumer.Received += (sender, args) =>
{
var body = args.Body;
eventConsumer.Model.BasicAck(args.DeliveryTag, false);
};
channel.BasicConsume(queue, false, eventConsumer);
If you have topology recovery activated, the consumer will be restored by the RabbitMQ Client and start receiving messages again.
For more granular control, hook up event handlers for ConsumerCancelled and Shutdown to detect connectivity problems and Registered to know when the consumer can be used again.

C# MQ Api How to fetch message without getting exception in case of empty queue

I have to periodically check messages in a queue within Websphere MQ. I haven't found better approach rather than try getting a message and handle 2033 reason code (which is NO_MSG_AVAILABLE) like this:
try
{
// ...
inQueue.Get(message);
}
catch (MQException exception)
{
if (exception.ReasonCode != 2033)
throw;
}
Is there better way to get message from queue? I think that there might be some openOptions flag that I'm not aware of, that wouldn't throw exception when no message available, but return null instead.

There are three ways to avoid or reduce this polling mechanism.
Here they are in oder of elegance(the higher the better):
MQGET with wait interval UNLIMITED and MQGMO_FAIL_IF_QUIESCING
Get your application be triggered by MQServer
Callback function - new with MQ V7 on both sides

You are missing the MQC.MQGMO_WAIT flag on MQGetMessageOptions.Options. Change it this way:
getOptions = new MQGetMessageOptions {WaitInterval = MQC.MQWI_UNLIMITED, Options = MQC.MQGMO_WAIT | MQC.MQGMO_FAIL_IF_QUIESCING}
Please note that this would make the calling thread to be blocked till a message arrives at the queue or some connection exception occurs. MQ has another client called IBM Message Service Client (aka XMS .NET) that provides a JMS specification implementation in .NET. This has a nice little Message Listener which gets automatically invoked whenever a message arrives in a queue. Unlike in the above example, the calling thread will not be blocked when Message Listener is used.
More details on XMS .NET can be found here. Samples are also shipped with MQ and for message listener sample, please refer "SampleAsyncConsumer.cs" source file.

I was getting this. I solved it by putting the Message initiator inside the loop:
_queueManager = new MQQueueManager(Queuemanager, _mqProperties);
MQQueue queue = _queueManager.AccessQueue(
Queuename,
MQC.MQOO_INPUT_AS_Q_DEF + MQC.MQOO_FAIL_IF_QUIESCING + MQC.MQOO_INQUIRE);
string xml = "";
while (queue.CurrentDepth > 0)
{
MQMessage message = new MQMessage();
queue.Get(message);
xml = message.ReadString(message.MessageLength);
MsgQueue.Enqueue(xml);
message.ClearMessage();
}
There must be something in the Message internally that errors when reusing it for another get.

How to determine that a WCF Service is ready?

I have the following scenario:
My main Application (APP1) starts a Process (SERVER1). SERVER1 hosts a WCF service via named pipe. I want to connect to this service (from APP1), but sometimes it is not yet ready.
I create the ChannelFactory, open it and let it generate a client. If I now call a method on the generated Client I receive an excpetion whitch tells me that the Enpoint was not found:
var factory = new ChannelFactory<T>(new NetNamedPipeBinding(), new EndpointAddress("net.pipe//localhost/myservice");
factory.Open()
var Client = factory.CreateChannel();
Client.Foo();
If I wait a little bit before calling the service, everything is fine;
var Client = factory.CreateChannel();
Thread.Sleep(2000);
Client.Foo();
How can I ensure, that the Service is ready without having to wait a random amount of time?

If the general case is that you are just waiting for this other service to start up, then you may as well use the approach of having a "Ping" method on your interface that does nothing, and retrying until this starts responding.
We do a similar thing: we try and call a ping method in a loop at startup (1 second between retries), recording in our logs (but ultimately ignoring) any TargetInvocationException that occur trying to reach our service. Once we get the first proper response, we proceed onwards.
Naturally this only covers the startup warmup case - the service could go down after a successfull ping, or it we could get a TargetInvocationException for a reason other than "the service is not ready".

You could have the service signal an event [Edited-see note] once the service host is fully open and the Opened event of the channel listener has fired. The Application would wait on the event before using its proxy.
Note: Using a named event is easy because the .NET type EventWaitHandle gives you everything you need. Using an anonymous event is preferable but a bit more work, since the .NET event wrapper types don't give you an inheritable event handle. But it's still possible if you P/Invoke the Windows DuplicateHandle API yourself to get an inheritable handle, then pass the duplicated handle's value to the child process in its command line arguments.

If you're using .Net 4.0 you could use WS-Discovery to make the service announce its presence via Broadcast IP.
The service could also send a message to a queue (MSMQ binding) with a short lifespan, say a few seconds, which your client can monitor.
Have the service create a signal file, then use a FileSystemWatcher in the client to detect when it gets created.
Just while (!alive) try { alive = client.IsAlive(); } catch { ...reconnect here... } (in your service contract, you just have IsAlive() return true)

I have had the same issue and when using net.pipe*://localhost/serviceName*, I solved it by looking at the process of the self-hosted application.
the way i did that was with a utility class, here is the code.
public static class ServiceLocator
{
public static bool IsWcfStarted()
{
Process[] ProcessList = Process.GetProcesses();
return ProcessList.Any(a => a.ProcessName.StartsWith("MyApplication.Service.Host", StringComparison.Ordinal));
}
public static void StartWcfHost()
{
string path = System.IO.Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
var Process2 = new Process();
var Start2 = new ProcessStartInfo();
Start2.FileName = Path.Combine(path, "Service", "MyApplication.Service.Host.exe");
Process2.StartInfo = Start2;
Process2.Start();
}
}
now, my application isn't called MyApplication but you get my point...
now in my client Apps that use the host i have this call:
if (!ServiceLocator.IsWcfStarted())
{
WriteEventlog("First instance of WCF Client... starting WCF host.")
ServiceLocator.StartWcfHost();
int timeout=0;
while (!ServiceLocator.IsWcfStarted())
{
timeout++;
if(timeout> MAX_RETRY)
{
//show message that probably wcf host is not available, end the client
....
}
}
}
This solved 2 issues,
1. The code errors I had wend away because of the race condition, and 2
2. I know in a controlled manner if the Host crashed due to some issue or misconfiguration.
Hope it helps.
Walter

I attached an event handler to client.InnerChannel.faulted, then reduced the reliableSession to 20 seconds. Within the event handler I removed the existing handler then ran an async method to attempt to connect again and attached the event handler again. Seems to work.