We Keep Coding

How I can poll messages from Solace queue (instead of default pushing behavior)?

I'd like to write parallel execution module based on Solace. And I use request-reply schema for this.
I have:
Multiple message consumers, which publish messages into the same queue.
Multiple message producers, which read queue and create reply messages.
Message execution time is between 10 seconds to 10 minutes.
Queue access type is non-exclusive (e.g. it does round-robin between all consumers).
Each producer and consumer is asynchronous, e.g. Solace API blocks execution during the connection only.
What I'd like to have: if produces works on the message, it should not receive any other messages. This is extremely important, because some tasks blocks executor for several minutes, however other executors can be free after couple of seconds.
Scheme below can be workable (possible), however blocking code appears below. I'd like to avoid it.
while(true)
{
var inputMessage = flow.ReceiveMsg( /*timeout 1s*/1_000); // <--- blocking code, I'd like to avoid it
flow.Ack(inputMessage.ADMessageId);
var reply = await ProcessMessageAsync(inputMessage); // execute plus handle exceptions
session.SendReply(inputMessage, reply)
}

Messages are only pushed to the consuming applications.
That being said, your desired behavior can be obtained by setting the "max-delivered-unacked-msgs-per-flow" on your queue to 1.
This means that each consumer bound to the queue is only allowed to have 1 outstanding unacknowledged messages.
The next message will be only sent to the consumer after it has acknowledged the message.
Details about this feature can be found here.
Do note that your code snippet does not appear to be valid.
IFlow.ReceiveMsg is only used in transacted sessions, which makes use of ITransactedSession.Commit to acknowledge messages.

Botframework: how to handle long running tasks with a bot?

How do I handle a long running tasks on a bot so the client dosnt retry to send the message after 15 seconds again.
I got a bot with the botframework v3 and connect the client with directline

The Direct Line channel connector itself does not retry sending messages. If it does not receive an ack within 15 seconds of sending a message to your bot, it will throw a Gateway Timeout.
If you are using the DirectLineClient, you can override the retry policy, ensuring the client does not retry messages:
DirectLineClientCredentials creds = new DirectLineClientCredentials(directLineSecret);
DirectLineClient directLineClient = new DirectLineClient(new Uri("https://directline.botframework.com"), creds);
directLineClient.SetRetryPolicy(new Microsoft.Rest.TransientFaultHandling.RetryPolicy(new Microsoft.Rest.TransientFaultHandling.HttpStatusCodeErrorDetectionStrategy(), 0));
If you have a long running process, that takes more than 15 seconds, consider queuing the message somewhere, so you can acknowledge the call immediately, then process the message on a background thread. This is conceptually called Proactive Messaging. More information can be found here: https://learn.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-proactive-messages?view=azure-bot-service-3.0
Edit: This blog post also explains one method for handling long operations within a bot, by using Azure Queue storage and an Azure Function which processes the operation and calls the bot when finished:
Manage a long-running operation
Another option is to process incoming messages, or long processing messages, on a background thread. This experimental sample demonstrates some methods using this design:
Immediate Accept Bot

MSMQ: Messages occasionally not being sent or received without error

I have the following setup and problem with MSMQ. Based on previous experience with MSMQ I'm betting that it is something simple I'm missing but I just don't know what it is.
The Setup
I have 3 load-balanced web servers (lets call them Servers W1, W2 and W3) and 1 server which processes certain events/data away from web requests (which I'll call P). All 3 of the web servers, once a particular event occurs within the web application, will send a message to a remote private queue on Server P, which will then process each message from the queue and carry out some task.
The Problem
For the most part - at a guess 95% of the time - everything runs fine, but occasionally Server P does not receive messages from the web servers. This is either because W1, W2 or W3 are not sending them or they are not being received by P, I just can't tell. This means I'm missing vital events happening from the users on the web application but I cannot find any errors listed in my own logs.
The Details
Here are all the details I can think of which may help explain my setup and what I've figured out so far:
The private queue on Server P is non-transactional.
The private queue has permissions setup for Everyone to both Send and Receive Messages.
This is the code I use (C#) to send the message to the remote private queue:
var queue = new MessageQueue(#"FormatName:DIRECT=OS:ServerP\PRIVATE$\MyMessageQueue");
var defaultProperties = queue.DefaultPropertiesToSend;
defaultProperties.AcknowledgeType = AcknowledgeTypes.FullReachQueue | AcknowledgeTypes.FullReceive;
defaultProperties.Recoverable = true;
defaultProperties.UseDeadLetterQueue = true;
defaultProperties.UseJournalQueue = true;
queue.Send(requestData);
Sending the message using the code above does not appear to throw an exception - if it did my error handler in the web application would have caught and logged it, so I'm assuming it is sent.
There are outgoing queues on W1, W2 and W3 all pointing to the private queue on P - all these are empty.
On W1, W2 and W3 I cannot see any "dead-letter" messages.
On P the private queue is empty so messages are being processed (which I can verify from my database).
On P there are no "dead-letter" messages. There are journal messages but they don't seem to correspond to any recent date/times.
All servers are running Windows Server 2012.
Most of the time messages are sent, received and processed just fine but, without any pattern visible to me, sometimes they are not. Can anyone see what is going wrong? Or explain to me how I can try and figure out what is happening?

Are you sure that the receiver on P does not crash/lose the message somehow? Because your queue is not transactional, if somehow processing fails then that's one lost message.
Anyway, there are many possible causes why this could fail.
What kind of logging do you have (DEBUG/INFO levels)?
I think the following will help tracking down the issue:
When an event is generated in the web app.
Right before you send an event from the web app, via MSMQ.
In the receiver when you get a message from the queue.
This way you could at least match sent messages to received messages and to processed messages.
As a side note, when you check for dead-letter messages you do so on the source computer and on any intermediary hops, not on the destination one. If you don't have any hops, then they will be relayed to the non-transactional dead-letter queue on the web servers.

How can we speed up receiving messages from MSMQ?

My application's bottleneck has become sending and receiving messages over MSMQ with MassTransit. The send and receive are both happening within the same application, but there are often too many messages to use an in-memory queue.
Here is my simple queue setup:
messageBus = ServiceBusFactory.New(sbc =>
{
sbc.UseMsmq();
sbc.VerifyMsmqConfiguration();
sbc.UseMulticastSubscriptionClient();
sbc.ReceiveFrom("msmq://localhost/msg_queue");
sbc.Subscribe(subs =>
{
subs.Handler<EventMessage>(msg => Enqueue(msg));
});
});
For my experiments, MSMQ currently has ~1 million messages and we are not pushing any new messages to it. We are not doing any work in Enqueue() except time how quickly messages are being sent.
With that information, we can only grab between 150 and 200 messages per second from MSMQ, when I'd hoped it would be at least 1000 messages per second when there is no network latency. Each messages is <2kb.
How can we speed up how quickly MSMQ passes messages to an application over MassTransit while maintaining the message ordering enforced by a queue?

I did addressed something similar before. If I remember it correctly,
We specified message expiration through TimeToBeReceived (The total time for a sent message to be received from the destination queue. The default is InfiniteTimeout.) . Refer this msdn link.

Microsoft ServiceBus Receiving BrokeredMessages Out of Order

I have a javascript logging utility that sends requests in bulk to my server which then relays them to a Queue Client (Microsoft.ServiceBus.Messaging.QueueClient). I want to send them in batch asynchronously to the ServiceBus and still have them processed in the order they are placed into the batch I am sending. The documentation for SendBatchAsync shows that the method is for "batch" processing. This makes me think I can send it a batch of requests and have them processed as a single unit (i.e.: sequentially). Although, it appears that the messages are getting processed out of order. I'm using OnMessage to receive the messages; I'm not sure if this is a limitation or what am I missing?
I get that async doesn't guarantee order vs. other async requests, but this is a single request. I don't want to have to wait for a response before responding to the javascript client as I'm just trying to send them off, but I still need to ensure they stay in order since they are sequential events.
Here is how I send them to the queue:
MyQueueClient.SendBatchAsync(MyListOfBrokerMessages);
Then I process them:
ServiceBus.TrackerClient.OnMessage((m) =>
{
try
{
ProcessMessage(m);
}
I don't get the point of the batch processing if it doesn't process as a batch other than maybe making a single request. There must be some way to send a batch and have it process in order??
EDIT:
I've tried using Send instead of SendBatchAsync and I've set MaxConcurrentCalls to 1 and yet the messages are still not in order.

Taken from MSDN:
SessionId: If a message has the
Microsoft.ServiceBus.Messaging.BrokeredMessage.SessionId property set, then Service Bus
uses the SessionId property as the partition key. This way, all messages that belong to
the same session are handled by the same message broker. This enables Service Bus to
guarantee message ordering as well as the consistency of session states.
For a coding sample employing SessionId and AcceptSessionReceiver see.

What you can do is to use Sessions here,
Set the same sessions id to all the messages in the batch
Receiving side, AcceptMessageSession() will give you a session
Call receive on the session (ReceiveBatch). This session will give you all the messages in that batch alone.