I am considering using SignalR for server-to-client real time communication. However, I need to guarantee delivery, so I need some form of ACK in the process.
I have seen answers here with suggestions for how to do this, but I also see that the Microsoft documentation for SignalR includes a Message.WaitForAck bool property. This makes me hopeful that perhaps Microsoft baked something in to do this--but I can find no postings at all of folks using this, nor any posts explaining what it does.
Is it just an inert flag? That is, are we still on the hook to roll our own ACK system?
Thanks.
WaitForAck is an internal thing. SignalR is build around a MessageBus where WaitForAck is used for some operations that should block until completed (or timed out). An example of such operation would be adding connection to a group.
If you want a guarantee delivery you need to implement it on your own on top of SignalR.
Related
I'm trying to figure out how to implement a fault-tolerant message publication solution using MassTransit. We'll focus on the simple scenario where we only need to commit a database change, and publish an event indicating that change. Because there is no (built-in) mechanism that allows an atomic "commit and publish", when our process crashes, we will get into an inconsistent state (some messages would only be committed to the database, and some might only be published to the message queue).
This documentation page offers a solution, where because we assume message handling is idempotent, we can rely on the entire operation to be retried in case of failure, and these partial commits will be resolved eventually. This is a great solution, but it only has one caveat: it assumes that the operation we are performing was triggered by a message, and if we won't send an ack, processing will be retried. This is not a reasonable assumption, as messaging is typically used only for internal communication inside the system, not for communication with the outside world. What should I do when I need to save-and-publish when handling an HTTP request from an external client?
One possible solution is to hack our way into the approach presented in the article, by only publishing (or sending) a message, and listening to it ourselves, then in the message handler we do the commit and the publishing of the actual event we want others to listen to. The main problem I have with this is that it assumes we never have to return anything in the HTTP response. What if we need to indicate the success or failure of the database transaction back to the HTTP client? (example: if we rely on a UNIQUE constraint to tell us whether or not we should accept the request, and we want to indicate failure to the client). We could solve it by using request-response over the message queue (with ourselves), but this is ugly and increases latency and complexity considerably, for what is actually a very common scenario.
The approach I see the most over the internet to solve this problem, is to use an outbox that is persisted to the same database we need to write to anyway, and thus we can wrap the two operations in a regular ACID database transaction. Then a background task polls this database for new events and publishes them to the message broker. Unlike other frameworks, I understand that MassTransit does not support this behavior out of the box. So I guess my question boils down to: before rushing to implement this relatively complex mechanism myself (once per database technology), is there another solution I'm missing? what is the accepted solution to this problem in the MassTransit community?
This has been asked several times, in a variety of forms, here and other places. But the short answer is simple.
In your controller, write to the message broker only. Let the consumer deal with the database, in the context of consuming a reliable message, with all the nice retry and redelivery options that are available in that context. Then you get all the benefits of the InMemoryOutbox, without adding extreme complexity related to having a third-party (HTTP, database, and broker) in a single conversation.
I have an application running on angular.js, WebAPI, and mongodb. My goal is to replace client side polling, with signalr. Since mongodb doesn't fire events on changes, I still need to poll on the server side, and if I detect a change, broadcast it using a signalr hub.
Is there a solid way to do that, without breaking the recurring task by IIS?
What have I tried:
Hangfire: Seems like the recommended solution, but currently supports only SQL Server and redis (not an option).
HostingEnvironment.QueueBackgroundWorkItem: Might work, but I fear that it is not widely used, and I might miss some pitfalls.
There's a way to kind of get events from mongo using "tailable cursor on capped collection" - not an option
The answer as always: "it depends".
HostingEnvironment.QueueBackgroundWorkItem. The latest addition in .NET 4.5.2. No guarantees on execution, see the limitations at the end of post:
WebBackgrounder. Quote from Scott Hanselman blog: "Its code hasn't been touched in years, BUT the WebBackgrounder NuGet package has been downloaded almost a half-million times".
Hangfire. Popular open source project, adding mongodb support is always a welcome PR :)
Using RavenDb instead of MongoDb. Their Changes API is really cool.
The first and second options don't guarantee job execution, the third and fourth options rely on data stores instead.
QueueBackgroundWorkItem is your best option consider what we discussed above, but its not 100% bullet proof. The app pool will still recycle. You can create a Windows service that signal the Web app to poll the DB.
Anyway, once you got the singal (through polling or what not) you could use event driven design from that point on. The benefit is decoupled domains and scalability.. I have created this framework for just that.
https://github.com/AndersMalmgren/SignalR.EventAggregatorProxy/wiki
Is there any way to setup a publish/subscribe system (1 publisher, 0..* subscribers) where every subscriber is guaranteed to receive every message exactly once and in the same order the messages were sent? Most bus systems (e.g. NServiceBus) will not guarantee this.
I don't care if I have to implement the system myself, but at least the transactional asynchronous communication (e.g. queue, or similar) should be able to do this.
Any hints/suggestions?
In my (limited) experience you can you RabbitMQ to achieve this:
https://www.rabbitmq.com
In specific, I recommend the pubslish/subscriber tutorial that they have:
https://www.rabbitmq.com/tutorials/tutorial-three-python.html
Please note that specific tutorial does not involve the concept of confirms which would be the next step to make sure that the messages are delivered to the consumers.
As for message ordering it may also be achievable, depending on your use case, as explained in this stackoverflow post:
RabbitMQ - Message order of delivery
Hope it helps.
I'm consuming a notoriously unreliable SOAP web service in a c# MVC project. I have no control over that service. It will occasionally just stop responding for extended periods. Is there a simple way, from the client side, to continue to retry sending the message, at a set interval, until I receive a response back from the service?
I've looked at MSMQ and ws-reliability but, it appears to me, both options require some control over the web service. Is there anything out there to help me to do this from the client side?
As you probably already figured out, your problem is a huge problem for many. Look up "idempotent" and "webservice". Idempotency means a lot more than just being able to ensure a request/response, but a search will give you plenty of good stuff to read.
If "stop responding for extended periods" means seconds while seldomly called upon, DarkWanderer showed a pretty brute force solution to such a problem.
But if you have many calls, sleeping may eat up your working threads, so then you simply have to rely on some kind of queue.
If your calls are non-transactional and non-critical, you could surely code your own queing mechanism. While this may seem easy, it may still require threading, complex callbacks, logging, active error handling and what not. Many poor souls have reported that what started as a simple solution was turned into a maintainance nightmare.
And now I see that one of your requirements is that it must survive an app-pool recycling. Then we are in the last category of critical (and maybe transactional) queing.
Why I would recommend MSMQ from the very start. It handles all your problems and the api is in .net and really good nowadays. Yes, it will add complexity to your overall solution, but that stems from your problem domain.
while (true) {
try {
var response = webServiceClient.CallMethod();
if (response.Successful())
break;
Sleep(retryInterval);
} catch {}
}
And that means, you just need to keep calling the web-service, no message queue or something is required. Does that answer your question?
What we need is RabbitMQ that actually works as a queue and doesn't do this. Messages should stay at the head of a queue untill client dequeues them explicitly.
It seems like a very straightforward scenario, but for some reason I can't find any broker to support it.. A broker should run on Windows OS.
Apache Qpid is probably your best option. Of all of the message queues, this one has a number of interesting things going for it, including strict ordering.
If it is only one message that is the problem, why not write it to a file (and flush the file) before you process the message. After acking the message, delete the file.
And if you are concerned about the message broker crashing, first step is to upgrade it to RabbitMQ 2.4.1 running on Erlang R14B02. Second step is to cluster it so that you have multiple servers acting as the MQ broker. And only then, change your app to track the messages that have been processed, either by timestamp or by saving message IDs. Then, if RabbitMQ requeues a message, you will already have it and will process it and remember it. When it comes around a second time you will ignore it.
You may need to set prefetch to 0 for this to work right.
And there is another alternative too. You could consider writing your own RabbitMQ plugin to provide the exact behaviour that you need. Erlang may look complex at first sight, but it really isn't that hard to learn for an experienced programmer who has already learned a few languages. In particular, if you have anyone with functional programming experience in languages like Haskell or CAML, they will quickly pick up enough Erlang to do the job.
Because of Erlang's internal model of message-passing processes, RabbitMQ plugins can essentially do anything that they want. There is no specific limited plugin API that they need to conform to.
In other words, if RabbitMQ only does 99% of what you need, consider yourself lucky that with a small amount of work, you can leverage that 99% and achieve everything that you need. But in order to do this you have to get away from the idea that RabbitMQ is yet another package that you install with your system's package installation tools. In cases like yours RabbitMQ should be considered to be a mission critical tool, and you should install Erlang and RabbitMQ from source, and configure them to your needs without letting your OS limit you.
RabbitMQ also supports strict ordering as of release 2.7.0 and so should be an option for your scenario again.