I use WCF to execute some code on a server (takes a long time). The request to execute that code is send from several client machines within the local network (TCP).
Since the code takes a while to execute and there are a bunch of clients, I need to somehow queue the requests.
I thought about using MSMQ as queueing mechanism. As far as I understood, MSMQ has to be installed on the client machines as well, not just the server.
Is it possible to not have to install MSMQ on all clients? Are there any alternatives to MSMQ, or simpler ways to have a queue in between the clients and the WCF-service?
Is it possible to not have to install MSMQ on all clients?
No. You must have msmq installed on all machines which want to participate in the sending and receiving of messages.
Are there any alternatives to MSMQ
Yes, many.
or simpler ways to have a queue in between the clients and the
WCF-service?
WCF uses msmq for queued transport on-premise. However, it is possible to use WCF with Azure Service Bus if you want to run in Azure. Link
Related
I just want to find out if this is possible or whether I'm over complicating it.
I have a web application (Let's call it central) that needs to interact with a WCF service that's running on multiple workstations. i.e. The user will select the workstations to send messages to and the web application will need to do a call to each of the workstations. These workstations won't be online all the time and in the worst case there can be just over 600 workstations that messages will need to be sent to at a single time.
I'm thinking of having a separate WCF service running on the central machine that will function as a sort of "proxy" between the web app (central) and the workstations. The web app will then make a single call to this service with a list of messages, the service will then process this list and add the messages to a queue.
From what I've read so far, the workstations will need to poll this central queue for messages but this seems like it will increase overhead quite a bit. Is it possible to push the messages down to the workstation as they are added to the queue?
I've never used MSMQ before and I'm fairly new to WCF as well so if there's a simpler way of achieving this do tell.
I'm thinking of having a separate WCF service running on the central machine that will function as a sort of "proxy" between the web app (central) and the workstations
I don't see any problem with you using an additional service to act as a "proxy".
the workstations will need to poll this central queue for messages
I would probably advise that you have a central "inbox" of messages to process on the central server. The "proxy" then locally processes the inbox. Have the web site send the jobs to the proxy.
Push model
For the workstations, rather than having them read from the server, have the server send the messages to the workstations. Have a unique queue for each workstation and that queue be situated on the workstation itself. Have the service send the message targeted for the workstation to the workstation's queue. As each message from the central queue/inbox is processed by the proxy, the proxy removes said message from the queue and moves onto the next one.
MSMQ Performance
Generally in MSMQ it's better to do network writes than network reads for scaling reasons. A multitude of computers all reading from the same queue is hurtful to performance. In other words, have the "central" service write to the remote queues (MSMQ will take care of transmitting the message when the machine is available).
the workstations will need to poll this central queue for messages but this seems like it will increase overhead quite a bit
Correct. Use BeginRead. Polling is hurtful to CPU and/or a waste of a good thread.
I want to send commands from one application (e.g. running on mobile device) to another application (e.g. running on embedded device) which is located in a different network.
I don't want to use VPN or something like port forwarding. So after some research I found some other ways to do that, for example via a cloud messaging service like Azure Service Bus.
Sending commands/messages from the first application to the service bus is not a problem for me. But I don't really understand how two get a connection from the cloud service to the second device? I know I can also send a message from the second device to a cloud service e.g. via HTTPS. And then the cloud service can keep that connection alive. As long as the connection is alive, I can send messages to the second device.
But there are some points I can't understand:
When I have thousands of devices, isn't that a problem to keep thousands of connections alive?
How can the second device listening the connection if there are new messages? Doesn't that needing too much ressources on the embedded device?
I also read about using "long polling" techniques and web sockets. I know too little to understand what are the advantages and disadvantages of those concepts. Which technique should I use for my problem?
To be more platform agnostic, I don't want to use services like Azure IoT Hub.
Edit:
Maybe I can use a web service and implement a MQTT Broker?
I think the mentioned MQTT Broker will get you there, especially as your usecase is exactly what MQTT and it's implementations (brokers and clients) have been built for.
The simplified story is the following:
A MQTT Client running on your Application 'publishes' a MQTT message using a 'topic' (think routing key) to the MQTT broker. A MQTT client running on your Devices have a subscription for the same 'topic' on the broker. This enables the broker to route the message from the application to the devices without the requirement that they know about each others.
As far as I understand your question your concerns are the following:
can all the devices be connected at the same time (thousands of open TCP connections) and therefore receiving messages published from your first application via the broker in 'realtime'.
assuming the devices will disconnect for whatever reasons, e.g. due to network problems or for decreasing energy consumption, how would be ensured that the devices will eventually receive the messages.
how will the devices connect to the broker.
Regarding 1. MQTT brokers are built to handle (and keeping) a massive amount of TCP connections. For example VerneMQ, a MQTT broker I can talk about, as I am one of the core devs, is able to handle over a million connections on one node (with proper server configuration it's actually mainly a matter of available RAM). However we'd only recommend such a setup if the devices are mainly sleeping. Using VerneMQ you can also add more nodes to the cluster and balance the connections among all your cluster nodes.
Regarding 2. A MQTT broker typically implements an offline storage for messages that haven't been send out to a client or haven't been acknowledged by a client. This allows your device to go offline for hours and receive the messages upon reconnect.
Regarding 3. This is specific to your usecase. In the simplest case you configure a fixed IP:Port on every device, and the MQTT client running on the device uses it to connect to the broker. Depending on the ability to reconfigure the devices it makes sense to use DNS lookups, or even to provide a 'backchannel' for reconfiguration.
For standard compliant MQTT client software have a look at Eclipse Paho. For an up-to-date list of available MQTT brokers consult the list of MQTT brokers.
I have messages that I want to send from multiple WPF client applications to a service that can be processed some time after being sent.
Because of expected intermittent connectivity issues between client and server and necessary down time for the service, I'm inclined to create a WCF service with a queued endpoint. This has worked well for me in the past when the client machines were actually other servers and few in number.
I'm concerned about doing this with many client machines primarily because I think it will be difficult to monitor so many outgoing queues to confirm that no traffic is being trapped on the client machines.
Has anyone tried doing this before?
If so, would you recommend it? Why or why not?
Even if you haven't done it, can you think of other pitfalls beside the operational issue of monitoring all those outbound queues?
Your question may be better worded as:
Should a system be rolled out with many nodes all using MSMQ?
If so this is the essence of messaging and is what such systems are designed for irrespective of whether they are JMS, Apache MQ, Websphere, SonicMQ, or MSMQ.
Also, "traffic is being trapped on the client machines" - how do you define trapped? Remember, the application may be quite happy for the message to be sitting locally for days before being forwarded to the remote host. Messaging systems have timeouts generally for both reaching the destination and for the destination to process it.
I think you will be fine.
HTTP transfer has been available since MSMQ 3.0, however I'm afraid MassTransit doesn't offer the feature to use HTTP protocol as transport protocol between queues.
There's a very similar question about this here, which has not been completely answered.
Does anyone know if it's possible for a client to subscribe to a bus and send/receive messages through HTTP? Here's the architecture I'm willing to implement:
I'll have 2 computers in the local network
Computer A runs a server application and MassTransit.RuntimeServices
Computer B runs a client application which sends messages to A
I want the communication between them to be done via http.
I tried to change the address in UseSubscriptionService to http instead of msmq, but it doesn't work. If I set computer's A MSMQ service to Hardened Mode, the client application running on computer B get's a timeout while trying to subscribe to mt_subscriptions.
Bus.Initialize(sbc =>
{
sbc.UseMsmq();
sbc.VerifyMsmqConfiguration();
sbc.UseMulticastSubscriptionClient();
sbc.ReceiveFrom("msmq://localhost/test_queue_client");
sbc.UseSubscriptionService("msmq://m3-dev1/mt_subscriptions"); // maybe I could use http instead of msmq, but it doesn't work
});
Any clues on that??
No, MassTransit does not support HTTP for MSMQ. You could, in theory, add your own transport that supports that. RabbitMQ is a way better transport than MSMQ in every way except if you need to enroll in distributed transactions. And RabbitMQ only requires a single port open between the boxes.
Our approach for distributed integration is to have a custom web service contract with proper security. This defines an input port. The input port actually publishes messages to mass transit buses.
On the other side, the same contract is used to deliver messages to subscribers.
By having a custom contract and http/https transport we are independent on actual message bus in the middle. And this pays of, we were using another bus for like 2 years and 2 years ago we migrated to mass transit based bus without ANY changes to clients (publishers/subscribers).
I was going through this article http://weblogs.asp.net/spano/archive/2007/10/02/choosing-the-right-wcf-binding.aspx to choose Binding options.
where i got an unusual doubt of what is mean by
offline or disconnected interaction, choose the NetMsmqBinding
Does that mean the even if service is not running still client using the service ?
Can you share some real time example ?
The 2 bindings are radically different.
NetTcpBinding can be thought of as an MS proprietary format (usually binary) similar in concept to RPC (e.g. can be used to replace .NET Remoting). It is synchronous, i.e. both client and server must be online at the same time, and the client receives a response (almost) immediately.
MSMQ is a Message Oriented Middleware solution by Microsoft, which revolves around asynchronous queues - e.g. if the destination server is offline when the client sends a message, the message will be queued on the client until the server comes back online. Each queue is one way only, although bidirectional communication can be achieved via a second queue back from server to client. Sending WCF MSMQ messages requires that the MSMQ Service be installed on the client. Messages on the queue can have a delivery 'timeout' else will be placed on an applicable dead letter queue.
Real world examples:
I would use NetTcpBinding with binary serialization for high performance, synchronous communication needs between a Microsoft WCF client and server, e.g. uploading files, media etc where Xml would not be useful (otherwise, I would use wsHttpBinding for synchronous Xml / SOAP messaging)
I would use MSMQBinding with DTC enabled to ensure reliable messaging between 2 or more systems (e.g. financial), with at least one of the endpoints being in .Net, and a 'compatable' server (not necessarily WCF, e.g. BizTalk, or other EAI hubs or ESB buses which have adapters for MSMQ, e.g. Bridges exist between MSMQ and MQSeries). Messages would typically be in an Xml format.
TL;DR
Does that mean the even if service is not running still client using the service
Yes. If MSMQ is running locally, the client will get an immediate return successful response (indicating that the message has been queued). This does not however mean that the message has been successfully received by the server.