We Keep Coding

Communicating with a Compute role with Service bus vs. WCF

Given a simple high level architecture, e.g a cloud service with a web role, and a compute role, under what circumstances would we choose to use WCF as the communication method between the web role and the compute role, rather than service bus.
There is a lot of documentation, and examples regarding service bus, but I would like to understand if there are any platform benefits to using Service Bus, rather than WCF.
Given the calls are synchronous, and short, e.g a typical API call for getting data onto the website, would you choose WCF over queuing messages and replies onto a queue?
It would appear logically that for a synchronous call WCF would offer the least amount of overhead and latency?
I don't fully understand if the platform offers any "clever" tricks to keep the service bus operating as quickly as a TCP connection over WCF, (Given the queuing overhead?) and would like to understand this further.
At the moment if I was to pick an implementation for this type of call I would choose WCF, which maybe a little naive.
Just to clear, the calls always return data, they are not long running, or fire and forget.
Thanks!

I think it depends on what specifically you want to do.
Service Bus is typically used more for what I would call constant contact type interactions. It should be more performant, but more complex to set up. It also has bi-directional communication capabilities. So you get a lot of extra flexibility out of it.
I would swap WCF for the more modern Web Api. Both solve the same core problem primarily in serving up content. I think of it as just that an API, not necessarily a platform for message passing and handling. They solve 2 different core problems.
I would actually solve the likely problem differently and use Azure Websites + WebJobs. Its the same sort of thing. You can bind the WebJob to an Azure Queue, table or blob and put messages on that storage mechanism, which the job picks up and does something with. The web role I do not believe should rely on content coming back from the job. The job may hit a SignalR Hub that you have on the AzureWeb site post completion, which pushes state back down to the affected parties.
Reference Materials:
WebJobs: https://azure.microsoft.com/en-us/documentation/articles/web-sites-create-web-jobs/
SignalR: http://signalr.net/
Azure Web Apps: https://azure.microsoft.com/en-us/services/app-service/web/

MSMQ listener with WCF?

Scenario :
I want to implement an MSMQ in which users input message through System.Messaging API's. And there should be a listener which always observes this queue so when ever there is a message in queue I want to make a database updation.
My first approach was to implement MSMQ trigger. I was able to implement a COM interop dll trigger. But I wasnt able to do database operation and I couldnt figure out what was wrong, I tried a lot . Then I came to know about this WCF MSMQ binding. As I am new to this WCF I have some doubts.
Which is the best approach to host WCF for this case. Is it IIS with WAS or Windows service?
And for this kind of listener service is a client necessary or can we write the database operations directly under the Service host operations without client invocation?

Then I came to know about this WCF MSMQ binding. As I am new to this
WCF I have some doubts
Well, that's valid. WCF has a fairly steep learning curve, is very config-heavy, and is not everyone's cup of tea.
However, if you're integrating to MSMQ, WCF has very good support and is rock solid in terms of the way it is implemented.
Which is the best approach to host WCF for this case. Is it IIS with
WAS or Windows service?
Unless you're hosting exclusively on a web environment, I would choose a windows service every time. Using something like topshelf, the deployment and management overhead is tiny, and there are no external dependencies.
Remember to use msmqIntegrationBinding rather than netMsmqBinding, as the latter relies on WCF at both ends, whereas the former supports System.Messaging on the client, or even better, use WCF on the client and then you can use netMsmqBinding which support "typed" messages.

How to implement publish/subscribe communication over the internet

I have a .Net service hosted on a server, and .Net clients connecting to this server over the internet.
I want to implement a publish subscribe model where clients can subscribe to events on the service and have data pushed to them as data becomes available. An alternative would be to have clients poll the server for data, however this will likely be too slow for what is required. Hence the need for a publish/subscribe type of communication.
I understand that the WCF binding WSDualHttpBinding does allow this, however it comes with a downside. According to "Programming WCF Services" author Juval Lowy,
...the WSDualHttpBinding is mostly
unusable, since it is practically
impossible to tunnel through various
communication barriers separating the
service from the client and the need
to find a specific web-server machine
makes this impractical.
I've interpretted this to mean (please correct me if I'm wrong) that to operate with WSDualHttpBinding, it is necessary for clients to open a port on their machines (along with any necessary router configuration) for the server to callback through. If this is the case using WSDualHttpBinding will not be an option for me. Using Windows Azure will also not be an option.
So the crux of my question is, how can I achieve a publish/subscribe/callback type of communication over the internet, without the need to open ports on the client machine? Open standards are ok but unnecessary as client and server are both .Net, Windows Azure is not an option.

WSDualHttpBinding contains two channels one from client to server and second from server to client. The later one indeed requires firewall and NAT configuration. What about Net.Tcp? It uses only single channel and supports callbacks (duplex communication) over that channel initiated from client to server. You mentioned that both client and server will be .NET application so it should be possible with some firewall configuration on the server.

You mention most of the options in your post.
There are 3 options:
Client polls the server. Does not require open ports but too slow.
WSDualHttpBinding requires opening of ports
Azure Service Bus, would do it but not an option.
There is actually a way to do it. If you look at how Azure service bus works, it tricks the client into thinking that it is on an out going port, when really it being used to send messages to the client. You could try to implement this functionality.
Another thing you could try is nservicebus at http://nservicebus.com/, this is an open source .net service bus.

Internet Communication Engine (ICE) offers IceStorm, a publish/subscribe service.
Its open source, and if you download the installation there is a sample Visual Studio project that demonstrates how to implement publish/subscribe (check out the "demos" .zip file and the "IceStorm" directory with the clock demo).
ICE will do all the heavy lifting for you, the learning curve is remarkably short, primarily because the documentation is massive, approachable and well written.

I strongly recommend you DDS (data distribution service from OMG) over the internet http://www.omg.org/news/meetings/workshops/Real-time_WS_Final_Presentations_2008/Session%203/03-02_Bertocci_et_al.pdf
From OMG, that's all I gotta say. Yes, I know you may think OMG is over. I don't, and as a government counselor I really push for standards. Keep in mind that besides the liberal ideology and crises, governments are still a huge customers and inter-operation is a must.
NServiceBus? Right, ok, that's fine, but ain't too complex to start right now? Learning curve is too ... steep? I guess is not but, any easier one?
ICE is a good choice. guys from CORBA world trying to make things better. Don't doubt it, use it, try it! Just a feeling: even with the storm service you might feel you are still in the request/reply world ... but is that a con?
But if you rather prefer a commercial but open solution, think about publish subscribe using protocol buffers (search Google protocol buffers) ... just a first approach http://protocolbus.casessite.org . This is my owm work ... sorry it's just an initial project but I'm working with a central broker distribution as an alternative transport (default is multicast but broadcast and udp is also available). Open source, so be free ...

Communication (interprocess) between applications

I'm about to write a "server" application that is responsible to talk with external hardware. The application shall handle requests from clients. The clients send a message to the server, and if the server is busy with doing stuff with the hardware the new messages shall be stored in a queue that will be processed later.
The client shall also be able to cancel a request (if it is in the server's queue.). When the server application is finished with the hardware it shall be able to send the result back to the client that requested the job.
The server and client applications may or may not be on the same PC. All development is done in .NET (C#) 2005.
What is the best way to solve this communication problem?
MSMQ? SOAP? WCF? Remoting? Other?

Assuming you can use .NET 3.0 or greater then you probably want to WCF as the communications channel - the interface is consistent but it will allow you to use an appropriate transport mechanism depending on where the client and server are in relation to each other - so you can choose to use SOAP or MSMQ or a binary format or others as appropriate (and can roll your own if needed). It also covers the need for two way communication.
Queuing the messages at the server should probably be regarded as a separate problem - especially given the need to remove queued messages.

If clients and server processes are on the same machine, I think named pipes will give you the fastest raw byte transfer rate.
If the processes are across different machines, you'd need to use sockets-based approach.
Remoting is reportedly very slow. Based on the target OSes that you're planning to deploy the solution on, you could have options like WCF et al. However, the overhead of these protocols is something you may want to look at while deciding.

Remoting
If all development is done in .NET 2005, Remoting is the best way to go.

MSMQ would make some sense, though there are then security and deployment considerations. You could look at a service bus (such s NServiceBus or MassTransit) and there's also SQL Server Service Broker that could help (and can also be used by a service bus as the transport).
WCF would be another thing to look at, however that's really the across-network transport, so you'd probably still want the WCF calls to put a message on the server queue.
I don't recommend remoting, because it's hard to maintain a separation of concerns, and before you know it you're developing a really chatty interface without realising it. Remote calls are expensive in relative terms, so you should be trying to keep the messages fairly coarse-grained. WCF would be my recommendation. Not least because you can set it up to use a HTTP transport and avoid a lot of deployment and security headache.

The .NET Framework provides several ways to communicate with objects in different application domains, each designed with a particular level of expertise and flexibility in mind. For example, the growth of the Internet has made XML Web services an attractive method of communication, because XML Web services are built on the common infrastructure of the HTTP protocol and SOAP formatting, which uses XML. These are public standards, and can be used immediately with current Web infrastructures without worrying about additional proxy or firewall issues.
Not all applications should be built using some form of XML Web service, however, if only because of the performance issues related to using SOAP serialization over an HTTP connection.
Choosing Communication Options in .NET helps you decide which form of interobject communication you want for your application.

Communication between server and client for WinForms

I have 50+ kiosk style computers that I want to be able to get a status update, from a single computer, on demand as opposed to an interval. These computers are on a LAN in respect to the computer requesting the status.
I researched WCF however it looks like I'll need IIS installed and I would rather not install IIS on 50+ Windows XP boxes -- so I think that eliminates using a webservice unless it's possible to have a WinForm host a webservice?
I also researched using System.Net.Sockets and even got a barely functional prototype going however I feel I'm not skilled enough to make it a solid and reliable system. Given this path, I would need to learn more about socket programming and threading.
These boxes are running .NET 3.5 SP1, so I have complete flexibility in the .NET version however I'd like to stick to C#.
What is the best way to implement this? Should I just bite the bullet and learn Sockets more or does .NET have a better way of handling this?
edit:
I was going to go with a two way communication until I realized that all I needed was a one way communication.
edit 2:
I was avoiding the traditional server/client and going with an inverse because I wanted to avoid consuming too much bandwidth and wasn't sure what kind of overhead I was talking about. I was also hoping to have more control of the individual kiosks. After looking at it, I think I can still have that with WCF and connect by IP (which I wasn't aware I could connect by IP, I was thinking I would have to add 50 webservices or something).

WCF does not have to be hosted within IIS, it can be hosted within your Winform, as a console application or as windows service.
You can have each computer host its service within the winform, and write a program in your own computer to call each computer's service to get the status information.
Another way of doing it is to host one service in your own computer, and make the 50+ computers to call the service once their status were updated, you can use a database for the service to persist the status data of each node within the network. This option is easier to maintain and scalable.
P.S.
WCF aims to replace .net remoting, the alternatives can be net.tcp binding or net.pipe

Unless you have plans to scale this to several thousand clients I don't think WCF performance will even be a fringe issue. You can easily host WCF services from windows services or Winforms applications, and you'll find getting something working with WCF will be fairly simple once you get the key concepts.
I've deployed something similar with around 100-150 clients with great success.
There's plenty of resources out on the web to get you started - here's one to get you going:
http://msdn.microsoft.com/en-us/library/aa480190.aspx

Whether you use a web service or WCF on your central server, you only need to install and configure IIS on the server (and not on the 50+ clients).
What you're trying to do is a little unclear from the question, but if the clients need to call the server (to get a server status, for example), then they just call a method on the webservice running on the server.
If instead you need to have the server call the clients from time to time, then you'll need to have each client call a sign-in method on the server webservice each time the client starts up. The sign-in method would take a delegate method from the client as a parameter. The server would then call this delegate when it needed information from the client.
Setting up each client with its own web service would represent an inversion of the traditional (one server, multiple clients) client/server architecture, and as you've already noted this would be impractical.

Do not use remoting.
If you want robustness and scalability you end up ruling out everything but what are essentially stateless remote procedure calls. Since this is exactly the capability of web services, and web services are simpler and easier to build, remoting is an essentially pointless technology.
Callbacks with remote delegates are on the performance/reliability forbidden list, so if you were thinking of using remoting for that, think again.
Use web services.
I know you don't want to be polling, but I don't think you need to. Since you say all your units are on a single network segment then I suggest UDP for broadcast change notifications, essentially setting a dirty flag, and allowing the application to (re-)fetch on demand. It's still not reliable but it's easy and very fast because it's broadcast.
As others have said you don't need IIS, you can self-host. See ServiceHost class for details on how to do this.

I'd suggest using .NET Remoting. It's quite easy to implement and doesn't require anything else.

For me its is better to learn networking.. or the manual way of socket communication.. web services are mush slower because it contains metadata..
your clients and the servers can transform to multithreaded application. just imitate the request and response architecture. it is much easy to implement a network application like this..

If you just need a status update, you can use much simpler solution, such as simple tcp server/client messaging or like orrsella said, remoting. WCF is kinda overkill here.
One note though, if all your 50+ kiosk is connected via internet, then you might need use VPN or have an open port on each kiosk(which is a security risk) so that your server can retrieve status update from each kiosk.
We had a similiar situation, but the status is send to our server periodically, so we only have 1 port to protect/secure. The frequency of the update is configurable as to accomodate slower clients.

As someone who implemented something like this with over 500+ clients and growing:
Message Queing is the way to go.
We have gone from an internal developed TCP server and client to WCF polling and ended up with Message queing. It's the only guaranteed way to get data to and from clients and servers over the internet. As a bonus, many of these solutions have an extensive framework makeing it trivial to implement publish-subscribe, Send-one-way, point-to-point sending, Request-reply. Some of these are possible with WCF but it will involve crying, shouting, whimpering and long nights not to mention gallons of coffee.
A couple of important remarks:
Letting a process poll the clients instead of the other way around = Bad idea.. it is not scalable at all and you will soon be running in to trouble when the process is take too long to complete.. Not to mention having to handle all the ip addresses ( do you have access to all clients on the required ports ? What happpens when the ip changes etc..)
what we have done: The clients sends status updates to a central message queue on a regular interval ( you can easily implement live updates in the UI), it also listens on it's own queue for a GetStatusRequest message. if it receives this, it answers ( has a timeout).. this way, we can see overal status of all clients at all times and get a specific status of a specific client when needed.
Concerning bandwidth: kiosk usually show images/video etc.. 1Kb or less status messages will not be the big overhead.
I CANNOT stress enough that the current design you present will have a very intensive development cycle AND will not scale or extend well ( trust me, we have learned this lesson). Next to this, building a good client/server protocol for this type of stuff is a hard job that will be totally useless afterwards if you make a design error ( migrating a protocol is not easy)
We have built our solution ontop of ActiveMQ ( using NMS library c#) and are currently extending Simple Service Bus for our internal workings.
We only use WCF for the communication between our winforms app and the centralized service(s)