I'm facing an issue in a High Available Windows Services I developed with a master/slave setup.
Context:
The services itself synchronize data to two endpoints. One endpoint is synced to a local database, and one is an external. The database that is local is duplicated on both machines, so both master and slave need to sync this. The external endpoint only needs to be synced once.
The Master will by default sync to the external service, the slave will take over when the master is down. When the master goes back up and the slave is still synchronizing to the external, master will ask slave to finish a portion of the work, and then tell the master is done so he can continue the remaining work.
All this needs to happen asynchronously, I do not want the program to stop and wait for the other to respond (like the slave still handling the data).
I already implemented all the logic for this.
Setup:
Two Windows services running on two different machines.
Currently the communication is done over Named Pipes.
The problem:
Named pipes isn't reliable enough for the throughput that is being done. It also often crashes, and isn't made for reconnecting/closing and reopening a lot of times. I also face the problem that it just 'hangs' a lot when sending/receiving messages. Retrying sometimes works but I think I shouldn't be retrying. I need to have a reliable communication between the two instances.
Solutions:
I've been looking for an alternative to Named Pipes, but can't seem to find a solution of which I'm convinced that would work. Mostly because a lot of the technologies are for communication between a service and a client over http.
WCF over MSMQ is also not what I need, because I only want communication to happen when both are online. WCF in general is also more focused on one endpoint receiving data and sending a response. I need bidirectional communication, so both instances need to be able to receive and send messages at any time.
I think my best option is SignalR, but I'm also not convinced.
Have you looked at MassTransit over RabbitMQ?
We have been using them together very successfully both for intra-service and client/service communication for a few years now.
Related
I am trying to use web sockets to allow two Windows services on different machines to pass data back and forth. Almost all the examples or information I have found are about using web sockets for Client/Server Side communication. I am having trouble figuring out how to set this up. I have considered using WebSocketHost as apart of Microsoft.ServiceModel.WebSockets, but then I am unsure how to bind it to a local port and not a URL.
Does any one have any suggestions
Thanks
I am trying to use web sockets to allow two Windows services on different machines to pass data back and forth.
You can open sockets on both machines using WebSockets as you found. The examples mention clients and servers because this is the typical usage, however the API really doesn't care. As long as each side has a listener and a sender they can communicate.
However I would like to mention that this isn't as simple as it sounds because both machines aren't always available. Sometimes one or the other is busy or the network is blocked or something else is going on, or the listener is too busy to respond right away, so you're going to end up needing some sort of queuing on both sides.
If you're doing a process based operation where one side tells the other "I want X" and it's a big operation like producing a document, I've found it much more resilient to build a queue in a database and toss the request in there, then wait for the other side to update the record to say it's done.
If they're smaller, faster requests, MSMQ would be more appropriate if you have it available.
However back to your original question, if you want to use it, any of the client-server examples should work just fine. The API doesn't care.
You can use SignalR Self-Host you really don't want to create your own WebSockets framework since this this will take a long time.
Here is a link on how to start a OWIN server in Windows services.
Hosting WebAPI using OWIN in a windows service
And how to set signalR in self host
Tutorial: SignalR Self-Host
You can accomplish this with Memory Mapped Files.
Inter-Process Communication with Memory-Mapped Files
I am trying to create a monitoring application for our operations department to be proactive when dealing with systems that are encountering problems. I created an app that does the job but it has some draw backs:
Each copy of the app running serves individual pings to the systems, when 1 ping would suffice.
I have 3 different api's for getting the status of our systems depending whether its hosted IIS, WCF or desktop.
To fix the first issue i was going to create a database which an interim service(app)(monitor) would make the pings, then the app would query the database for updates. After thinking about this I realized the second issue and decided it is a future problem.
So my thought was to, rather than have the interim application pinging the systems, simply have each system have one interface in which it posts it status to the database every x time. But then I ran into a problem with the WCF and IIS services we have. These services can sit for days without anyone actually using the service. How would I make these services continue to post its data?
My questions are:
Is it better to have data REQUESTED or PUSHED in this type of situation?
If REQUESTED, what is a suggested practice for maintaining a single API across mulitple platforms(IIS, WCF, Desktop)?
If PUSHED, how would you handle the case of the Web services which are instance based and not continuously running?
For web services, one solution might be to implement a health-check end point , something that you can simply call like: webservice/isServiceUp?
I prefer that this information is PULLED. If a service / web Service/ Application is down, then you can't possibly rely on it to write something to the DB... it would be possible but highly risky and unreliable.
In a real world situation, it is a little more complicated than that because something might happen between your service host and the consumer (DNS problem for example), in which case, you would want to consider the case of not getting anything back from the isServiceUp (no true no false, just a 400 lvl error)...
Consider using your load balancer for checking on APPS / web services and proactively switching to a different IP in case of issues... it is a possibility.
I had occasion to use TCPView to look at network connections while I was developing a .NET thick client that connects to a WCF service and noticed that for every request, a new port is opened and then sticks around in the TIME_WAIT state for a while.
Yes, I've used WCF for a while now, but I didn't know this. I read that it's normal (part of the way TCP works) but it seems kind of bizarre to me.
I'm worried that a user could fill up all the available ports if they keep clicking around the application really fast (causing lots of service requests, sometimes multiple calls per click). Is that ever a practical problem, or should I just not worry about it?
I have 50+ kiosk style computers that I want to be able to get a status update, from a single computer, on demand as opposed to an interval. These computers are on a LAN in respect to the computer requesting the status.
I researched WCF however it looks like I'll need IIS installed and I would rather not install IIS on 50+ Windows XP boxes -- so I think that eliminates using a webservice unless it's possible to have a WinForm host a webservice?
I also researched using System.Net.Sockets and even got a barely functional prototype going however I feel I'm not skilled enough to make it a solid and reliable system. Given this path, I would need to learn more about socket programming and threading.
These boxes are running .NET 3.5 SP1, so I have complete flexibility in the .NET version however I'd like to stick to C#.
What is the best way to implement this? Should I just bite the bullet and learn Sockets more or does .NET have a better way of handling this?
edit:
I was going to go with a two way communication until I realized that all I needed was a one way communication.
edit 2:
I was avoiding the traditional server/client and going with an inverse because I wanted to avoid consuming too much bandwidth and wasn't sure what kind of overhead I was talking about. I was also hoping to have more control of the individual kiosks. After looking at it, I think I can still have that with WCF and connect by IP (which I wasn't aware I could connect by IP, I was thinking I would have to add 50 webservices or something).
WCF does not have to be hosted within IIS, it can be hosted within your Winform, as a console application or as windows service.
You can have each computer host its service within the winform, and write a program in your own computer to call each computer's service to get the status information.
Another way of doing it is to host one service in your own computer, and make the 50+ computers to call the service once their status were updated, you can use a database for the service to persist the status data of each node within the network. This option is easier to maintain and scalable.
P.S.
WCF aims to replace .net remoting, the alternatives can be net.tcp binding or net.pipe
Unless you have plans to scale this to several thousand clients I don't think WCF performance will even be a fringe issue. You can easily host WCF services from windows services or Winforms applications, and you'll find getting something working with WCF will be fairly simple once you get the key concepts.
I've deployed something similar with around 100-150 clients with great success.
There's plenty of resources out on the web to get you started - here's one to get you going:
http://msdn.microsoft.com/en-us/library/aa480190.aspx
Whether you use a web service or WCF on your central server, you only need to install and configure IIS on the server (and not on the 50+ clients).
What you're trying to do is a little unclear from the question, but if the clients need to call the server (to get a server status, for example), then they just call a method on the webservice running on the server.
If instead you need to have the server call the clients from time to time, then you'll need to have each client call a sign-in method on the server webservice each time the client starts up. The sign-in method would take a delegate method from the client as a parameter. The server would then call this delegate when it needed information from the client.
Setting up each client with its own web service would represent an inversion of the traditional (one server, multiple clients) client/server architecture, and as you've already noted this would be impractical.
Do not use remoting.
If you want robustness and scalability you end up ruling out everything but what are essentially stateless remote procedure calls. Since this is exactly the capability of web services, and web services are simpler and easier to build, remoting is an essentially pointless technology.
Callbacks with remote delegates are on the performance/reliability forbidden list, so if you were thinking of using remoting for that, think again.
Use web services.
I know you don't want to be polling, but I don't think you need to. Since you say all your units are on a single network segment then I suggest UDP for broadcast change notifications, essentially setting a dirty flag, and allowing the application to (re-)fetch on demand. It's still not reliable but it's easy and very fast because it's broadcast.
As others have said you don't need IIS, you can self-host. See ServiceHost class for details on how to do this.
I'd suggest using .NET Remoting. It's quite easy to implement and doesn't require anything else.
For me its is better to learn networking.. or the manual way of socket communication.. web services are mush slower because it contains metadata..
your clients and the servers can transform to multithreaded application. just imitate the request and response architecture. it is much easy to implement a network application like this..
If you just need a status update, you can use much simpler solution, such as simple tcp server/client messaging or like orrsella said, remoting. WCF is kinda overkill here.
One note though, if all your 50+ kiosk is connected via internet, then you might need use VPN or have an open port on each kiosk(which is a security risk) so that your server can retrieve status update from each kiosk.
We had a similiar situation, but the status is send to our server periodically, so we only have 1 port to protect/secure. The frequency of the update is configurable as to accomodate slower clients.
As someone who implemented something like this with over 500+ clients and growing:
Message Queing is the way to go.
We have gone from an internal developed TCP server and client to WCF polling and ended up with Message queing. It's the only guaranteed way to get data to and from clients and servers over the internet. As a bonus, many of these solutions have an extensive framework makeing it trivial to implement publish-subscribe, Send-one-way, point-to-point sending, Request-reply. Some of these are possible with WCF but it will involve crying, shouting, whimpering and long nights not to mention gallons of coffee.
A couple of important remarks:
Letting a process poll the clients instead of the other way around = Bad idea.. it is not scalable at all and you will soon be running in to trouble when the process is take too long to complete.. Not to mention having to handle all the ip addresses ( do you have access to all clients on the required ports ? What happpens when the ip changes etc..)
what we have done: The clients sends status updates to a central message queue on a regular interval ( you can easily implement live updates in the UI), it also listens on it's own queue for a GetStatusRequest message. if it receives this, it answers ( has a timeout).. this way, we can see overal status of all clients at all times and get a specific status of a specific client when needed.
Concerning bandwidth: kiosk usually show images/video etc.. 1Kb or less status messages will not be the big overhead.
I CANNOT stress enough that the current design you present will have a very intensive development cycle AND will not scale or extend well ( trust me, we have learned this lesson). Next to this, building a good client/server protocol for this type of stuff is a hard job that will be totally useless afterwards if you make a design error ( migrating a protocol is not easy)
We have built our solution ontop of ActiveMQ ( using NMS library c#) and are currently extending Simple Service Bus for our internal workings.
We only use WCF for the communication between our winforms app and the centralized service(s)
I hope someone can guide me as I'm stuck... I need to write an emergency broadcast system that notifies workstations of an emergency and pops up a little message at the bottom of the user's screen. This seems simple enough but there are about 4000 workstations over multiple subnets. The system needs to be almost realtime, lightweight and easy to deploy as a windows service.
The problem started when I discovered that the routers do not forward UDP broadcast packets x.x.x.255. Later I made a simple test hook in VB6 to catch net send messages but even those didn't pass the routers. I also wrote a simple packet sniffer to filter packets only to find that the network packets never reached the intended destination.
Then I took a look and explored using MSMQ over HTTP, but this required IIS to be installed on the target workstation. Since there are so many workstations it would be a major security concern.
Right now I've finished a web service with asynchronous callback that sends an event to subscribers. It works perfectly on a small scale but once there are more than 15 subscribers performance degrades considerably. Polling a server isn't really an option because of the load it will generate on the server (plus I've tried it too)
I need your help to guide me as to what technology to use. has anyone used the comet way with so many clients or should I look at WCF?
I'm using Visual C# 2005. Please help me out of this predicament.
Thanks
Consider using WCF callbacks mechanism and events. There is good introduction by Juval Lowy.
Another pattern is to implement blocking web-service calls. This is how GMail chat works, for example. However, you will have to deal with sessions and timeouts here. It works when clients are behind NATs and Firewalls and not reachable directly. But it may be too complicated for simple alert within intranet.
This is exactly what Multicast was designed for.
A normal network broadcast (by definition) stays on the local subnet, and will not be forwarded through routers.
Multicast transmissions on the other hand can have various scopes, ranging from subnet local, through site local, even to global. All you need is for the various routers connecting your subnets together to be multicast aware.
This problem i think is best solved with socket.
Open a connection to the server, and keep it open.
Could you have a slave server in each subnet that was responsible for distributing the messages to all the clients in the subnet?
Then you could have just the slaves attached to the central server where the messages are initiated.
I think some of you are vastly overthinking this. There is already a service built into every version of Windows that provides this exact functionality! It is called the Messenger service. All you have to do is ensure that this service is enabled and running on all clients.
(Although you didn't specify in the question, I'm assuming from your choices of technology that the client population of this network is all Windows).
You can send messages using this facility from the command line using something like this:
NET SEND computername "This is a test message"
The NET SEND command also has options to send by Windows domain, or to specific users by name regardless of where they are logged in, or to every system that is connected to a particular Windows server. Those options should let you easily avoid the subnet issue, particularly if you use domain-based security on your network. (You may need the "Alerter" service enabled on certain servers if you are sending messages through the server and not directly to the clients).
The programmatic version of this is an API called NetMessageBufferSend() which is pretty straightforward. A quick scan of P/Invoke.net finds a page for this API that supplies not only the definitions you need to call out to the API, but also a C# sample program!
You shouldn't need to write any client-side code at all. Probably the most involved thing will be figuring out the best set of calls to this API that will get complete coverage of the network in your configuration.
ETA: I just noticed that the Messenger service and this API are completely gone in Windows Vista. Very odd of Microsoft to completely remove functionality like this. It appears that this vendor has a compatible replacement for Vista.