I hope someone can guide me as I'm stuck... I need to write an emergency broadcast system that notifies workstations of an emergency and pops up a little message at the bottom of the user's screen. This seems simple enough but there are about 4000 workstations over multiple subnets. The system needs to be almost realtime, lightweight and easy to deploy as a windows service.
The problem started when I discovered that the routers do not forward UDP broadcast packets x.x.x.255. Later I made a simple test hook in VB6 to catch net send messages but even those didn't pass the routers. I also wrote a simple packet sniffer to filter packets only to find that the network packets never reached the intended destination.
Then I took a look and explored using MSMQ over HTTP, but this required IIS to be installed on the target workstation. Since there are so many workstations it would be a major security concern.
Right now I've finished a web service with asynchronous callback that sends an event to subscribers. It works perfectly on a small scale but once there are more than 15 subscribers performance degrades considerably. Polling a server isn't really an option because of the load it will generate on the server (plus I've tried it too)
I need your help to guide me as to what technology to use. has anyone used the comet way with so many clients or should I look at WCF?
I'm using Visual C# 2005. Please help me out of this predicament.
Thanks
Consider using WCF callbacks mechanism and events. There is good introduction by Juval Lowy.
Another pattern is to implement blocking web-service calls. This is how GMail chat works, for example. However, you will have to deal with sessions and timeouts here. It works when clients are behind NATs and Firewalls and not reachable directly. But it may be too complicated for simple alert within intranet.
This is exactly what Multicast was designed for.
A normal network broadcast (by definition) stays on the local subnet, and will not be forwarded through routers.
Multicast transmissions on the other hand can have various scopes, ranging from subnet local, through site local, even to global. All you need is for the various routers connecting your subnets together to be multicast aware.
This problem i think is best solved with socket.
Open a connection to the server, and keep it open.
Could you have a slave server in each subnet that was responsible for distributing the messages to all the clients in the subnet?
Then you could have just the slaves attached to the central server where the messages are initiated.
I think some of you are vastly overthinking this. There is already a service built into every version of Windows that provides this exact functionality! It is called the Messenger service. All you have to do is ensure that this service is enabled and running on all clients.
(Although you didn't specify in the question, I'm assuming from your choices of technology that the client population of this network is all Windows).
You can send messages using this facility from the command line using something like this:
NET SEND computername "This is a test message"
The NET SEND command also has options to send by Windows domain, or to specific users by name regardless of where they are logged in, or to every system that is connected to a particular Windows server. Those options should let you easily avoid the subnet issue, particularly if you use domain-based security on your network. (You may need the "Alerter" service enabled on certain servers if you are sending messages through the server and not directly to the clients).
The programmatic version of this is an API called NetMessageBufferSend() which is pretty straightforward. A quick scan of P/Invoke.net finds a page for this API that supplies not only the definitions you need to call out to the API, but also a C# sample program!
You shouldn't need to write any client-side code at all. Probably the most involved thing will be figuring out the best set of calls to this API that will get complete coverage of the network in your configuration.
ETA: I just noticed that the Messenger service and this API are completely gone in Windows Vista. Very odd of Microsoft to completely remove functionality like this. It appears that this vendor has a compatible replacement for Vista.
Related
I am in the need of creating a C# program that will run on couple of our local Windows client machines. These 'client' programs will have to take commands from a 'admin' program run on another machine.
The commands could be to reboot the client computers, return some local information about IP address etc back to the 'admin' program.
But how to accomplish this? I know a little about WCF but is that the right way to go?
If I go with WCF I will then have to make the client programs run a service method, like every second, to check for new commands. With sockets I establish a 'direct' connection and the client just waits for a command to receive - isn't that correct understood?
Which way would be the right way for me to go?
We are talking about ~10 clients and I want a maximum delay (send command - receive info back) of 1 second.
Any hints would also be appreciated.
Best regards
Duplex WCF server. Basically, the clients all connect into the server (so only 1 server), and the server uses its duplex channel to call back to the clients whenever it needs to. No polling, scales well, etc. The most headache you'll need to deal with is to set a long timeout in case that you don't send anything for a while so that the channels time out.
WCF will end up being much simpler in the end.
A couple of links:
http://msdn.microsoft.com/en-us/library/ms731064.aspx
and
http://www.codeproject.com/Articles/491844/A-Beginners-Guide-to-Duplex-WCF
I hope those help.
You can make WCF clients act as servers and with command & control program connect to them that is no problem. Go for WCF if you don't want to mess with ugly stuff that sockets can bring. WCF can be configured nicely in app.config, and you can make it really self hosted command line application even so no need for IIS server. Configuration will be then resuable and easier to maintain.
You could use .NET Remoting which can provide a "push" backchannel from the server to the client (using "callbacks"). It does not need a 2nd TCP connection in the other direction so you don't need to mess with client's firewalls and routers.
Remoting is considered kind of obsolete but it has its places.
In any case I would not use a WCF polling technique. That leads to bad latency and a DDOS situation for the server.
If you can make the clients open a port then hosting a WCF service there is probably the best idea.
I would recommend a socket implementation as in the long run it probably gives you greater flexibility. You could create this from scratch yourself using the socket namespace. As an alternative you could use an off the shelf network library solution. Checkout lidgren and NetworkComms.Net.
Disclaimer: I'm a developer for NetworkComms.Net.
With the small number of machines and modest performance requirements you mentioned, I think WCF would end up being easier than sockets.
You might look into Duplex WCF. I've never used it, and WCF has given me headaches in the past any time I've needed anything unusual, but it's for the sort of problem you're talking about.
If all the machines are on one network, here's one creative alternative loosely inspired by message queues: you could use a database table as a place where messages appear and get read by clients at their leisure. The clients could just query it and say: get me all messages where MessageID > LastReceivedMessageID.
The downsides of that last approach is that (a) you're still doing polling although your database server should be able to handle it and (b) if you might ever need this outside of your network, you would need a VPN or a new solution.
you could use MSMQ... couldn't be easier to implement
http://msdn.microsoft.com/en-us/library/windows/desktop/ms711472(v=vs.85).aspx
I use MSMQ for a number of similar applications. Works perfectly.
I am planning a SaaS system, to be written in C#, ASP.NET using WCF that has two separate components:
On a static IP web server in the cloud will be a web app, common to all clients.
Inside each client's office will be another app, installed on a server with IIS.
The site app will obviously be able to connect to the web services published on the web site. But here's the rub - I also want the web app to be able to initiate a connection to the site app... and the on-site server may not necessarily have a static IP. I can't control this, because we may have hundreds of clients at some point in the future, and we cannot limit our saleability by insisting that the customer has a server with fixed IP.
So, how to do this?
I could have the site apps "checking in" with the web every minute or so, to give the web app the possibility of responding with a "while you're here, please do x,y,z..." but that seems very inelegant. Also, if we're talking about hundreds of clients, I don't want to be bombarding my web server with all these "hi there!" messages if they're not actually required.
Is there a better way?
WCF? Here we go:
Use a message based approach (exchange message, no stateful method calls).
Clients connect to the server. Establish a HTTP-based TWO WAY CONNECTION. This way the server can call back to connected clients. This is standard WCF stuff and works well through NAT with version 4 of the .NET framework.
Voila. In case of a disconnect the client can re-connect, re-identify himself and gets the pending messages.
IIRC "push communication" is done by letting the client do a HTTP Request with an indefinate timeout. Then the server responds when he has something to say. After the respons the client immediately makes a new request.
It works out the same way like the server is making the connection and takes far less resources than polling.
Dynamic DNS is one possibility, but depends on your clients/customers.
If the site app is created by you, it only has to contact the web server when its address has changed (or when the site server/web app is restarted). Still, a keep-alive heart beat of, say, every 30 min. to 1 hour isn't a bad idea.
Edit: I think SNMP services may provide the answer but I'm not a networking expert. You'll have to do some digging or ask a separate question on stackoverflow.
What would you say about Comet technology?
Sounds like you'll definitely need some sort of registry on the server, then it could attempt to call out to the client apps if it needs work doing.
Generally it is client apps that check in with the server every X seconds - this is how Selenium grid works anyway. With a central hub with which clients register. When the hub receives a request to run some tests it passes the jobs out to the clients to perform.
You may not need the "checking in". The server could just attempt to call out to a registered client app until it finds one that is available.This way only the server would need a static address (could use a DNS name instead of an IP to make it more robust).
Also have a look at XMPP PubSub. This could be a more robust and standardised way to handle this.
In the end I decided to go with NetTcpBinding, for reasons best given by #Allon Guralnek here. It's worth clicking through and reading what he has to say...
I have a .Net service hosted on a server, and .Net clients connecting to this server over the internet.
I want to implement a publish subscribe model where clients can subscribe to events on the service and have data pushed to them as data becomes available. An alternative would be to have clients poll the server for data, however this will likely be too slow for what is required. Hence the need for a publish/subscribe type of communication.
I understand that the WCF binding WSDualHttpBinding does allow this, however it comes with a downside. According to "Programming WCF Services" author Juval Lowy,
...the WSDualHttpBinding is mostly
unusable, since it is practically
impossible to tunnel through various
communication barriers separating the
service from the client and the need
to find a specific web-server machine
makes this impractical.
I've interpretted this to mean (please correct me if I'm wrong) that to operate with WSDualHttpBinding, it is necessary for clients to open a port on their machines (along with any necessary router configuration) for the server to callback through. If this is the case using WSDualHttpBinding will not be an option for me. Using Windows Azure will also not be an option.
So the crux of my question is, how can I achieve a publish/subscribe/callback type of communication over the internet, without the need to open ports on the client machine? Open standards are ok but unnecessary as client and server are both .Net, Windows Azure is not an option.
WSDualHttpBinding contains two channels one from client to server and second from server to client. The later one indeed requires firewall and NAT configuration. What about Net.Tcp? It uses only single channel and supports callbacks (duplex communication) over that channel initiated from client to server. You mentioned that both client and server will be .NET application so it should be possible with some firewall configuration on the server.
You mention most of the options in your post.
There are 3 options:
Client polls the server. Does not require open ports but too slow.
WSDualHttpBinding requires opening of ports
Azure Service Bus, would do it but not an option.
There is actually a way to do it. If you look at how Azure service bus works, it tricks the client into thinking that it is on an out going port, when really it being used to send messages to the client. You could try to implement this functionality.
Another thing you could try is nservicebus at http://nservicebus.com/, this is an open source .net service bus.
Internet Communication Engine (ICE) offers IceStorm, a publish/subscribe service.
Its open source, and if you download the installation there is a sample Visual Studio project that demonstrates how to implement publish/subscribe (check out the "demos" .zip file and the "IceStorm" directory with the clock demo).
ICE will do all the heavy lifting for you, the learning curve is remarkably short, primarily because the documentation is massive, approachable and well written.
I strongly recommend you DDS (data distribution service from OMG) over the internet http://www.omg.org/news/meetings/workshops/Real-time_WS_Final_Presentations_2008/Session%203/03-02_Bertocci_et_al.pdf
From OMG, that's all I gotta say. Yes, I know you may think OMG is over. I don't, and as a government counselor I really push for standards. Keep in mind that besides the liberal ideology and crises, governments are still a huge customers and inter-operation is a must.
NServiceBus? Right, ok, that's fine, but ain't too complex to start right now? Learning curve is too ... steep? I guess is not but, any easier one?
ICE is a good choice. guys from CORBA world trying to make things better. Don't doubt it, use it, try it! Just a feeling: even with the storm service you might feel you are still in the request/reply world ... but is that a con?
But if you rather prefer a commercial but open solution, think about publish subscribe using protocol buffers (search Google protocol buffers) ... just a first approach http://protocolbus.casessite.org . This is my owm work ... sorry it's just an initial project but I'm working with a central broker distribution as an alternative transport (default is multicast but broadcast and udp is also available). Open source, so be free ...
I would like to track messages sent and received though Windows Live Messenger. I would then like to collate these messages into a database (not in the scope of this question).
The question is how and where should I track these messages. The simplest way it to force all clients to keep history files and read those, but it is not really the solution that I am looking for. Is there a way to track them from a server running in the same domain, I have read a little into Windows Communicator, I have also seen a lot of people chat about http://dev.live.com/messenger/ but I was hoping that someone may have addressed this problem already :)
I would like to do this using C# .NET 3.5
Check out MSNPSharp. Its a .NET msn library. Its very powerful and allows you to sign in from multiple locations. So you can sign in and listen to other conversations happening on a given account.
Its very straight forward to use. Download the full source code, there's a sample application that demonstrates its use in full detail.
http://code.google.com/p/msnp-sharp/
Here is two idea that might work.
The first one is the easiest but can be easily avoided by the user if he doesn't want to be logged. It would be to use MSN Plus over the MSN. With MSN Plus you have an API that let you get all messages from any Chat Windows... and a lot more. Of course, if the user is not you, the user can simply uninstall Msn Plus and your program will not log any data.
The second idea is better if you have a network that you require to check all Msn Conversation. If you use WireShark you can see that conversation are not crypted (well the last time I did it) and you can check the port and protocol to simply get the data from the network.
Hope it gives you a way to what you need.
Just two ideas
1. First the standard MSN protocol is plain text (from what I understand) so you could intercept the messages on the firewall and then put them in the DB and do the correlation there.
2. If this is in an organisation you could use Office Communicator which is the "corporate" version of MSN and has that functionality built in already. You can then just go in via their SDK and get the correlated data.
I managed to find two ways of doing this, though both are not really programmatic solutions, so may not appeal to this audience.
Make use of a Jabber gateway to set up forwards between your jabber client and the other IM networks. Traffic flows between your jabber enabled client and the jabber server via the jabber server. The Jabber server then translates this to the destination networks protocol and forwards the message. Likewise messages from the external IM networks are routed and translated by the Jabber server. An example of this is PSI <-> IceWarp Merak <-> MSN
Make use of Symantec IM Manager to intercept messages from the messaging clients on your network. You will need to either use host files or local DNS rules to convince the your local PCs that Messenger.hotmail.com is actually located at 192.168.0.59 and not at Microsoft.
Hope it helps other people that may want to do the same.
I have 50+ kiosk style computers that I want to be able to get a status update, from a single computer, on demand as opposed to an interval. These computers are on a LAN in respect to the computer requesting the status.
I researched WCF however it looks like I'll need IIS installed and I would rather not install IIS on 50+ Windows XP boxes -- so I think that eliminates using a webservice unless it's possible to have a WinForm host a webservice?
I also researched using System.Net.Sockets and even got a barely functional prototype going however I feel I'm not skilled enough to make it a solid and reliable system. Given this path, I would need to learn more about socket programming and threading.
These boxes are running .NET 3.5 SP1, so I have complete flexibility in the .NET version however I'd like to stick to C#.
What is the best way to implement this? Should I just bite the bullet and learn Sockets more or does .NET have a better way of handling this?
edit:
I was going to go with a two way communication until I realized that all I needed was a one way communication.
edit 2:
I was avoiding the traditional server/client and going with an inverse because I wanted to avoid consuming too much bandwidth and wasn't sure what kind of overhead I was talking about. I was also hoping to have more control of the individual kiosks. After looking at it, I think I can still have that with WCF and connect by IP (which I wasn't aware I could connect by IP, I was thinking I would have to add 50 webservices or something).
WCF does not have to be hosted within IIS, it can be hosted within your Winform, as a console application or as windows service.
You can have each computer host its service within the winform, and write a program in your own computer to call each computer's service to get the status information.
Another way of doing it is to host one service in your own computer, and make the 50+ computers to call the service once their status were updated, you can use a database for the service to persist the status data of each node within the network. This option is easier to maintain and scalable.
P.S.
WCF aims to replace .net remoting, the alternatives can be net.tcp binding or net.pipe
Unless you have plans to scale this to several thousand clients I don't think WCF performance will even be a fringe issue. You can easily host WCF services from windows services or Winforms applications, and you'll find getting something working with WCF will be fairly simple once you get the key concepts.
I've deployed something similar with around 100-150 clients with great success.
There's plenty of resources out on the web to get you started - here's one to get you going:
http://msdn.microsoft.com/en-us/library/aa480190.aspx
Whether you use a web service or WCF on your central server, you only need to install and configure IIS on the server (and not on the 50+ clients).
What you're trying to do is a little unclear from the question, but if the clients need to call the server (to get a server status, for example), then they just call a method on the webservice running on the server.
If instead you need to have the server call the clients from time to time, then you'll need to have each client call a sign-in method on the server webservice each time the client starts up. The sign-in method would take a delegate method from the client as a parameter. The server would then call this delegate when it needed information from the client.
Setting up each client with its own web service would represent an inversion of the traditional (one server, multiple clients) client/server architecture, and as you've already noted this would be impractical.
Do not use remoting.
If you want robustness and scalability you end up ruling out everything but what are essentially stateless remote procedure calls. Since this is exactly the capability of web services, and web services are simpler and easier to build, remoting is an essentially pointless technology.
Callbacks with remote delegates are on the performance/reliability forbidden list, so if you were thinking of using remoting for that, think again.
Use web services.
I know you don't want to be polling, but I don't think you need to. Since you say all your units are on a single network segment then I suggest UDP for broadcast change notifications, essentially setting a dirty flag, and allowing the application to (re-)fetch on demand. It's still not reliable but it's easy and very fast because it's broadcast.
As others have said you don't need IIS, you can self-host. See ServiceHost class for details on how to do this.
I'd suggest using .NET Remoting. It's quite easy to implement and doesn't require anything else.
For me its is better to learn networking.. or the manual way of socket communication.. web services are mush slower because it contains metadata..
your clients and the servers can transform to multithreaded application. just imitate the request and response architecture. it is much easy to implement a network application like this..
If you just need a status update, you can use much simpler solution, such as simple tcp server/client messaging or like orrsella said, remoting. WCF is kinda overkill here.
One note though, if all your 50+ kiosk is connected via internet, then you might need use VPN or have an open port on each kiosk(which is a security risk) so that your server can retrieve status update from each kiosk.
We had a similiar situation, but the status is send to our server periodically, so we only have 1 port to protect/secure. The frequency of the update is configurable as to accomodate slower clients.
As someone who implemented something like this with over 500+ clients and growing:
Message Queing is the way to go.
We have gone from an internal developed TCP server and client to WCF polling and ended up with Message queing. It's the only guaranteed way to get data to and from clients and servers over the internet. As a bonus, many of these solutions have an extensive framework makeing it trivial to implement publish-subscribe, Send-one-way, point-to-point sending, Request-reply. Some of these are possible with WCF but it will involve crying, shouting, whimpering and long nights not to mention gallons of coffee.
A couple of important remarks:
Letting a process poll the clients instead of the other way around = Bad idea.. it is not scalable at all and you will soon be running in to trouble when the process is take too long to complete.. Not to mention having to handle all the ip addresses ( do you have access to all clients on the required ports ? What happpens when the ip changes etc..)
what we have done: The clients sends status updates to a central message queue on a regular interval ( you can easily implement live updates in the UI), it also listens on it's own queue for a GetStatusRequest message. if it receives this, it answers ( has a timeout).. this way, we can see overal status of all clients at all times and get a specific status of a specific client when needed.
Concerning bandwidth: kiosk usually show images/video etc.. 1Kb or less status messages will not be the big overhead.
I CANNOT stress enough that the current design you present will have a very intensive development cycle AND will not scale or extend well ( trust me, we have learned this lesson). Next to this, building a good client/server protocol for this type of stuff is a hard job that will be totally useless afterwards if you make a design error ( migrating a protocol is not easy)
We have built our solution ontop of ActiveMQ ( using NMS library c#) and are currently extending Simple Service Bus for our internal workings.
We only use WCF for the communication between our winforms app and the centralized service(s)