I am working on a web application in C#, ASP.NET, and .NET framework 4.5 with the use of WebSockets. In order to plan for scalability in the future, the application pool has the option for web gardens enabled to simulate multiple web servers on my single development machine.
The issue I am having is how to handle re-connects on the websocket side. When a new websocket session is initially created, the client browser can indirectly lock records in a SQL database. But when the connection is lost, my boss would like the browser to attempt to re-connect to the same instance of the websocket server session so it doesn't need to re-lock anything.
I don't know if something like this is possible because on re-connect the load balancer will "randomly" select which web server to handle the new connection. I was thinking of some hack to work around this but it isn't very clean:
Client opens initial websocket connection on Server A and locks a record.
Client temporarily loses internet connection and the websocket closes. (It is important to note that the server side will wait up to 60 seconds before it "disposes" itself; therefore, the SQL record will remain locked until the 60 seconds has elapsed).
Client internet connection is restored and reconnects to the website but this time on Server B.
Server B sees that this context was initially connected on Server A; therefore, transfers the session to Server A.
Server A checks the process id to see if it is running in the correct worker process (in the case of a web garden).
Server A has found the initial instance and handles the connection.
I tried Googling this question but it doesn't seem like a very common issue because I don't think most websocket web apps keep records locked for as long that my applications does (which is could be up to an hour).
Thanks in advance for all of your help!
Update 3/15/2016
I was hoping that the Server.TransferRequest would have been helpful however it doesn't seem to work for web sockets. Would anyone know of a way to best transfer a websocket context from one process to another?
First, you might want to re-examine why you're locking records for a long time and requiring a client to come back to the same server every time. That is not the usual type of high scale web architecture and perhaps you're just creating this need to reconnect to the identical server because of that requirement when maybe you should rethink how that is designed so that your application would work just fine no matter which host a user connects to.
That would certainly simplify scaling to large numbers of users and servers if you could remove that requirement. You can always then implement local caching and semi-sticky connections later as a performance enhancement, but only after you release the requirement to 100% of the time connect to the same host.
If you're going to stick with that requirement to always connect to the same host, then you will ultimately need some sort of sticky load balancing. There are a lot of different schemes. Some are driven by the networking infrastructure in front of your server, some are driven by your server and some are even client driven. They all have different tradeoffs. Here's a brief run-down of some of the schemes:
Hardware, networking load balancer. Here you have a fairly transparent mechanism by which a hardware load balancer (which is really just software running on a custom piece of hardware) sits in front of your web server farm and uses various techniques to make sure whatever server a given user is originally connected to it will get reconnected to on subsequent connections. This can be based on various schemes (IP address, cookie value, etc...) as the key to identifying a particular user and it typically has a number of possible configurations for how it can work.
Proxy load balancer. This is essentially an all software version of the hardware load balancer. Here a proxy sits in front of your server farm and directs connections to a particular server based on some algorithm (IP address, cookie value, etc...).
Server Redirect. Here an incoming connection is randomly assigned to a server. Upon connection the server figures out where the connection is supposed to be connected to an returns a 302 redirect to the actual host causing the client to reconnect to the proper server. This involves one less layer of infrastructure (no physical load balancers), but exposes the different server endpoints to the outside world which the first two options do not.
Client Selection Algorithm. Here the client is given knowledge of the various server endpoints and is coded with an algorithm for consistently selecting one for this user. It could be a hash of a userID that is then divided into the server bucket pool and the end result is that client ends up choosing a particular DNS name such as cl003.myserver.com which it then connects to. This choice requires the least work server-side so can be simpler to implement, but it requires changing the client code in order to modify the algorithm.
For an article on sticky load balancing for Amazon Web Services to give you an idea on how one mechanism works, you can read this: Elastic Load Balancing: Configure Sticky Sessions for Your Load Balancer.
Here's another article on how the nginx proxy is configured for sticky load balancing.
You can find lots of other articles with a Google search for "sticky load balancing".
A discussion of the pros/cons of the various schemes is the subject of a much longer discussion and some of it involves knowledge of more specific requirements and specific capabilities of your infrastructure.
Related
For a chat application, I use Azure architecture with SignalR, with the web-role acting as SignalR server (the messages are not broadcast type but are intended for specific user/client).
I want to scale out SignalR server along with the web-roles, to handle heavy user load. Although, SignalR documentation doesn't recommend to use the pre-baked SignalR scale out methods using backplane (Redis, Service bus) for such cases when the number of messages increase as more users are connected (or in user-event driven scenario). It explicitly states: "Client-to-client (e.g., chat): In this scenario, the backplane might be a bottleneck if the number of messages scales with the number of clients; that is, if the rate of messages grows proportionally as more clients join."
Question:
Does anyone know of any custom scale-out solution for such high-frequency case, which doesn't push messages to each server instance or some other scale-out solution?
Already looked everywhere in SignalR documentation and the related videos but couldn't find anything, other than a word "filtered-bus", which was not explained what it is and how it should be used.
I figured it out myself: Basic idea is server affinity/sticky sessions.
Each instance of web-role acts as a stand-alone SignalR server. At the first connection time, I let the Azure load balancer choose any instance of web-role and save the IP address of that web-role instance with the client identifier in a map. If there is another connect request coming from the same client (e.g after page refresh) then I check the IP address of the current role instance and if it matches the entry in map then I let it proceed otherwise I disconnect the client and connects it to the correct instance of web-role.
Each instance of worker-role also acts as SignalR .net client and connects to all the available SignalR servers (all instances of web-role). Before sending a message to the SignalR server (web-role), I look up in the map to determine the correct SignalR server instance (depending on the intended JS recipient).
Benefits:
There is no need of back-plane technology (and hence no delays in message delivery).
Each web-role instance care about the client connected to it and each message doesn't have to be duplicated on every SignalR server. Hence it can scale pretty good.
Easy to implement.
The current situation: I have written an c# application server, which communicate with some applications (Computer/Smartphone/Web). Now I have the problem, that the application server has to deal with a lot of requests and it is going to be very slow.
My idea was to change the application server to work in a software cluster. To select the correct application server I want to write a load-balancer who choose the application server with the lowest workload.
My problem is, that I don't know how to write the load-balancer. Should the load-balancer work as a proxy, so that all the traffic goes through the load-balancer or should the load-balancer redirect to the application server and the application communicate directly with the application server.
Actually there are off-the-shelf products which do exactly what you're looking for, one of the most established ones is HAProxy that acts as a HTTP/TCP Load Balancer/ HA proxy, it can select appropriate server based on previous client requests (e.g. by cookie -insertion, it supports other methods), which I believe does exactly what you need.
back to the question,
Should the load balancer work as a proxy, so that all the traffic goes through the load balancer or should the load balancer redirect to the application server
Proxy implementation is a normal route to take, and Redirecting is not such a good idea and cause some disturbing issues on client-side specially browsers (e.g. bookmarks won't work as intended) and I would say it wouldn't have much gain over using proxy (aside from removing load balancer node if balancing is going to be done on client-side)
that i don't know how to write the load balancer
Short answer is you don't need to write your own, as I said before there are well established products in this area, however if you want to write your own HAProxy Architecture manual and Writing a load balancer proxy from ground up would be good start.
Answering in two parts:
You need a Proxy functionality, and not a redirect or a router
function. A redirect would reveal the IP/URL for your backend server
pool to the client, which you certainly do not want. The clients
could always bypass your LB once they know the backend IPs. Thus,
all the traffic must flow through the proxy.
I would not recommend entering the realm of writing a
LB. Its a pretty specialized function, and there are many
free/commercial baked products that can be deployed for this. You
might choose one of HAProxy, Appache HTTPD, Microsoft NLB, NginX. Each one offers a configuration choice of many load balancing algorithms, that you may want to use.
Redirecting would change the URL for the end-user, which is usually not a good idea.
What you're attempting to do is possible, but very complicated. There are numerous factors that constitute 'workload', including CPU, drive activity (possibly on multiple drives), network activity (possibly on multiple network cards), and software locking. Being able to effectively monitor all of those things is a very large project (I've never even heard of anyone taking locks into account). Entire companies are dedicated to doing stuff like that.
For your situation, I would recommend Microsoft's built-in Network Load Balancing. It does more of a random load balancing, but it gets the job done, and for the vast majority of applications, random distribution of requests results in a fairly even workload.
If that's not sufficient, get a hardware load balancer, or plan on at least two weeks of hardcore coding to properly balance based on CPU, drive activity, and network activity.
There are ready to use load balancer like Apache + mod_cluster.
Configuration can be created like .... Apache+mod_cluster -> Tomcat1 , Tomcat2 , Tomcat3 ,Tomcat4.
All request will come to Apache+mod_cluster and if it not static than distributed between Tomcat1, Tomcat2 , Tomcat3 , Tomcat4.
If request is static type than it will be handle by Apache only .
It is possible and advisable to configure Stick Session.
Main advanteage of mod_cluster is that Server-side load balance.
Apache + mod_cluster can handle HTTPS request also.
http://mod-cluster.jboss.org/
we are evaluating a new project which will have a .NET Server which is available in the internet. We have access to the server but the hosting is done by a 3rd party company.
We are evaluating using WCF on the .NET Server. (I have no professional experience with WCF and just reading into the topic). The WCF service will talk to a SQL Server to perform its duties.
Here is the scenario:
Multiple client machines running our own ActionScript software will connect to that .NET Server.
Clients might be online 24/7 and should periodically poll our server to tell the server that they are there.
A client needs to be able to login, and only if the login has worked the other calls will be allowed and at some point it logs out. So we need to "remember" the state with a particual client...
Highest expected load is around 1000 Clients, of which 500 will only do polling while the other 500 will be "active". "Active" means a maximum of 1 call each minute, no heavy payload in each call, neither in the request nor in the response, just 1-3 database accesses per call.
We already tested some "HelloWorld" with ActionScript and WCF using BasicHttp(s)Binding.
But because we need session handling we were thinking about taking using the wsHttpBinding binding because it can provide us WCF Sessions.
So far so good, but then I stumbled upon the fact that it should
However:
I find that in my Oreilly WCF Services 3rd edition book (Page 177) it is written
and even Microsoft is writing to be careful using that:
http://msdn.microsoft.com/en-us/magazine/cc163590.aspx
"A service configured for private sessions cannot typically support more than a few dozen (or perhaps up to a few hundred) outstanding clients due to the cost associated with each such dedicated service instance."
So because we need to identify the state with each client, we could of course implement our own "Session Handling" on top of stateless HttpBindingBinding, and make a call to that SessionHandling class each time when my WCF methods get called, but I am reluctant to do anything like that, it looks to me like thousands of people should already have faced the same problem.
So, my question now is:
Do you think wsHttpBinding on my server could handle the payload?
How "bad" is it really to go with wsHttpBinding on WCF? Does anybody already have experience with this? Can I use it? What would you use?
Final Remarks:
I am not limited to WCF if we dont like it, we just shall do an evaluation.
From the companies point of view it would also be fine to go for a protobuf-RPC or XML-RPC solution over TCP and the ActionScript clients implementing that. (just examples!) So no need for hosting WCF in IIS on the server as long as the coding part is comfortable (enough) for the programmers on both sides and the ADMINISTRATION on the deployed server is not too much either. With just making some TCP-ports based communication I am a bit afraid what it would mean for the administration in regards to firewall and stuff. Payload is not an issue, client processing power is also not an issue. The only thing I am concerned about is scalability of the server and security.
Thanks in advance for any suggestions!
I would not be concerned with scalability. You can always add a server or two to your farm in case of issues.
I would rather be concerned with your architecture and the need to store anything in session - are you sure about that?
Note that you don't need ws binding to support sessions, basic binding supports sessions as well.
Please actually read my post before placing it on hold!!
Let me start by saying I've been searching for a solution all afternoon and so far I have seen plenty of examples for WCF but none that would do what I need.
I have developed an application in c# that will be installed on customer servers and accesses a sql server on the customer's local network. The application also has the ability to control network relays on the customer's local network and records the status of these in sql. I am trying to figure out a way to have the customer's server establish a connection to our datacenter and be able to issue commands back to the customer's server (retrieve datasets from sql, control the network relays, etc). I have found plenty of ways to have a client call classes on a server but have so far been unsuccessful in finding the reverse. One consideration was writing a web service as part of the application on the customer's server but need a way to establish this connection for customers with dynamic IP addresses and without having to publish through firewalls, etc.
Have you considered using
VPN - Virtual private network
or
Configuring a Port Forwarding redirect on the ADSL modem, and using a solution like www.noip.com ?
If I understand correctly you want to get information from the customer's database, which is behind a firewall and has no known static ip, in addition there might be several hundred customers so a dedicated VPN to the customer is not viable.
First of all: you should not contact the customer database directly. Databases are not designed for this scenario and would probably be left open to attack if exposed directly to the internet.
So you need a service on top of the database. There are two main options you can use for this service:
Polling service
The service is actually a client calling some web service on your network and asking for instructions.
Benefits: easy to implement and deploy.
Downsides: With polling there is always the cost-benefit of scalability/bandwidth use vs. speed of service. There are also some considerations in selecting the time to poll to prevent all the client polling at the same time.
The service is a tcp-server
This can be a usual web service (or RESTfull service) or some other service. The only difference is that it needs to advertise itself. For that you need to have a known directory server. When the service starts it then connects to the directory service and tells it the port it can be contacted on (the directory knows the ip from the connection). It will then need to periodically contact the directory to let it know it is still alive and so any change in IP is detected.
A client on your network would now query the directory to find the address of the client and connect directly to it to issue commands.
Benefit: Scalable and bandwidth efficient.
Downside: More difficult to implement. Requires firewall traversal solutions (UPNP or firewall exceptions).
I am planning a SaaS system, to be written in C#, ASP.NET using WCF that has two separate components:
On a static IP web server in the cloud will be a web app, common to all clients.
Inside each client's office will be another app, installed on a server with IIS.
The site app will obviously be able to connect to the web services published on the web site. But here's the rub - I also want the web app to be able to initiate a connection to the site app... and the on-site server may not necessarily have a static IP. I can't control this, because we may have hundreds of clients at some point in the future, and we cannot limit our saleability by insisting that the customer has a server with fixed IP.
So, how to do this?
I could have the site apps "checking in" with the web every minute or so, to give the web app the possibility of responding with a "while you're here, please do x,y,z..." but that seems very inelegant. Also, if we're talking about hundreds of clients, I don't want to be bombarding my web server with all these "hi there!" messages if they're not actually required.
Is there a better way?
WCF? Here we go:
Use a message based approach (exchange message, no stateful method calls).
Clients connect to the server. Establish a HTTP-based TWO WAY CONNECTION. This way the server can call back to connected clients. This is standard WCF stuff and works well through NAT with version 4 of the .NET framework.
Voila. In case of a disconnect the client can re-connect, re-identify himself and gets the pending messages.
IIRC "push communication" is done by letting the client do a HTTP Request with an indefinate timeout. Then the server responds when he has something to say. After the respons the client immediately makes a new request.
It works out the same way like the server is making the connection and takes far less resources than polling.
Dynamic DNS is one possibility, but depends on your clients/customers.
If the site app is created by you, it only has to contact the web server when its address has changed (or when the site server/web app is restarted). Still, a keep-alive heart beat of, say, every 30 min. to 1 hour isn't a bad idea.
Edit: I think SNMP services may provide the answer but I'm not a networking expert. You'll have to do some digging or ask a separate question on stackoverflow.
What would you say about Comet technology?
Sounds like you'll definitely need some sort of registry on the server, then it could attempt to call out to the client apps if it needs work doing.
Generally it is client apps that check in with the server every X seconds - this is how Selenium grid works anyway. With a central hub with which clients register. When the hub receives a request to run some tests it passes the jobs out to the clients to perform.
You may not need the "checking in". The server could just attempt to call out to a registered client app until it finds one that is available.This way only the server would need a static address (could use a DNS name instead of an IP to make it more robust).
Also have a look at XMPP PubSub. This could be a more robust and standardised way to handle this.
In the end I decided to go with NetTcpBinding, for reasons best given by #Allon Guralnek here. It's worth clicking through and reading what he has to say...