I have a company network under my control and a couple of closed customer networks. I want to communicate from a web application in my network to a database inside a customer network. My first idea was:
Web application stores query in a database in the company network and waits for answer.
Windows service inside client network polls our database a couple of times every second through a (WCF) web service also in our company network.
If a query is available the Windows service executes it in it's local database and stores the answer in the company database.
I've been thinking about removing the polling idea and instead using persistent connection between a client in the customer network and a server in our company network. The client initiates the connection and then waits for queries from the server. What would be better or worse compared to polling through a web service? Would WCF be the right thing to use here?
you have few approaches:
WCF Duplex, Once the web application stores a query in database, you initiate a call to the client (in this case the Windows Service) instead of making the windows service polls every few seconds. net.tcp will be good choice but still you can use http.
Long polling, Instead of letting your Windows Service client sends a request every few seconds, let it send the request, the channel is open, set the timeout in both client and WCF service for longer time, let the server method loops and checks the database for new notifications. Once new notifications found, the method returns with them. At Client side, once you get a return send another request to the server and then process the data. If timeOut error occure, send another request. Just Google Long polling you will find a lot.
Regarding querying the database every few seconds, the better approach would be making a table for notifications, So instead of querying a large table with a complex sql string every few seconds you can let the client add the notifications in a separate table (after they are done adding them the main table), so your query will be much simpler and takes less resources. you can add direct pointers (Like Ids) in the notifications table to save time. Later clean up the notifications table..
Related
I am working on a web application in C#, ASP.NET, and .NET framework 4.5 with the use of WebSockets. In order to plan for scalability in the future, the application pool has the option for web gardens enabled to simulate multiple web servers on my single development machine.
The issue I am having is how to handle re-connects on the websocket side. When a new websocket session is initially created, the client browser can indirectly lock records in a SQL database. But when the connection is lost, my boss would like the browser to attempt to re-connect to the same instance of the websocket server session so it doesn't need to re-lock anything.
I don't know if something like this is possible because on re-connect the load balancer will "randomly" select which web server to handle the new connection. I was thinking of some hack to work around this but it isn't very clean:
Client opens initial websocket connection on Server A and locks a record.
Client temporarily loses internet connection and the websocket closes. (It is important to note that the server side will wait up to 60 seconds before it "disposes" itself; therefore, the SQL record will remain locked until the 60 seconds has elapsed).
Client internet connection is restored and reconnects to the website but this time on Server B.
Server B sees that this context was initially connected on Server A; therefore, transfers the session to Server A.
Server A checks the process id to see if it is running in the correct worker process (in the case of a web garden).
Server A has found the initial instance and handles the connection.
I tried Googling this question but it doesn't seem like a very common issue because I don't think most websocket web apps keep records locked for as long that my applications does (which is could be up to an hour).
Thanks in advance for all of your help!
Update 3/15/2016
I was hoping that the Server.TransferRequest would have been helpful however it doesn't seem to work for web sockets. Would anyone know of a way to best transfer a websocket context from one process to another?
First, you might want to re-examine why you're locking records for a long time and requiring a client to come back to the same server every time. That is not the usual type of high scale web architecture and perhaps you're just creating this need to reconnect to the identical server because of that requirement when maybe you should rethink how that is designed so that your application would work just fine no matter which host a user connects to.
That would certainly simplify scaling to large numbers of users and servers if you could remove that requirement. You can always then implement local caching and semi-sticky connections later as a performance enhancement, but only after you release the requirement to 100% of the time connect to the same host.
If you're going to stick with that requirement to always connect to the same host, then you will ultimately need some sort of sticky load balancing. There are a lot of different schemes. Some are driven by the networking infrastructure in front of your server, some are driven by your server and some are even client driven. They all have different tradeoffs. Here's a brief run-down of some of the schemes:
Hardware, networking load balancer. Here you have a fairly transparent mechanism by which a hardware load balancer (which is really just software running on a custom piece of hardware) sits in front of your web server farm and uses various techniques to make sure whatever server a given user is originally connected to it will get reconnected to on subsequent connections. This can be based on various schemes (IP address, cookie value, etc...) as the key to identifying a particular user and it typically has a number of possible configurations for how it can work.
Proxy load balancer. This is essentially an all software version of the hardware load balancer. Here a proxy sits in front of your server farm and directs connections to a particular server based on some algorithm (IP address, cookie value, etc...).
Server Redirect. Here an incoming connection is randomly assigned to a server. Upon connection the server figures out where the connection is supposed to be connected to an returns a 302 redirect to the actual host causing the client to reconnect to the proper server. This involves one less layer of infrastructure (no physical load balancers), but exposes the different server endpoints to the outside world which the first two options do not.
Client Selection Algorithm. Here the client is given knowledge of the various server endpoints and is coded with an algorithm for consistently selecting one for this user. It could be a hash of a userID that is then divided into the server bucket pool and the end result is that client ends up choosing a particular DNS name such as cl003.myserver.com which it then connects to. This choice requires the least work server-side so can be simpler to implement, but it requires changing the client code in order to modify the algorithm.
For an article on sticky load balancing for Amazon Web Services to give you an idea on how one mechanism works, you can read this: Elastic Load Balancing: Configure Sticky Sessions for Your Load Balancer.
Here's another article on how the nginx proxy is configured for sticky load balancing.
You can find lots of other articles with a Google search for "sticky load balancing".
A discussion of the pros/cons of the various schemes is the subject of a much longer discussion and some of it involves knowledge of more specific requirements and specific capabilities of your infrastructure.
Please actually read my post before placing it on hold!!
Let me start by saying I've been searching for a solution all afternoon and so far I have seen plenty of examples for WCF but none that would do what I need.
I have developed an application in c# that will be installed on customer servers and accesses a sql server on the customer's local network. The application also has the ability to control network relays on the customer's local network and records the status of these in sql. I am trying to figure out a way to have the customer's server establish a connection to our datacenter and be able to issue commands back to the customer's server (retrieve datasets from sql, control the network relays, etc). I have found plenty of ways to have a client call classes on a server but have so far been unsuccessful in finding the reverse. One consideration was writing a web service as part of the application on the customer's server but need a way to establish this connection for customers with dynamic IP addresses and without having to publish through firewalls, etc.
Have you considered using
VPN - Virtual private network
or
Configuring a Port Forwarding redirect on the ADSL modem, and using a solution like www.noip.com ?
If I understand correctly you want to get information from the customer's database, which is behind a firewall and has no known static ip, in addition there might be several hundred customers so a dedicated VPN to the customer is not viable.
First of all: you should not contact the customer database directly. Databases are not designed for this scenario and would probably be left open to attack if exposed directly to the internet.
So you need a service on top of the database. There are two main options you can use for this service:
Polling service
The service is actually a client calling some web service on your network and asking for instructions.
Benefits: easy to implement and deploy.
Downsides: With polling there is always the cost-benefit of scalability/bandwidth use vs. speed of service. There are also some considerations in selecting the time to poll to prevent all the client polling at the same time.
The service is a tcp-server
This can be a usual web service (or RESTfull service) or some other service. The only difference is that it needs to advertise itself. For that you need to have a known directory server. When the service starts it then connects to the directory service and tells it the port it can be contacted on (the directory knows the ip from the connection). It will then need to periodically contact the directory to let it know it is still alive and so any change in IP is detected.
A client on your network would now query the directory to find the address of the client and connect directly to it to issue commands.
Benefit: Scalable and bandwidth efficient.
Downside: More difficult to implement. Requires firewall traversal solutions (UPNP or firewall exceptions).
I am developing a Windows RT application that needs to get data from a MVC WebApi server.
The problem is that the response can take from few seconds to 3 minutes.
Which is the best approach to solve it?
For now, I call async to the web api and put a long timeout value to avoid exceptions. Is it a good way? I do not like too much because the server have a open connection opened all time. Can it affect significantly to the server performance?
Is there some thing like "callback" but for web services? I mean that the server calls to the client to send the data.
Yes, there are ways to get server to callback client, for example WCF duplex communication. However, such techniques will usually keep the connection open (in most cases this is TCP session). Most web servers do not support numerous concurrent requests and thus each prolonged call to the server will increment the number of concurrently connected clients. This will lead to heavy resource utilisation at the point where it shouldn't be. If you have many clients, such architecture is bound to fail.
REST requests shall be lightweight, small and fast. Consider using a database to store temporary results and worker servers, to process the load. This is a server-side problem, not client-side.
Finally I solved it using WebSockets (thanks oleksii). It keeps the connection open but I avoid to poll for the result repeatedly. Now, when the server finishes the process, sends the data directly to the client. WebSockets is a protocol that relays over TCP and has been standardized.
http://en.wikipedia.org/wiki/WebSocket
Do to some constraints with a client's system and setup, I am wondering if it is possible for SQL Server to fire off a web request?
I need to be able to send emails after a sql server column gets updated. I had a trigger with database mail but the database server on the client won't allow that. The alternative would have that figure fire off a web request when ready. I would do it via a web service on the web server and have it just poll the table, but that seems like a lot of wasteful cycles.
Any ideas?
You can execute managed code from SQL server. Create a windows service and let that do whatever you want in C# to fulful your need and SQL server call it.
Checkout this link for sample program
Do not call web services from SQLCLR, and especially do not call web services (or send email) via SQLCRL from triggers. There are many reasons why using SQLCLR for external http or smtp access is a bad idea, but doing it from a trigger is horrendous. Just think how will your application behave in a day when the web service responds with a 10 second latency...
You must do the mail interaction from an external application. You should decouple the interaction between your application and the emailer application via a queue. See Using Tables as Queues for how to use a SQL Server table as a queue. You can enqueue the mailing request directly from your application, or use a stored proc that both inserts the data and enqueue the request, or as the last resort, use a trigger on the table to enqueue the mailing request. You emailer application (a separate service) should monitor this queue and service it by doing the actual email send (or http web request , for the matter).
I am trying to write a monitoring tool to monitor some information
It will gonna work on azure normally. So i gonna host the database on azure also the webservice will be hosted at azure.
On the client's i read from the config file how many time's he need to update the information to the azure database ( with the webservice on azure ).
Now i want to send also some commands to the client itself. Like start service, .... what is the best way to do that?
How can i send it from a website that is hosted on the azure platform?
I think you should consider implementing a WCF service at the client as well. The Azure side of your software could call operations from this service when it needs to instruct the client to do something.
The WCF service at the client should be something simple,hosted in a Windows Service or in your actual client (whatever it is... win forms, console, etc).
Since you have no VPN, it sounds like you may have a problem with hosting a WCF service on the client. If the client is behind a firewall, you would have to modify the firewall configuration to allow your server to connect to this service.
Last time I had to do a service like this, I used Comet. The server maintains a queue of messages to be sent to the client. Your client connects to the web service and requests any available messages. If messages are available, the server returns them. If not, the server leaves the request open for some time. As soon as a message arrives, the server sends it down the already-open connection. The client will either periodically time out/reconnect or send a keep-alive message (perhaps once per minute) in order to keep the connection alive in the intervening firewalls.