I recently had penetration testing performed on my website and as a result of that there is a certain API call which I would like to implement an IP based form of throttling on to prevent abuse.
What I am looking for is a way to add and remove IP addresses from a blacklist, that is applied to my azure website.
I'm aware that it is possible to restrict certain IP Addresses/ranges from the security section of the web.config but I have been unable to find a way to achieve this programtically.
Additionally I can't go in and edit the web.config on the fly because that would recycle my app pool causing me to loose session & cache data which could adversely affect users.
I have considered implementing a solution reliant on storing the blacklist on my database but as this is for security against automated attacks I would rather block it before this point to avoid using resource querying the database.
Related
Hi we have been working with the Azure Load Balancer and have the distribution mode set to IP Source. We are of course running multiple web machines.
We are using this with an MVC application. So far we haven't found any issues with maintaining session in Test.
My question is;
Is it normal practice to use IP Source to maintain sticky session's so that the client is always sent to the correct box running In Proc session?
Are there any issues with or pit falls with this setup.
I can't seem to find a definite answer anywhere.
The current best practice is to avoid server-side session state when possible; which ends up being about 95% of the time generally. To do this you would use Cookie-based Authentication in an ASP.NET MVC application. And most data stored within Session state can be cached or loaded on the fly with minimal impact to the system. After all, natively the web is stateless anyway.
IP Source based load balancing is fine; so is using server-side session state if necessary. However, the application really should be designed / built in a fashion that supports it being more stateless. That is that any Session data is just loaded appropriately if it doesn't exist. You really want to gracefully handle user requests without throwing an exception if at all possible so the user can keep getting their job done.
By keeping things more stateless you may get better application performance by using either Round-Robin or Performance based load balancing.
The current situation: I have written an c# application server, which communicate with some applications (Computer/Smartphone/Web). Now I have the problem, that the application server has to deal with a lot of requests and it is going to be very slow.
My idea was to change the application server to work in a software cluster. To select the correct application server I want to write a load-balancer who choose the application server with the lowest workload.
My problem is, that I don't know how to write the load-balancer. Should the load-balancer work as a proxy, so that all the traffic goes through the load-balancer or should the load-balancer redirect to the application server and the application communicate directly with the application server.
Actually there are off-the-shelf products which do exactly what you're looking for, one of the most established ones is HAProxy that acts as a HTTP/TCP Load Balancer/ HA proxy, it can select appropriate server based on previous client requests (e.g. by cookie -insertion, it supports other methods), which I believe does exactly what you need.
back to the question,
Should the load balancer work as a proxy, so that all the traffic goes through the load balancer or should the load balancer redirect to the application server
Proxy implementation is a normal route to take, and Redirecting is not such a good idea and cause some disturbing issues on client-side specially browsers (e.g. bookmarks won't work as intended) and I would say it wouldn't have much gain over using proxy (aside from removing load balancer node if balancing is going to be done on client-side)
that i don't know how to write the load balancer
Short answer is you don't need to write your own, as I said before there are well established products in this area, however if you want to write your own HAProxy Architecture manual and Writing a load balancer proxy from ground up would be good start.
Answering in two parts:
You need a Proxy functionality, and not a redirect or a router
function. A redirect would reveal the IP/URL for your backend server
pool to the client, which you certainly do not want. The clients
could always bypass your LB once they know the backend IPs. Thus,
all the traffic must flow through the proxy.
I would not recommend entering the realm of writing a
LB. Its a pretty specialized function, and there are many
free/commercial baked products that can be deployed for this. You
might choose one of HAProxy, Appache HTTPD, Microsoft NLB, NginX. Each one offers a configuration choice of many load balancing algorithms, that you may want to use.
Redirecting would change the URL for the end-user, which is usually not a good idea.
What you're attempting to do is possible, but very complicated. There are numerous factors that constitute 'workload', including CPU, drive activity (possibly on multiple drives), network activity (possibly on multiple network cards), and software locking. Being able to effectively monitor all of those things is a very large project (I've never even heard of anyone taking locks into account). Entire companies are dedicated to doing stuff like that.
For your situation, I would recommend Microsoft's built-in Network Load Balancing. It does more of a random load balancing, but it gets the job done, and for the vast majority of applications, random distribution of requests results in a fairly even workload.
If that's not sufficient, get a hardware load balancer, or plan on at least two weeks of hardcore coding to properly balance based on CPU, drive activity, and network activity.
There are ready to use load balancer like Apache + mod_cluster.
Configuration can be created like .... Apache+mod_cluster -> Tomcat1 , Tomcat2 , Tomcat3 ,Tomcat4.
All request will come to Apache+mod_cluster and if it not static than distributed between Tomcat1, Tomcat2 , Tomcat3 , Tomcat4.
If request is static type than it will be handle by Apache only .
It is possible and advisable to configure Stick Session.
Main advanteage of mod_cluster is that Server-side load balance.
Apache + mod_cluster can handle HTTPS request also.
http://mod-cluster.jboss.org/
I'm working on a Cloud-Hosted ZipFile creation service.
This is a Cross-Origin WebApi2 service used to provide ZipFiles from a file system that cannot host any server side code.
The basic operation goes like this:
User makes a POST request with a string[] of Urls that correlate to file locations
WebApi reads the array into memory, and creates a ticket number
WebApi returns the ticket number to the user
AJAX callback then redirects the user to a web address with the ticket number appended, which returns the zip file in the HttpResponseMessage
In order to handle the ticket system, my design approach was to set up a Global Dictionary that paired a randomly generated 10 digit number to a List<String> value, and the dictionary was paired to a Queue storing 10,000 entries at a time. ([Reference here][1])
This is partially due to the fact that WebApi does not support Cache
When I make my AJAX call locally, it works 100% of the time. When I make the call remotely, it works about 20% of the time.
When it fails, this is the error I get:
The given key was not present in the dictionary.
Meaning, the ticket number was not found in the Global Dictionary Object.
We (with the help of Stack) tracked down the issue to multiple servers in the Cloud.
In this case, there are three.
That doesn't mean there is a one-in-three chance of this working, what seems to be going on is this:
Calls made while the browser is on the cloud site work 100% of the time because the same machine handles the whole operation end-to-end
Calls made from other sites work far less often because there is no continuity between the machine who takes the AJAX call, and the machine who takes the subsequent REDIRECT to the website to download the file. It's simple luck of the draw if the same machine handles both.
Now, I'm sure we could create a database to handle requests, but that seems like a lot more work to maintain state among these machines.
Is there any non-database way for these machines to maintain the same Dictionary across all sessions that doesn't involve setting up a fourth machine just to handle queue?
Is the reason for the dictionary simply to have a queue of operations?
It seems you either need:
A third machine that hosts the queue (despite your objection). If you're using Azure, an obvious choice might be the distributed Azure Cache Service.
To forget about the dictionary and just have the server package and deliver the requested result, perhaps in an asynchronous operation.
If your ASP.NET web app uses session state, you will need to configure an external session state provider (either the Redis Cache Service or a SQL Server session state provider).
There's a step-by-step guide here.
We are departing from our typical native application development and are developing some new web-based application services. We are using .Net and WCF and most likely hosting on IIS.
Our services will need to use a database to persist data. In all cases we intend to make the database server a different box than our host. Most of the resources we find seem to indicate storing the database connection string in the web.config (or app.config file for self-hosting). Our concern is that since the service is web-facing the box it is running on is a more likely to be hacked/compromised. If that happens, the attacker now had the DB connection string and will have full access to all of the data.
An initial thought was to make the WCF service simply proxy to a server application that lives on another machine that holds the connection information and does all the processing logic. That way, if the the web-host is compromised they would have to figure out how to communicate with our server application, or compromise that machine as well, to gain database access. Though we do have concerns of the performance costs of proxying and possibly having a bottle-neck at the server application.
The concern with that approach is that we are not finding much in the way of WCF/IIS literature that recommending on such an approach, but we might just not know what it is called in order to find such information. The question: is proxying like that a good practice and why/(why not) (or where might I find more information on this topic), or is there a better practice?
As with these things, it all depends on how secure you need things. You can encrypt your web.config:
http://msdn.microsoft.com/en-us/library/dtkwfdky.aspx
But you have to ask that if your attacker has the ability to compromise your web host machine, they will be able to find your other server, and may also then have the ability to compromise that - not 100% sure what the net gain here is (apart from making your life more difficult).
Is it possible to detect whether a website has a dedicated or shared ip address from it's url using C# (Windows Forms application) ? I want to implement a functionality in my application to let write a web address in a TextBox than i click on the Test button. and then show ( Success ) MessageBox if the site has a Dedicated ip address or show a ( Failure ) MessageBox otherwise.
How can i detect whether a website has a Shared or Dedicated IP Address using C#.NET?
You can try, but you'll never have a good result. The best I think you could do is to check the PTR records of the IP, and then check if there are associated A records from different websites. This would still suck however, since a website could have two seemingly different domains that pertain to the same organization (googlemail.com/gmail.com for example).
Also, this assumes the existence of PTR records, multiple ones. I don't think I've seen such a setup supported by most VPS/sharing hosting.
Well, the way I would do it is:
Send HTTP GET to the URL and save the result.
Resolve the URL to an IP.
Send HTTP GET to the IP and save the result.
Compare the two results. (You can do sample checks between the two result)
If the results are the same, then this is dedicated hosting, if the result is different then this is a shared hosting.
Limitations for this method that I can think of now:
Will take you time to figure our a proper comparing method for the
two results.
If shared hosting is configured to default route to the site which you are checking.
Functions to resolve URLs, and do web requests for different programming languages are scattered across the Internet.
From a technical standpoint, there's no such thing as a "shared" or "dedicated" IP address; the protocol makes no distinction. Those are terms used to describe how an IP is used.
As such, there's no programmatic method to answer "is this shared or dedicated?" Some of the other answers to this question suggest some ways to guess whether a particular domain is on a shared IP, but those methods are at best guesses.
If you really want to go down this road, you could crawl the web and store resolved IPs for every domain. (Simple, right?) Then you could query your massive database for all the domains hosted on a given IP. (There are tools that seem to do this already, although only the first one was able to identify the multiple domains I have hosted on my server.)
Of course, this is all for naught with VPS (or things like Amazon EC2) where the server hardware itself is shared, but every customer (domain) gets one or more dedicated IPs. From the outside, there's no way to know how such servers are set up.
TL;DR: This can't be done in a reliable manner.