this might be a stupid question but I have to ask. I've never used Azure before but a client is looking to send some SQL databases and their web server to the cloud. On the Azure site they refer to billing for usage per hour.
If I create 10 SQL Databases, is usage considered the actual amount of time they were used by the application, or am I charged for the amount of time I had the database instances themselves? Same with a web application...if the web application goes 2 weeks without any web traffic, does that still count as usage since I have the app live in Azure? If the app is not used then the databases wouldn't be either, so both would be idle and not used at the moment.
I guess I'm just confused as to what the word "usage" is actually referring to.
Meaning of Usage in Azure varies based on the type of resource. For some items, usage is calculated in terms of consumed hours (websites, virtual machines etc. would come there) whereas for certain items it is calculated in terms of consumed space (azure storage is a good example of that).
Also, please note that pricing is not based on the utilization (e.g. how many times a website got hit) but based on provisioning. So in your example, if a website is provisioned for you, you will pay for it irrespective of the fact that anybody is using that website or not.
I would recommend taking a look at Azure Pricing Calculator to understand approximately how much are you going to pay by resource type.
Related
Azure documentation says...
Azure Functions is a solution for easily running small pieces of code,
or "functions," in the cloud.
Why is there the qualifier "small" in this text? Are there any numbers to elaborate on this claim?
Why is there the qualifier "small" in this text?
I think they are trying to tell us that Azure Functions are not intended to take the place of large, long-running programs. There are other Azure solutions better suited for running a high-resolution graphics renderer or a program to calculate Pi to some enormous number of places. Azure functions are intended to fill a different niche where a trigger causes some code to run, calculate a result fairly quickly and then return that result and stop.
Which is not to say that Azure Functions couldn't (in theory) support longer running code but that's not what it is optimized for.
Are there any numbers to elaborate on this claim?
The Azure Functions scale and hosting page indicates that the default timeout for functions on the Consumption plan is five minutes and the maximum is ten minutes. However the page also notes that http triggered functions must respond in 230 seconds or less no matter what the timeout value is.
Functions on the App Service plan can have an unlimited maximum timeout - presumably because we're paying whether we use it or not (but that's just a semi-educated guess).
If we compare other services that Microsoft offers like Service Fabric, Functions, and App Services, Azure Functions are tiny. Service Fabric, and App Services have focused on deploying complete services. An Azure Function is really just a method call. As result a complete microservice may actually be made up of a collection of Azure Functions. An example of this is an API, each route for your API would actually be a completely independent Function. This may be an ideal world for you, or you may view this as too much effort.
you can check out more here
Can you please explain what is the best way to estimate the transactions number on windows azure storage using the development environment? I though about implementing an int variable and increment that ex: i++ each time I make a call to azure storage? What do you think? Have you done such thing before? I just need to have an estimated amount of transactions ...
there's the Windows Azure Storage Services REST API which you can use, it contains a a full API stack for Storage Analytics: http://msdn.microsoft.com/en-us/library/windowsazure/hh343270.aspx. hope this helps (of course you can use the native monitoring through the portal also for starters)
#techmike2kx gave you the REST API info. Instead, let me address your other question regarding the use of a local transaction counter. That approach won't really help you at all, for a few reasons:
If you have multiple instances of your app running (e.g. 2 web role instances), you'd need a single counter across instances, which means you're now synchronizing, or you're accumulating numbers somewhere. And... you'll probably store these instance-specific counters in something like table storage, which will result in additional transactions.
What if you use an attached disk with your VM? There will be transactions generated since the vhd is stored in a blob. You'll have no visibility into those transactions.
Your storage account could be used by multiple apps. How will you track that?
Your storage account could be used for logging and diagnostics, which you don't have much control over, regarding how those calls are made.
You'll need to track unsuccessful transactions since these are not billed (these are documented).
Some calls make multiple storage transactions. For instance: If you query table storage, and exceed what can be returned in a single transaction, you'll end up with multiple calls to storage, under the hood (hidden by the language-specific SDK you're using).
What happens when you serve web content directly from blobs (e.g. http://mysite.blob.core.windows.net/images/logo.jpg)? You'd have no control over this access, so no way to track it
when will you roll your counter back to zero? How will you know the exact month-end of your billing cycle?
I'm sure there are other gotchas, but the bottom line is: You shouldn't be trying to track transaction consumption, since it's given to you via storage analytics.
I'm looking for a simple way to implement a local memory store which can be used on an Azure .NET instance
I've been looking at Azure Co-located Caching and it seems to support all of my requirements:
Work on both web roles and worker roles
Implement a simple LRU
Keep cached objects in memory (RAM)
Allow me to define the cache size as a percentage of the machine's total RAM
Keep the cache on the same machine of the web/worker role (co-located mode)
Allow me to access the same cache from multiple AppDomains running on the same machine (Web Roles may split my handlers into different AppDomains)
The only problem I have with Azure Co-located caching is that different instances communicate and try to share their caches - and I don't really need all that.
I want every machine to have its own separate in-memory cache. When I query this cache, I don't want to waste any time on making a network request to other instances' caches.
Local Cache config?
I've seen a configuration setting in Azure Caching to enable a Local Cache - but it still seems like machines may communicate with each other (ie. during cache miss). This config also requires a ttlValue and objectCount and I want TTL to be "forever" and the object count to be "until you fill the entire cache". It feels like specifying maxInt in both cases seems wrong.
What about a simple static variable?
When I really think about it, all this Azure caching seems like a bit of an overkill for what I need. I basically just need a static variable in the application/role level.. except that doesn't work for requirement #6 (different AppDomains). Requirement #4 is also a bit harder to implement in this case.
Memcached
I think good old memcached seems to do exactly what I want. Problem is I'm using Azure as a PaaS and I don't really want to administer my own VM's. I don't think I can install memcached on my roles.. [UPDATE] It seems it is possible to run memcached locally on my roles. Is there a more elegant "native" solution without using memcached itself?
You can certainly install memcached on Web and Worker roles. Steve Marx blogged getting memcached running on Azure Cloud Service several years ago before the Virtual Machine features were present. This is an older post, so you may run into other ways of dealing with this, such as using start up tasks instead of the OnStart method in RoleEntryPoint, etc.
I have used the "free" versions of SQL Server for local caching and they have worked great. It depends on what you are doing, but I have ran both SQL Server Express/Compact for storing entire small static data sets for a fantasy football site I wrote that included 5 years of statistics. They worked really well even on a small/medium Azure instances, because of the small footprint.
http://blogs.msdn.com/b/jerrynixon/archive/2012/02/26/sql-express-v-localdb-v-sql-compact-edition.aspx
Best part is you can use t-sql. Your cache requirements might be more complex or not scale to this.
I am developing a web service which will serve users all over the world.
The server is based on a C# WCF application hosted on IIS.
It uses an MsSQL Configuration Database (access time is not important here),
and a MongoDB database which contains all the important data (access time is VERY important here).
Also it serves small images (48px * 48px JPEGs).
Now, for the image hosting I will probably use Amazon's CloudFront CDN hosting (unless you guys have better suggestions).
My issue is maintaining a low access time (ping) to both the Web Application and the MongoDB.
I was thinking to lease 4 servers in Singapore + US + Europe + Middle East to get a low response time.
Each server will hold the Web Application and an instance of MongoDB.
And one server will hold the MsSQL instance.
I need all MongoDB's to be synced (not instantly if its an issue).
What design would you use?
Low access time is a function of cost vs benefit. First you need to identify, how low is low. Do you need a response time of 100ms overall from the app? or 1s?
Once you do, you map out the different costs.
Total time taken = time for request + //across internet
processing by web app + request for data +
preparing the response + response back to client.
If your desired latency is 100ms, there is a good chance that it can't be done regardless of how fast your servers are, simple because network traffic might take too long.
You need to analyze your dataset. Querying 1000 documents is different from querying 1 billion docs. You need to calculate how much size the index is taking, and is it in RAM or not. If index is not in RAM, your access is going to be slow.
Mongodb configuration
Mongodb can work in a cluster, with automatic syncing (immediate or delayed, this is configurable), and automatic failover (or manual, this is configurable too). It also supports sharding if your dataset is huge, so request is sent to the server that actually contains data.
Similarly, you need to have a look at your app server and figure out how slow/fast components are to get a guaranteed response time.
With the information you have provided, this is about as detailed a response I can give.
Profile and then optimize
If 80% of your requests come from middle east, then you ought make it fast for them first. Using the same principal, you need to figure out the slowest components in response time, and improve them. In order to do that, you need to gather the data.
Clustering
Setting up a cluster in one continent or across continents, will help you provide redundancy, automatic failover (if configured), and load balancing (depending on how you configure it). If you have alot of data, consider sharding.
Consider going through the docs for replication and sharding.
Example Server Setup
Suppose you want to have 10 shards with replication factor of 3 i.e your data is divided across 10 servers and each server really is a replica set of 3 servers (for availability and fail over) i.e each server in the replica set contains duplicate data.
here notation s1p1 means, shard1 - primary 1 and s1s1 is shard 1 secondary 1 and so on
s1p1 s2p1 ... s10p1
s1s1 s2s1 ... s10s1
s1s2 s2s2 ... s10s2
Shards 1-10 divide the data, where each shard approximately keeps 1/10 of total. Each shard comprises of a replica set with a primary and 2 secondaries. You can increase this if you need more redundancy. Try to keep it to odd, so during elections there is a tie breaker. If you want to have only 2 copies of the data, then you can also introduce an "Arbiter" to break the tie.
You could analyze your queries, and choose a shard key so that they go to the closest server, or the server which serves the region. You'll most likely have to do some sort of analysis to optimize this bit.
Hope it helps.
I am trying to setup in Windows Azure a global counter which would keep track of the number of games started within a day. Each time a player starts a game, a Web Service call is made from the client to the server and a global counter would be incremented by one. This should be fairly simple to do with a database... But I wonder how I could efficiently do this. The database approach is good for a few hundreds clients simultaneously, but what will happen if I have 100,000 clients?
Thanks for your help/ideas!
A little over a year ago, this was a topic in a Cloud Cover episode: Cloud Cover Episode 43 - Scalable Counters with Windows Azure. They discussed how to create an Apaythy Button (similar to the Like Button on Facebook).
Steve Marx also discusses this in detail in a blog post with source code: Architecting Scalable Counters with Windows Azure. In this solution they're doing the following:
On each instance, keep track of a local counter
Use Interlock.Increment to modify the local counter
If the counter changed, save the new value in table storage (have a timer do this every few seconds). For each deployment/instance, you'll have 1 record in the counters table.
To display the total count, take the sum of all records in the counters table.
Well, there are a bunch of choices. And I don't know which is best for you. But I'll present them here with some pros and cons and you can come to your own conclusions given your requirements.
The simplest answer is "put it in storage." Both SQL Azure and the core Azure table or blog storage options are out there for you. One issue to contend with is performance in the face of large scale concurrency, but I'd also encourage you to think about correctness. You really want something that supports atomic increment to outsource this problem IMO.
Another variation of a storage oriented option would be a highly available VM. You could spin up your own VM on Azure, back a data drive on to Azure Drives, and then use something on top of the OS to do this (a database server, an app that uses the file system directly, whatever). This would be more similar to what you'd do at home but would have fairly unfortunate trade-offs...your entire cloud is now reliant on the availability of this one VM, cost is something to think about, scalability of the solution, and so on.
Splunk is also an option to consider, if you look at VMs.
As an earlier commenter mentioned, you could compute off of log data. But this would likely not be super real time.
Service Bus is another option to consider. You could pump messages over SB for these events and have a consumer that reads them and emits a "summary." There are a bunch of design patterns to consider if you look at this. The SB stack is pretty well documented. Another interesting element of SB is that you might be able to trade off 100% correctness for perf/scale/cost. This might be a worthy trade-off for you depending upon your goals.
Azure also exposes queues which might be a fit. I'll admit I think SB is probably a better fit but it is worth looking at both if you are going down this path.
Sorry I don't have a silver bullet but I hope this helps.
I would suggest you follow the pattern described in .NET Multi-Tier Application. This would help you decouple the Web role which faces your clients and the Worker role, which will store the data to a persistence medium (either SQL Server / Azure Storage) by using the Service Bus.
Also, this is an efficient model to scale as you can span new instances of web role or worker role or both. For the dashboard depending on the load you can Cache your data periodically and server it from the Cache. This would compromise on the accuracy of the data, but would still provide with an option for easy scaling. You can even invalidate the cache every 1 minute and get it loaded from the persistence medium to get the latest value.
Regarding to use SQL Server or Azure storage, if there is no need for relational capabilities like JOINS etc, you can very well go for the Azure storage.