I have code that parses 100 to 200 html webpages using HtmlAgilityPack every hour. I am collecting the parsed data and I am making an xml file which would then be consumed by users.
Now I want to move this code to Azure. Would it be possible to parse websites on Windows azure? If so: what kind of service provided by Azure should I use?
I am not familiar with Azure. Any hints would be appreciated. Thanks.
Azure offers you a couple of solutions:
IaaS
IaaS or Infrastructure as a Service. This means that you get the infrastructure from Azure and that you can host your own virtual machine on it. This does require you to maintain the server yourself but you get full flexibility.
PaaS or Platform as a Service. This not only gives you a VM to run your code on but also an operating system that is automatically maintained for you. This is what Azure is really about and what you should aim for.
When creating a PaaS application you can choose for a regular website or for a cloud service. A cloud service contains roles, web roles and worker roles. Web roles are regular IIS hosted websites. Worker roles are the Azure equivalent of a Windows service.
In your case I would look at worker roles. They can run continually and are ideal for exeucting scheduled operations. From your worker role you can access Azure Blob storage to store the XML files that you create. Those files can be exposed to external users in a secure way.
Windows Azure is a hosting environment. According to what kind of project you created, you may be looking for a Cloud Service, where you can just host any code you've written.
Related
We have active directory in place which runs internally (a server on our own company network).
We also have a web app hosted in Azure (entirely separate to our own network obviously), which all works - but we've now introduced functionality into it that means we need to authenticate.
Rather than roll out a separate membership implementation, is it possible the app in the cloud can offer login features which integrate with our own network's AD remotely?
What's involved in this?
Is it made easier if we move our AD to Azure?
Definitely It will much easier if you move to Azure. You can follow the Configure your App Service app to use Azure Active Directory sign-in doc which discuss about this in detail.
You can look at Hybrid Identity and Azure AD Connect
Many organizations run into a similar scenario where they are already using an on-premises identity system (like Active Directory) and now want to utilize cloud based capabilities.
Hybrid identity can help especially when you have a mixture of on-premises and cloud based applications that are being accessed by same set of users (or identities)
There is a bit of setup involved with Azure AD Connect and multiple options available that you'll need to choose from depending on more specific requirements.
Amongst the different options, I have seen the Federation option work quite well in a few cases (doesn't mean you have to go with exactly this option, pick what works best as per your case).
In this case an organization that already had applications working with on-premises Active Directory, now takes Azure subscription(s) and Azure Active Directory tenant, which adds a verified domain that federates to the on-premises Active Directory (using Azure AD Connect and ADFS).
This way on-premises applications can continue to use Active Directory. At the same time newer cloud based applications trust Azure Active Directory tenant, which can federate the authentication process to on-prem infrastructure (after the right setup).
Once you have the right setup, Azure hosted web-app can trust Azure Active Directory which in turn works with on-premises AD.
You haven't mentioned which Azure service you're using to host the web app, a PaaS service like an Azure App Service or simply using Azure IaaS like a VM (or scaleset). You should be able to work with Azure Active Directory in all options AFAIK, but feel free to add more details in case you face issues.
I've got a console application, with logging, scheduling, etc., running on a VM. It works with Entity Framework, Azure SQL Server, Azure blob storage and a bunch of external APIs.
I'd like to turn it into a service.
I understand that this can be done with worker roles, but looking at various tutorials for worker roles, it seems like it would be a ton of work to rewrite the whole thing.
If I just publish it as a web job, will this be secure, assuming I don't have any exposed endpoints? I need to make sure that nobody outside our active directory can access it.
Is there a way that I can create a dummy app service which will have no endpoints and publish it there?
If I just publish it as a web job, will this be secure, assuming I don't have any exposed endpoints? I need to make sure that nobody outside our active directory can access it.
As Peter Bons said, if it does not expose an API or endpoint, nobody can access it.
Is there a way that I can create a dummy app service which will have no endpoints and publish it there?
Normally we run WebJobs in a Azure App Service web app, and that Azure App Service web app can be accessed/browsed via URL. If you want to prevent users from browsing to that Azure App Service web app, you can add a rewrite rule to site’s web.config to block web access.
<rule name="Block unauthorized traffic to staging sites" stopProcessing="true">
<match url=".*" />
<conditions>
<!-- Enter your site host name here as the pattern-->
<add input="{HTTP_HOST}" pattern="^sitehostname\." />
<!-- Enter your white listed IP addresses -->
<add input="{REMOTE_ADDR}" pattern="123\.123\.123\.1" negate="true"/>
<!-- Add the white listed IP addresses with a new condition as seen below -->
<!-- <add input="{REMOTE_ADDR}" pattern="123\.123\.123\.2" negate="true"/> -->
</conditions>
<action type="CustomResponse" statusCode="403" statusReason="Forbidden"
statusDescription="Site is not accessible" />
</rule>
what exactly your exe is? will someone invoke it and scheduled to run on a regular intervals?
we have couple of options here, if the processing logic is not a complex one you can go for Azure functions which uses WebJobs SDK itself but doesn't require an App Service to be configured for it.
or you can go for Azure Scheduler which can take your executable and run at a scheduled intervals.
I'm working on a project that includes an ASP.NET Web API site to be consumed by an Android and iOS app. The API is connected to a SQL Server database. I'm at a point where I want to publish my project to the Azure Cloud, but am trying to figure out what would be the best configuration.
The configuration needs to be scalable and reliable as the plans for this product will be accessed by many at the consumer level.
API Hosting
What do I use within Azure to host the API? Do I go with the Azure Web Site service? Or the Cloud service? or something else? I know I don't want to go Virtual Machine as I don't want to have a lot of maintenance.
One thing I should add here, I plan on having a Test/Qual environment as well as a Production environment in Azure.
SQL Database
This one seems easy for me, I will need to utilize the SQL Database service through Azure.
Scheduler
I have one final need of various jobs that need to run at night on some form of schedule. Would you agree that utilizing the Azure Scheduler service would be much cheaper than having either a Cloud service or Virtual Machine running scheduled tasks? Instead I could use Scheduler to utilize various web services at set times?
While Azure Web sites and Azure Web roles (cloud services) are very similar, the below are the most notable differences in my opinion:
Web Roles are cheaper than Web sites. There is a free offering for websites, but it has significant drawbacks.
Web sites are easier to manage and operate
You can deploy web sites from git.
You can use remote desktop to connect to Web roles.
You can use Worker Roles for background tasks from Web roles. Recently Websites introduced web roles which lessens the need for worker roles, note that this is as of today still in beta.
I wanted to host my WebApi project on azure. But I am not getting sure which way should i use to run it on azure. Like there are Websites, Cloud Services that contain Web role and Worker role. Then which one should i choose. If cloud service is the option then which one out of Web role and worker role is good?
Any help is appreciated.
For hosting a simple web API (that you can scale according to usage, etc.) you'll want to use Websites. Assuming you're not looking for more complex / heavy-weight features (network configuration, more complex architectures e.g. offloading background processing different instances via queueing mechanisms, RDP into the host machine, etc.), then Websites are becoming the de-facto way to host websites on Azure.
The following page from the Azure documentation will give you a full feature comparison between the two:
http://azure.microsoft.com/en-us/documentation/articles/choose-web-site-cloud-service-vm/, but in short, if you simply have a web API project in VS that you want to host in Azure without worrying about the underlying infrastructure, then use Websites.
If you only want to host a Headless (No Web Interface) Web API, I recommend you using Azure Web Service - Web Role.
For Worker Role, it is like a console application that you want to use it to process background task. Normally, we use it to process from Message Queue (Azure Service Bus - Queue).
Azure create a Optimize VM to run those two type of Roles with no unnecessary junks. So you will get the most of it.
However, I still suggest you to read more detail document from azure website to see what environment which is best fit for your long term plan.
You need to login in Azure (https://manage.windowsazure.com) -> Web Sites -> Create New
After that you will find the Name of the Web Site Under the Web Sites.
Click Web Sites -> Select New Created Web Site -> Go TO -> Deployement
In Deployement -> Find -> Integrate source control
Select Appropriate Option i.e GitHub or Dropbox etc..
Using Dropbox => Now Publish your WebApi Project and Paste that Data into Dropbox Folder With the same name of your New Created Web Site in Azure
After Upload Go To -> Azure -> Select Web Site -> Deployement -> Sync.
It will take all the data from Dropbox and You can run your WebApi Project From Azure
More Details Link, Link2, Link3
Below is the task i have to complete in a few weeks, my question is does anyone know of any tutorials or books which will be of use to me? I also only have little knowledge of C#.
You will design the relevant code in ASP.NET and C# so that you can
deploy a web role on Azure that can:
a) read in a zip file from input on the web page
b) uncompress the zip file which will have an executable, an
arbitrary list of arguments and data files
c) run the executable with the arguments (and hence read in the
data files).
d) store the output logs for access later.
You will test this by running a piece of code that reads in a set of
arguments which correspond to test files.
The executable will then read in the text files and then print them
out.
You will also have to design an appropriate UML to explain how the
different classes you have written interact.
You can deploy your service/role to Azure using Service Management API. Write a module which does the deployment and call this in your web.
Check these links that may help you.
Deploying An Azure Application
Publish Azure WebRole using Installer (like WIX)
About the Service Management API
The Windows Azure Accelerator for Web Roles makes it quick and easy
for you to deploy one or more websites across multiple Web Role
instances using Web Deploy. The accelerator includes a Visual Studio
project template that creates a Windows Azure web role to host one or
more websites.
Azure Tutorial; Be in cloud (Part 3) [Web Role]
Windows Azure and SQL Azure Tutorials - Tutorial 1: Using Windows Azure Web Role and Windows Azure Table Service
Check Azure in Action Book and it's content.
Hope these help you to get an idea that you want to implement.
That's hard homework: Upload a zip file to a cloud based server and run the .exe. Interesting...
Anyway, the Azure SDK is a great place to start. Also scan through Smarx's Cloud Cover shows. They cover pretty much everything.