I want to write a code, similar to the code at the bottom of this link (https://azure.microsoft.com/en-us/blog/automating-azure-analysis-services-processing-with-azure-functions/) in Visual Studio and building a DLL file. However instead of using the connection string, i would like to use an existing Linked Service from my Azure portal.
The goal is to create a DLL that refreshes my Cube, while at the same time using an existing Linked Service which is already in my Azure Portal.
Is this possible?
Thanks.
#r "Microsoft.AnalysisServices.Tabular.DLL"
#r "Microsoft.AnalysisServices.Core.DLL"
#r "System.Configuration"
using System;
using System.Configuration;
using Microsoft.AnalysisServices.Tabular;
public static void Run(TimerInfo myTimer, TraceWriter log)
{
log.Info($"C# Timer trigger function started at: {DateTime.Now}");
try
{
Microsoft.AnalysisServices.Tabular.Server asSrv = new Microsoft.AnalysisServices.Tabular.Server();
var connStr = ConfigurationManager.ConnectionStrings["AzureASConnString"].ConnectionString; // Change this to a Linked Service connection
asSrv.Connect(connStr);
Database db = asSrv.Databases["AWInternetSales2"];
Model m = db.Model;
db.Model.RequestRefresh(RefreshType.Full); // Mark the model for refresh
//m.RequestRefresh(RefreshType.Full); // Mark the model for refresh
m.Tables["Date"].RequestRefresh(RefreshType.Full); // Mark only one table for refresh
db.Model.SaveChanges(); //commit which will execute the refresh
asSrv.Disconnect();
}
catch (Exception e)
{
log.Info($"C# Timer trigger function exception: {e.ToString()}");
}
log.Info($"C# Timer trigger function finished at: {DateTime.Now}");
}
So I guess you're using the Data Factory and you want to process your analysis services model from your pipeline. I don't see what your question actually has to do with the Data lake store.
To trigger Azure Functions from the Data Factory (v2 only), you'll have to use a web activity. It is possible to pass a Linked Service as part of your payload, as shown in the documentation. It looks like this:
{
"body": {
"myMessage": "Sample",
"linkedServices": [{
"name": "MyService1",
"properties": {
...
}
}]
}
However, there is no Analysis services linked service in the Data Factory, at least, I didn't hear of such a thing. Passing in a connectionstring from the pipeline seems like a good idea however. You could pass it as a pipeline parameter in your body of the webrequest.
Create a parameter in your pipeline
Add it to your Web Activity Payload
{
"body": {
"AzureASConnString": "#pipeline().parameters.AzureASConnString"
}
You can retrieve this value from functions like described here
Related
We want to create an azure function in c# that retrieve the list of azure web app contained in the subscription (basically we want to call dynamically, for each webapp, the same API endpoint changing the subdomain of the api).
It's possible with c# retrieve the list of the web app contained in the same azure function subscriptions?
Usually we connect to the master database, we query the sys.databases to collect the dbname and understand the webapp names. But we are searching for a smartest way.
If you're in C# land, I'd look at using the ArmClient class to retrieve what you're looking for.
Install these (I've got a few others installed but start with that and see how you go, there may be a couple of others needed) Nuget packages ...
Azure.Identity;
Azure.ResourceManager;
Azure.ResourceManager.AppService
... and from there, using the DefaultCredential approach (if you've never used it, read up on it here -> https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/README.md) you can query your subscriptions webApps ...
using Azure.Identity;
using Azure.ResourceManager;
using Azure.ResourceManager.AppService;
using System;
using System.Threading.Tasks;
namespace AzureManagement
{
internal class Program
{
static void Main(string[] args)
{
GetAzureResources().Wait();
}
static async Task GetAzureResources()
{
var credential = new DefaultAzureCredential();
var armClient = new ArmClient(credential);
var subscription = await armClient.GetDefaultSubscriptionAsync();
var webSitesEnumerator = subscription.GetWebSitesAsync().GetAsyncEnumerator();
try
{
while (await webSitesEnumerator.MoveNextAsync())
{
var webSite = webSitesEnumerator.Current;
Console.WriteLine($"Web App Name ........ {webSite.Data.Name}");
Console.WriteLine($"Default Host Name ... {webSite.Data.DefaultHostName}\n");
}
}
finally
{
await webSitesEnumerator.DisposeAsync();
}
Console.ReadLine();
}
}
}
The above is obviously not a function app but the core code will still work for you and can be ported as need be.
Note: I could be telling you how to suck eggs, but, once deployed to Azure, you'll need to do the necessary work to ensure that the function app has the required access to retrieve all of the resource information you're looking for.
If you're unfamiliar with that, read up on the managed identity concept. It's very easy to setup -> https://learn.microsoft.com/en-us/azure/app-service/overview-managed-identity
Yes, one easy way is to use HttpClient and send a request to Azure Rest API:
GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Web/sites?api-version=2022-03-01
https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/list
PS: you first need to acquire an authentication token.
https://www.youtube.com/watch?v=6b1J03fDnOg&t=329s
I have a console application which I want to convert to an Azure Function Timer Trigger app which will run every hour after some data processing and uploads are done. The data processing and uploads are being done via classes which are injected in the program.cs file of the console application. Somewhere in the classes I have a task.delay by 1hour where it will query new data after the data has been queried and uploaded for the first time. So, I copied the entire code of the console application with its packages to the Azure Function Timer trigger app. What I am trying to do is to run the program.cs file of the console application first in the azure function app in order to do its job (data processing, querying data, uploading data to azure...). and then initiate the timer trigger. Is that doable ? What line of code can I add in the run method of the azure function app to execute the program.cs file first and then initiate the trigger. You can find here the startup code of the azure function time trigger app.
using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
namespace ExportServiceFunctionApp
{
public static class ExportServiceFunctionApp
{
[FunctionName("ExportServiceFunctionApp")]
public static void Run([TimerTrigger("0 0 */1 * * * ")]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
}
}
}
There are a few solutions to achieve this.
Solution 1. Temporarily replace the timer trigger with http trigger
While debugging the app we just comment the first line of the original Run function and add an http trigger instead like this:
public static async Task Run([HttpTrigger] Microsoft.AspNetCore.Http.HttpRequest req, ILogger log)
// public static async Task Run([TimerTrigger("0 0 * * * *")] TimerInfo myTimer, ILogger log)
{
// YOUR REGULAR CODE HERE
}
Then when running the app you'll see an endpoint like this:
Just open the endpoint in browser (or postman) and the function will get called.
And right before pushing the code to the repo, just bring back the original Run function and remove the http trigger one.
Solution 2: Add another http trigger that calls the timer function
Add the following function to your app to expose an http trigger.
[FunctionName("Test")]
public static async Task Test([HttpTrigger] Microsoft.AspNetCore.Http.HttpRequest req, ILogger log)
{
Run(null, log);
}
The function basically calls the Run function.
So when you run the app, again you'll get an endpoint from the console that can be used from the browser to trigger the function.
The url will look like this:
http://localhost:7071/api/Test
Azure functions is event driven in nature. If function trigger means the event handled.
Run method of function means that function has triggered and its entry point for it.
If you want any processing or code execution before it you may need to write one more function and perform the steps and then trigger another function of either timer trigger or ant different type.
Is there now a way to set the Trigger Properties(Name/Connection) using the value from Azure App Configuration?.
I added a startup class that reads the data from Azure App Configuration but it seems the trigger set its properties earlier than that, therefore not able to bind the data that came from the app configuration.
I also found this thread about it but im not sure if there is a new update?:
https://github.com/MicrosoftDocs/azure-docs/issues/63419
https://github.com/Azure/AppConfiguration/issues/203
You can do this. The following code gets the name of the queue to monitor from an app setting, and it gets the queue message creation time in the insertionTime parameter:
public static class BindingExpressionsExample
{
[FunctionName("LogQueueMessage")]
public static void Run(
[QueueTrigger("%queueappsetting%")] string myQueueItem,
DateTimeOffset insertionTime,
ILogger log)
{
log.LogInformation($"Message content: {myQueueItem}");
log.LogInformation($"Created at: {insertionTime}");
}
}
Similarly, you can use this approach for other triggers.
I have a .Net Core Web MVC Application and I want to send a notification to a client with Azure SignalR, when in CosmosDB the change feed is triggered.
FeedToSignalR trigger on new data in CosmosDB and broadcast it through SignalR to a clients.
SignalRConfiguration initialize the SignalR Websocket connection.
The problem is I don’t know how I can call this methods.
Can I call the methods in my Program.cs or in the Startup.cs?
public static class SignalRConfiguration
{
private static AzureSignalR signalR = new AzureSignalR(Environment.GetEnvironmentVariable("AzureSignalRConnectionString"));
/// <summary>
/// This HttpTriggered function returns the SignalR configuration to the web client.
/// </summary>
[FunctionName("SignalRConfiguration")]
public static HttpResponseMessage Run([HttpTrigger(AuthorizationLevel.Anonymous)]HttpRequestMessage req, TraceWriter log)
{
return req.CreateResponse(HttpStatusCode.OK,
new {
hubUrl = signalR.GetClientHubUrl("cosmicServerlessHub"),
accessToken = signalR.GenerateAccessToken("cosmicServerlessHub")
});
}
}
public static class FeedToSignalR
{
private static AzureSignalR signalR = new AzureSignalR(Environment.GetEnvironmentVariable("AzureSignalRConnectionString"));
/// <summary>
/// This function Triggers upon new documents in the Cosmos DB database and broadcasts them to SignalR connected clients.
/// </summary>
[FunctionName("FeedToSignalR")]
public static async Task Run([CosmosDBTrigger(
databaseName: "ToDoList",
collectionName: "Items",
ConnectionStringSetting = "AzureCosmosDBConnectionString",
LeaseConnectionStringSetting = "AzureCosmosDBConnectionString",
CreateLeaseCollectionIfNotExists = true,
LeaseCollectionName = "leases")]IReadOnlyList<Document> documents, TraceWriter log)
{
if (documents != null && documents.Count > 0)
{
var broadcast = documents.Select((d) => new
{
id = d.GetPropertyValue<string>("id"),
price = d.GetPropertyValue<string>("price")
});
await signalR.SendAsync("cosmicServerlessHub", "NewMessages", JsonConvert.SerializeObject(broadcast));
}
}
}
These are two Functions that each get triggered by different events.
Your FeedToSignalR is using the Cosmos DB Trigger which will trigger itself automatically when a new document is added in the Items collection as described in the comment above the function's code.
SignalRConfiguration on the other hand is using an HTTP Trigger which gets called whenever there is an HTTP call to it's HTTP endpoint.
If I recognize the code correctly, it is coming from https://github.com/ealsur/serverlessnotifications
Based on the explanation:
The browser will do an HTTP call to the SignalRConfiguration function to get the configuration
When a new message is typed, it will save that message in Cosmos DB by invoking another Function.
That new document in the collection, will then trigger the Cosmos DB Trigger Function who will in turn send it through Signal R.
Yes, this is exactly the code. I think the problem is that I don't have the json “function.json” like here "https://learn.microsoft.com/de-de/azure/azure-functions/functions-bindings-cosmosdb-v2#trigger---c-script-example" to bind the Cosmos DB Trigger and the other json to bind HTTP Trigger.
This “https://anthonychu.ca/post/cosmosdb-real-time-azure-functions-signalr-service/” example does exactly what I want, but Mr. Chu used the .Net Standard, I need do this in .Net Core MVC or better .Net Framework MVC. On the webpage is a picture of the architecture, that describes what I want to do, I tried do reproduced the Step 3 to 5 with your example.
The only thing that I need to do is, to put the two json files in my project to bind the HTTP Trigger and the Cosmos DB Trigger? How the .Net Core project know to call that json files?
I'm trying to find what I'm doing wrong regarding an Azure WebJobs QueueTrigger method that should be triggered from an Azure Storage Queue.
I've read a couple of documents (as in blog posts / msdn articles). But I'm still not clear.
Main question / misunderstood aspect:
What should be the name of the connection string for Azure storage console app App.config or Windows Azure Configuration (portal). So far I have the following name set at both places.
AzureJobsStorage
AzureWebJobsStorage
AzureJobsRuntime
AzureJobsDashboard
AzureJobsData
Here's my WebJobs console app code.
static void Main()
{
JobHost host = new JobHost();
host.RunAndBlock();
}
public static void CreateLeague([QueueTrigger("temp")] string msg)
{
var task = JsonConvert.DeserializeObject<QueueTask>(msg);
if (task.TaskType == QueueTask.TaskTypes.Pdf)
RenderPdf(task.Id);
}
This console app is continuously running on my Azure Website.
I can access its "debug" page where I can toggle output and I see it is started / running.
My code to add queue (from my ASP.NET MVC app):
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConfigurationManager.AppSettings["StorageConnectionString"]);
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference("temp");
queue.CreateIfNotExists();
Common.QueueTask task = new Common.QueueTask();
task.TaskType = Common.QueueTask.TaskTypes.Pdf;
task.Id = p.Id;
CloudQueueMessage msg = new CloudQueueMessage(JsonConvert.SerializeObject(task) );
queue.AddMessage(msg);
This code is executed, and queue are added to my Storage Account. But they did not get "dequeue" or read from the WebJobs.
Hmm, the WebJobs class had to be public.
using Microsoft.Azure.WebJobs;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using Newtonsoft.Json;
using Proceed.Common;
using System;
using System.Configuration;
using System.IO;
public class WebJobsTask {
public static void Main()
{
JobHost host = new JobHost();
host.RunAndBlock();
}
public static void CreateLeague([QueueTrigger("temp")] string msg)
{
var task = JsonConvert.DeserializeObject<QueueTask>(msg);
if (task.TaskType == QueueTask.TaskTypes.Pdf)
RenderPdf(task.Id);
}
}
Also found a quick way to explore my queues: https://azurestorageexplorer.codeplex.com/.
In my case, I had assumed that QueueTrigger was referring to Service Bus Queues instead of Azure Queues, and I actually needed to use ServiceBusTrigger.
You can use the server explorer in VS to explore the content of the Storage queues.
The queue triggers for the WebJobs SDK will exponentially back off if there is no work to do. There might be a delay between the moment a message is put in a queue and the moment when it is picked up. You can configure the maximum back off through the JobHostConfiguration.Queues.MaxPollingInterval property.
For the latest SDK you need two storage connection strings AzureWebJobsStorage and AzureWebJobsDashboard
This is a great place for more resources: https://learn.microsoft.com/en-us/azure/app-service-web/websites-webjobs-resources