Azure kafka trigger exposing Autooffset property in KafkaOptions - c#

I am a beginner to Azure kafka trigger and learning how to use it.
Can someone kindly give a sample of how to use the KafkaOptions.AutoOffsetReset when using the Azure Kafka trigger.
I have truly looking for a example but couldn’t find any?
I cannot add the property in kafka trigger but i need the property in my Azure kafka trigger. How do i add it?

Based on this documentation, you can add this configuration in your host.json file:
{
"version": "2.0",
"extensions": {
"kafka": {
"maxBatchSize": 100,
"auto.offset.reset": "latest"
}
}
}
Check some samples here.
The KafkaTriggerAttribute itself does not expose an autooffset property.

Related

Durable Function Doesn't Correlate Telemetry

According to this developer blog post
Durable functions can be configured (and according the the host.json schema are configured by default) to automatically create and correlate trace telemetry with the W3C tracing protocol.
Currently, my host.json looks like this, and includes the properties shown in the blog.
{
"version": "2.0",
"logging": {
"applicationInsights": {
"httpAutoCollectionOptions": {
"enableW3CDistributedTracing": true
},
"samplingExcludedTypes": "Request",
"samplingSettings": {
"isEnabled": true
}
},
"logLevel": {
"default": "Information",
"blueprints_webcp_gateway": "Debug"
}
},
"functionTimeout": "00:10:00",
"extensions": {
"queues": {
"visibilityTimeout": "00:00:10",
"batchSize": 4
},
"durableTask": {
"tracing": {
"DistributedTracingProtocol": "W3CTraceContext"
}
}
}
}
Yet, when I go to the Activity Log page in Azure Portal, each orchestrator and activity shows up as its own operation, which is definitely not what I'm expecting to see. Is any additional configuration needed? As far as I can tell, telemetry logged by the durable function host should be correlated by default.
I face same issue and have exactly same results(durable functions traces have different OperationId). Have not found the solution in bigger project I am working on. We have created smaller project to try full end-to-end functionality in more clear environment. I found out that some packages may break the correlation of DF traces f.e. (System.DiagnosticTool >= 4.7.0, MS.Azure.ServiceBus - this SDK is whole deprecated, ...) But there is a light in the end of the tunnel. When I have upgraded each package, target framework and version of Azure functions everything was working correctly in our smaller project. To update everything and make it work in bigger project will be another challenge.

Disable Azure Function from C# for runtime version 3.x?

How can I disable an Azure Function using code in C#?
I'm using Azure Functions Runtime version 3.x
I'm implementing a distributed circuit-breaker inspired by Serverless circuit breakers with Durable Entities. When the circuit opens I need to disable a queue-trigged Azure Function, instead of stopping the entire function app.
I see from How to disable functions in Azure Functions that the recommended way to disable a function is to set the AzureWebJobs.<FUNCTION_NAME>.Disabled app setting. But I haven't found an API for doing that in C#. I'm hoping there is something that I can call from my C# code that is equivalent to the Azure CLI's az functionapp config appsettings set command.
I saw similar questions on SO like:
azure set environment variable programmatically to disable an azure function
and How to Enable/Disable Azure Function programmatically
But those have answers from back in 2017 that use kudu APIs to change the disabled property in the function.json file, and I'm hoping that there is a better way to do that now. Especially because the Docs at How to disable functions in Azure Functions say:
The generated function.json file for a class library function is not
meant to be edited directly. If you edit that file, whatever you do to
the disabled property will have no effect.
Unfortunately I was not able to find any documentation as such. The closest I got was
https://learn.microsoft.com/en-us/rest/api/appservice/webapps/createfunction
For instance to create the function :
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/sites/{name}/functions/{functionName}?api-version=2019-08-01
However this documentation also did not take me near to your requirement of updating the Config File. Or I may have overlooked few modules. Request you check further before implementing the below steps
So here's is what I did, I was kind of trying to reverse engineer, I ran the commands in Azure CLI and captured the traces - my thought process - the Azure CLI internally run on python and issues the API request to the Azure.
Ran the below command and captured Fiddler :
az functionapp config appsettings set --name <myFunctionApp> \
--resource-group <myResourceGroup> \
--settings AzureWebJobs.QueueTrigger.Disabled=true
And Yes ! The python process was issuing request to https://management.azure.com to update appsetting :
The set property is sent in the Request Body :
We can hardcode the properties or get it dynamically.
So I ran the below Azure CLI command
az functionapp config appsettings list --name <> --resource-group <>
I was able to see the above properties that was passed along the PUT request
Took the fiddler for the above command
Saw there is a POST Request to the below endpoint :
https://management.azure.com/subscriptions//resourceGroups//providers/Microsoft.Web/sites//config/appsettings/list?api-version=2019-08-01
These are the same set of property bags which are sent as the request bodies in the PUT in order to set the property.
So in your case you will have to request the above end point to get the list of properties. It is json output. Update the value of AzureWebJobs.QueueTrigger.Disabled to True.
Issue the Updated properties using the PUT method along with the headers such as Bearer Token & Content-Type: application/json; charset=utf-8
Request URI :
https://management.azure.com/subscriptions//resourceGroups//providers/Microsoft.Web/sites//config/appsettings?api-version=2019-08-01
Headers :
Authorization: Bearer <> Content-Type: application/json;
charset=utf-8`
Request Body:
{"kind": "<class 'str'>", "properties": }
I hope you will be able to achieve your requirement.
I hope this helps you :)
I don't recommend this for your prod. Pls try and monitor in your Dev env.

Can I configure a default value for an Azure Functions binding expression?

I want to simplify deployment of my Azure Functions v3 project as much as possible.
As an example, take this function, which uses a binding expression to refer to the DoTheThingSchedule appsetting:
[FunctionName("DoTheThing")]
public async Task Run([TimerTrigger("%DoTheThingSchedule%")]TimerInfo timer)
{
// do the thing
}
For local development I can configure a value for DoTheThingSchedule in local.settings.json, but I haven't found a way to configure a default value in production, which means I'll always have to explicitly configure this setting in Azure.
Does anyone know of a way to work around this?
Azure needs you to add settings in configuration settings in azure. And it will not upload local.settings.json file to azure. But if you are using VS to publish, you can set the settings before publish:

Debug an Azure Function that is triggered using an Event Grid

I have an Azure Function (C#) that is triggered by an Event Grid. I am having difficulty debugging my Function locally as it can only be triggered via the Event Grid.
I'm not sure if there is a way to wire the Event Grid up to my local host address for my Function? I have tried using a Web Hook but it only support HTTPS which my localhost uses HTTP.
public static void Run([EventGridTrigger]EventGridEvent eventGridEvent, ILogger log)
{
ID = eventGridEvent.Subject;
log.LogInformation($"ID: {ID.ToString()}");
}
Any advice is appreciated.
I found this blog post that helped me resolve this when debugging event grid azure function locally: https://blog.mcilreavy.com/articles/2018-12/debug-eventgrid-triggered-azure-function (Web archive link)
One piece that was missing in my case was the aeg-event-type header.
aeg-event-type: Notification
After adding it, I was able to debug locally using this url:
http://localhost:7071/runtime/webhooks/EventGrid?functionName=nameOfYourFunction
And include this event json in the request body, something like this:
[ { "id": "'1", "eventType": "yourEventType", "eventTime":"10:59:00.000", "subject": "subject1", "data":{"make": "Ducati", "model": "Monster"}, "dataVersion": "1.0" } ]
Actually, there is a nice and easy way of doing this using a ngrok as a tunnel. So you basically create a EventGrid Subscription that publishes your events to your ngrok tunnel (e. g. https://ec291dd9.ngrok.io/runtime/webhooks/EventGrid?functionName=myfunc) and start debugging your function in Visual Studio. You don't need to change your function at all.
This approach is also well documented from Microsoft: Azure Function Event Grid Trigger Local Debugging
You're right. At this moment, tooling for Event Grid is being improved and there's no easy way to test it locally.
In my Event Grid Functions, I'm separating the logic from the trigger part, this way I can unit test the logic without the need of a real event in order to test it. After that, I'm capturing the event through Application Insights and a log.LogInformation(eventGridEvent.Data.ToString());

Azure Cloud Service - Monitoring a deployment

Is there an event raised the moment a website is published and updated?
I've tried Application_End in global.asax but that event does not seem to be raised.
I suggest to use both Kudu and Microsoft ASP.NET WebHooks preview.
Kudu is the engine behind git deployments, WebJobs, and various other features in Azure Web Sites (Kudu source is on GitHub)
With Kudu, Azure Web Sites have a support for web hooks. There is an event "PostDeployment" that will be invoked whenever a deployment is complete with the result of that deployment.
Microsoft ASP.NET WebHooks preview provides a common model for receiving and processing WebHooks from any number of WebHook providers includign support for Kudu (Azure Web App Deployment).
So you may use Kudu WebHooks to get notified when an update has been deployed. (But that will require using Git Deploy instead of other ways to publish your web site).
Here is the way to do it :
First Install the Microsoft.AspNet.WebHooks.Receivers.Azure Nuget package.
The, Add these two lines to the WebApiConfig.Register method:
config.InitializeReceiveKuduWebHooks();
config.InitializeReceiveAzureAlertWebHooks();
Then Add a handler :
public class KuduWebHookHandler : WebHookHandler
{
public KuduWebHookHandler()
{
Receiver = "kudu";
}
public override Task ExecuteAsync(string generator, WebHookHandlerContext context)
{
// Convert to POCO type
KuduNotification notification = context.GetDataOrDefault<KuduNotification>();
// Get the notification message
string message = notification.Message;
// Get the notification author
string author = notification.Author;
return Task.FromResult(true);
}
}
Then configure a secret that can validates that the WebHook requests indeed come from Kudu.
Use high-entropy values such as a SHA256 hash or similar, which you can get from http://www.freeformatter.com/hmac-generator.html. Also, set them through the Azure Portal instead of hard-coding them in the Web.config file
Also, set them through the Azure Portal instead of hard-coding them in the Web.config file.
There is a more complete Post on the subject here (http://blogs.msdn.com/b/webdev/archive/2015/10/04/receive-webhooks-from-azure-alerts-and-kudu-azure-web-app-deployment.aspx)
Hope this helps
Best regards
Stéphane
I figured it out. In the Application_Start event I bind
RoleEnvironment.Stopping += RoleEnvironmentStopping;
private void RoleEnvironmentStopping(object sender, RoleEnvironmentStoppingEventArgs e)
{
// do something ...
}

Categories