I have a KafkaClusterOptions class in my Azure Functions project that contains the configuration values for connecting to a Kafka cluster:
public class KafkaClusterOptions
{
[ConfigKey("MP/Kafka/BootstrapServer")]
public string BootstrapServer { get; set; }
[ConfigKey("MP/section/Kafka/SaslUsername")]
public string SaslUsername { get; set; }
[ConfigKey("MP/section/Kafka/SaslPassword")]
public string SaslPassword { get; set; }
}
I also have a KafkaTrigger function that needs to connect to the Kafka cluster:
public Task KafkaTrigger(
[KafkaTrigger(
BootstrapServer = "<bootstrap-server>",
TopicName = "<topic-name>",
ConsumerGroup = "<consumer-group>",
SslCaLocation = "confluent_cloud_cacert.pem",
Protocol = BrokerProtocol.SaslSsl,
AuthenticationMode = BrokerAuthenticationMode.Plain,
Username = "<sasl-username>",
Password = "<sasl-password>",
IsBatched = true)]
string[] kafkaEventsStr,
FunctionContext context)
{
// Process Kafka events
}
I would like to pass the configuration values from the KafkaClusterOptions class to the KafkaTrigger function instead of hard-coding them in the attribute. Can someone provide an example of how to inject the KafkaClusterOptions instance into the KafkaTrigger function using dependency injection, and how to use the configuration values in the KafkaTrigger attribute?
Thank you!
Related
Using C#, the Windows Form Template from Visual Studio and the Microsoft Azure Client SDK
I'm trying to send a message to an IOT Hub. I tested connection String with a Virtual Pi and it works there, I can see the connection and incoming messages in the Azure Shell.
Imports:
using System.Windows.Forms;
using Microsoft.Azure.Devices;
using Message = Microsoft.Azure.Devices.Message;
static string connectionString = "HostName=****.azure- devices.net;DeviceId=****;SharedAccessKey=*******=";
static string deviceId = "SensorTest";
static ServiceClient serviceClient;
serviceClient = ServiceClient.CreateFromConnectionString(connectionString);
SendMessageToCloud("takemystringyoupieceof***").Wait();
private static async Task SendMessageToCloud(string s)
{
MyData data = new MyData
{
Thing1 = "string",
Thing2 = "string",
Thing3 = 1234,
Thing4 = "string",
Thing5 = s
};
var serializeData = JsonConvert.SerializeObject(data);
var commandMessage = new Message(Encoding.ASCII.GetBytes(serializeData));
await serviceClient.SendAsync(deviceId, commandMessage);
}
This throws an Inner Exception:
{"Put token failed. status-code: 401, status-description:
Unauthorized: When KeyName is empty the resource URI must include a device id (Value '****.azure- devices.net').."}
System.Exception {Microsoft.Azure.Devices.Common.Exceptions.UnauthorizedException}
I would like some help understanding the error message:
Isn't KeyName "SharedKey" in this instance?`
The deviceID is included as you can see above?
The Value in the error message is the value for hostname?
For working code example, myData class:
internal class MyData
{
public string Thing1 { get; set; }
public string Thing2 { get; internal set; }
public int Thing3 { get; set; }
public string Thing4 { get; set; }
public string Thing5 { get; set; }
}
You are using a DeviceClient connection string with the ServiceClient. The SendAsync you are calling in your SendMessageToCloud is actually a Cloud to Device API. So depending on your intention, the answer is you either need to use the DeviceClient (a helpful sample can be found here) or you want to use an appropriate Service connection string which can be found in the Shared access policies blade for the IoTHub in the portal. Practicing least privilege, the Service key allows cloud-to-device messages (details on shared access policies can be found here)
So I have a simple configuration class PubsubSettings:
public class PubSubSettings
{
public string ProjectId { get; set; }
public string TopicId { get; set; }
public int PartnerId { get; set; }
public string SubscriptionId { get; set; }
}
I have previously only had one of these configured in my appsettings.json but now I want to be able to handle an arbitrary number of them.
I have another class, PubSub, that I usually inject an IOptions<PubSubSettings> into. And this, in turn, gets injected into my Worker class.
services.Configure<PubSubSettings>(configuration.GetSection(nameof(PubSubSettings)));
...
services.AddHostedService<Worker>();
So, what I want to do now, is add a new Worker as a hosted service for each entry in my AppSettings PubSubSettings section and inject the relevant IOptions<PubSubSettings> into each of these (along with the standard ILogger).
So in essence, I'd like this config block:
"PubsubSettings": [
{
"ProjectId": "project1",
"TopicId": "topic",
"PartnerId": 1,
"SubscriptionId": "sub1"
},
{
"ProjectId": "project2",
"TopicId": "topic2",
"PartnerId": 2,
"SubscriptionId": "sub2"
}
]
To end up with two hosted services being created, one with the first set of options and the other with the second.
I've seen a few questions looking for similar things but nothing I could find quite lined up with this so I'm a bit stumped. Any ideas?
The solution is Dotnet 5.
So from what I've been able to find, there's no way to do this out-of-the box.
However, This can be done manually using a combination of ActivatorUtilities and Configuration.Bind().
private void CreateWorkers(IServiceCollection services, IConfigurationRoot configuration)
{
List<PubSubSettings> pubsubSettings = new();
configuration.Bind(nameof(PubSubSettings), pubsubSettings);
foreach (PubSubSettings setting in pubsubSettings)
{
services.AddSingleton<IHostedService>(s => ActivatorUtilities.CreateInstance<Worker>(s, ActivatorUtilities.CreateInstance<PubSub.PubSub>(s, setting)));
}
}
Essentially, you can use Bind to get the configuration objects from the JSON. Then you can manually construct the Worker for the call to AddHostedService using CreateInstance.
Two calls are needed in this case, one to generate the PubSub for the worker (in which we pass the setting parameter) and the other to generate the Worker itself.
ActivatorUtilities essentially injects everything you need for the object except the parameters you've provided.
We need to use .AddSingleton<IHostedService> because of the way that the framework checks for dupes with AddHostedService().
Maybe you could try creating a class only for the object and let the PubSubSettings class only for the array:
public class PubSubSettings
{
public PubSubObject[] PubSubs { get; set; }
}
public class PubSubObject
{
public string ProjectId { get; set; }
public string TopicId { get; set; }
public int PartnerId { get; set; }
public string SubscriptionId { get; set; }
}
Then in the startup class you should use Bind to get the current value of the array to create a Worker for each PubSub:
PubSubSettings settings = new PubSubSettings();
Configuration.GetSection(nameof(PubSubSettings)).Bind(settings);
...
foreach(PubSubObject item in settings.PubSubs)
{
services.AddHostedService<Worker>();
}
Then in the PubSub class you need to search the PartnerId inside the Array.
Or you could follow the approach described in the section Named options support using IConfigureNamedOptions in the Microsoft docs: Options pattern in ASP.NET Core
I currently have a class with around 40 dependency injection. It is a hard to maintain and unit test. I am not sure a good way around.
The code is done for any type of application process that is needed to process (New License, License Renewal, Student Registration, ...), there are around 80 different types applications and what sections are associated with each application type is determined by a database table.
I have a class with all of the possible properties, there are a several more than listed but you should get the idea. Each the properties have their own set of properties that are basic data types or object pointing to other classes.
class Application
{
[JsonProperty(PropertyName = "accounting")]
public Accounting Accounting { get; set; }
[JsonProperty(PropertyName = "application")]
public Application Application { get; set; }
[JsonProperty(PropertyName = "applicationType")]
public ApplicationType ApplicationType { get; set; }
[JsonProperty(PropertyName = "document")]
public List<Attachment> Document { get; set; }
[JsonProperty(PropertyName = "employment")]
public List<Employment> Employment { get; set; }
[JsonProperty(PropertyName = "enrollment")]
public Enrollment Enrollment { get; set; }
[JsonProperty(PropertyName = "individualAddressContact")]
public IndividualAddressContact IndividualAddressContact { get; set; }
[JsonProperty(PropertyName = "instructors")]
public List<Instructor> Instructors { get; set; }
[JsonProperty(PropertyName = "license")]
public License License { get; set; }
[JsonProperty(PropertyName = "licenseRenewal")]
public LicenseRenewal LicenseRenewal { get; set; }
[JsonProperty(PropertyName = "MilitaryService")]
public List<MilitaryService> MilitaryService { get; set; }
[JsonProperty(PropertyName = "paymentDetail")]
public PaymentDetail PaymentDetail { get; set; }
[JsonProperty(PropertyName = "photo")]
public List<Attachment> Photo { get; set; }
[JsonProperty(PropertyName = "portal")]
public Portal Portal { get; set; }
[JsonProperty(PropertyName = "section")]
public List<Section> Section { get; set; }
[JsonProperty(PropertyName = "testingCalendar")]
public TestingCalendar TestingCalendar { get; set; }
[JsonProperty(PropertyName = "testingScore")]
public List<TestingScore> TestingScore { get; set; }
[JsonProperty(PropertyName = "USCitizen")]
public USCitizen USCitizen { get; set; }
}
So this class is sent/received to an Angular 10 front end using Web API's.
When an application is requested the sections and the different properties are initiated and if the application has be started the progress will be reloaded. So it is possible some of properties will be pulled from the database and sent to the Angular app.
So I have something such as
Load(applicationTypeId, applicationId)
{
Get the sections for the application type
For each section in the sections
switch sectionid
case Documents
Load all of the documents required for the application type and get any documents uploaded
case Accounting
Load the payment details, if no payment made calculate the payment
case IndividualAddressContact
Load the person name/address/contact and set a few defaults if the person hasn't started.
.....
next
}
Save()
{
Save the application
switch current section
case Documents
Save all of the documents for the application
case Accounting
Save the payment details for the application
case IndividualAddressContact
Save the person name/address/contact for the application
.....
get the next section
Update the application current section
}
I have put all of the items in the switch into their own classes but in the end I still have 1 point for serialization/deserialization and still end up with to many dependencies injected. Creating a unit test with over 40 dependencies seems hard to maintain and given I won't know which properties will/won't used until an application is requested and loaded from database. I am unsure how to get around the switch, without at some point and time having to have all of the dependencies injected into 1 class.
I would appreciate some ideas of how to get around this.
"I currently have a class with around 40 dependency injection..." - Oh my gosh!
"It is a hard to maintain and unit test..." - I don't doubt that in the least!
SUGGESTED REFACTORING:
Create a class that manages "Applications" (e.g. "ApplicationManager").
Create an abstract class "Application".
One advantage of "abstract class" over "interface" here that you can put "common code" in the abstract base class.
Create a concrete subclass for each "Application" : public class NewLicense : Application, public class LicenseRenewal : Application, etc. etc.
... AND ...
Use DI primarily for those "services" that each concrete class needs.
I'll bet the constructors for your individual concrete classes will only need to inject three or four services ... instead of 40. Who knows - maybe your base class won't need any DI at all.
This is actually a design we're actually using in one of our production systems. It's simple; it's robust; it's flexible. It's working well for us :)
I would recommend using convention over configuration principle, with the Service Locator.
Declare something like IApplicationHandler interface in your program, e.g.
public interface IApplicationQueryHandler
{
Application Populate(Application application);
}
public interface IApplicationSaveHandler
{
Bool Save(Application application);
}
Then, write pieces of your code, with dependencies and such, e.g.
public class AccountingApplicationQueryHandler : IApplicationQueryHandler
{
public Application Populate(Application application) {
//// Load the payment details, if no payment made calculate the payment
return application;
}
}
public class AccountingApplicationSaveHandler : IApplicationSaveHandler
{
public Bool Save(Application application) {
//// Save the payment details for the application
return true; // this just flags for validation
}
}
// repeat for all other properties
Then in your controller, do something like
public class ApplicationController: Controller
{
public readonly IServiceProvider _serviceProvider;
public ApplicationController(IServiceProvider sp) {
_serviceProvider = sp;
}
public Application Load(string applicationTypeId, string applicationId)
{
var application = new Application(); // or get from db or whatever
var queryHandlers = _serviceProvider.GetServices(typeof(IApplicationQueryHandler));
foreach(var handler in queryHandlers) {
application = handler.Populate(application);
}
return application;
}
[HttpPost]
public bool Save(Application application)
{
var result = true;
var saveHandlers = _serviceProvider.GetServices(typeof(IApplicationSaveHandler));
foreach(var handler in queryHandlers) {
result = handler. Save(application);
}
return result;
}
}
You would need to register your handlers, which you can do e.g. like so:
var queryHandlers = Assembly.GetAssembly(typeof(IApplicationQueryHandler)).GetExportedTypes()
.Where(x => x.GetInterfaces().Any(y => y == typeof(IApplicationQueryHandler)));
foreach(queryHandler in queryHandlers) {
services.AddTransient(typeof(IApplicationQueryHandler), queryHandler);
}
// repeat the same for IApplicationSaveHandler
Now finally, you can write unit tests for part of the code like so
[TestClass]
public class AccountingApplicationQueryHandlerTests
{
[TestMethod]
public void TestPopulate()
{
// arrange
var application = new Application();
var handler = new AccountingApplicationQueryHandler(); // inject mocks here
// act
var result = handler.Populate(application);
// Assert
Assert.AreEqual(result. PaymentDetail, "whatever");
}
}
And you can test that your controller calls the right things by mocking IServiceProvider and injecting that with a couple of dummy handlers to confirm they are called correctly.
Following zaitsman's answer you also could create AggregatedApplicationQueryHandler and AggregatedApplicationSaveHandler and pass collection of concrete implementation of IApplicationQueryHandler and IApplicationSaveHandler to its constructor.
Then you don't need foreach loop inside controller(you loop over handlers inside aggregated handler) and always have only one handler passed to controller. Passing its by constructor parameter shouldn't be so much painful.
You also could create facade over some small services and aggregate theirs functions into one bigger facade service.
I am using standard configuration pattern for ASP.NET Core applications and I can not bind configuration to my class as it has construtor with parameters.
In appsettings.json I included desired config:
"MyServiceConfig": {
"Identity": {
"Version": "1.0",
"ComplicatedUri": {
"Scheme": "http",
"Authority": "localhost",
"Path": "SuperService"
}
}
},
My config class and it's dependencies look like that:
public class MyServiceConfig
{
public MyIdentity Identity { get; set; }
}
public class MyIdentity
{
public string IdentityName { get; set; }
public string Version { get; set; }
public MyComplicatedUri ComplicatedProperty { get; set; }
public MyIdentity(string version, MyComplicatedUri complicatedProperty)
{
Version = version;
ComplicatedProperty = complicatedProperty;
IdentityName = complicatedProperty.Path;
}
}
public class MyComplicatedUri
{
public string Scheme { get; set; }
public string Authority { get; set; }
public string Path { get; set; }
}
I have already tried code like that:
private MyServiceConfig GetMyConfig(IConfiguration configuration)
{
var config = new MyServiceConfig();
configuration.GetSection("MyServiceConfig").Bind(config);
return config;
}
It throws exception:
'Cannot create instance of type 'MyIdentity' because it is missing
a public parameterless constructor.'
That behaviour can make sense in some cases but in that particular one not so much. Mappings could be straightforward - by property names which have public setters or by constructor parameter names.
Another idea would be adding converter in AddJsonOptions in Startup class for my types - IConfiguration.Bind could infer how to construct it but I also tried that with no success.
Have you encoutered similar problems and found some reasonable solution to that?
Edit: Adding parameterless constructor will work of course, but sometimes I need to deal with some classes from external packages I'd like to use as parts of my config class so let's assume we can not modify them. I'd like to avoid adding new types for mapping only as well. Ideally I'd like to force ASP.NET Core engine to use existing constructor with parameters and by parameter name map with json properties - which currently is not working.
You should just add a default constructor in MyIdentity class.
.bind() binds the configuration into the object using the default constructor.
So, add the required default constructor in your MyIdentity class and it will be fine.
public MyIdentity(){}
Also, you can use Options.
In ConfigureServices, add the following:
services.AddOptions();
services.ConfigureOptions<MyServiceConfig>();
and then use dependency injection to initialize it.
In addition, use your own JsonConverter
Say you have a multi-tenant app. A Tenant has various properties:
public class Tenant{
public string TenantName {get; set;}
public string TenantUrl {get; set;}
}
This way when my service layer sends emails, for example, I can do the following:
SendEmail(Tenant.FromEmailAddress, recipientEmailAddress)
This works well for properties. In many places throughout my business logic, I'm encountering cases where tenant-specific behaviors must be accounted for. One example is retrieving photos for the homepage:
public List<string> GetPhotoUrls(){
if(currentTenant == TenantA){
// logic to go off to retrieve from one third party
} else if (currentTenant == TenantB){
// totally different logic
} else... // one for each tenant
// do some stuff
// return stuff
}
GetPhotoUrls is a simple example - but there are cases like this in many places in my business logic. I'm looking for a simple pattern where I can define and implement tenant-specific logic. The overall goal is to get all tenant-specific logic in one place so tenant creation and definition is easy.
I would like the developer experience to read along the lines of:
public List<string> GetPhotoUrls(){
currentTenant.GetPhotoUrls(); // define this logic on the tenant object somehow
// do some stuff
// return stuff
}
What patterns/constructs are available to achieve this?
Use the strategy pattern in your case. The pattern is best applied when you see switch statements or multiple if statements to simplify the client so that it delegates custom implementation to dependent interfaces. You may also use in combination of factory pattern. To illustrate this:
public interface ITenant{
List<string> GetPhotoUrls();
}
public class TenantA:ITenant{
public string TenantName {get; set;}
public string TenantUrl {get; set;}
public List<string> GetPhotoUrls(){
//A implementation
}
}
public class TenantB:ITenant{
public string TenantName {get; set;}
public string TenantUrl {get; set;}
public List<string> GetPhotoUrls(){
//B implementation
}
}
public class SomeTenantApp{
public SomeTenantApp(ITenant tenant){
_tenant = tenant;
}
public void DoSomething(){
var urls = _tenant.GetPhotoUrls();
//do something
}
}
public static class TenantFactory{
public static ITenant Create(string id)
{
//logic to get concrete tenant
return concreteTenant;
}
}
class Program
{
static void Main(string[] args)
{
var tenant = TenantFactory.Create("A");
var app = var SomeTenantApp(tenant);
app.DoSomething();
}
}
The client (SomeTenantApp) won't have to change. You delegated the implementation to the concrete class which owns the logic.
If you want to build SaaS, I'd strongly recommend using ASP.NET Core and dependency injection to overcome multi-tenancy issue.
You can defined your tenant class :
public class AppTenant
{
public string Name { get; set; }
public string[] Hostnames { get; set; }
}
Next you can resolve a tenant from the current request
public class AppTenantResolver : ITenantResolver<AppTenant>
{
IEnumerable<AppTenant> tenants = new List<AppTenant>(new[]
{
new AppTenant {
Name = "Tenant 1",
Hostnames = new[] { "localhost:6000", "localhost:6001" }
},
new AppTenant {
Name = "Tenant 2",
Hostnames = new[] { "localhost:6002" }
}
});
public async Task<TenantContext<AppTenant>> ResolveAsync(HttpContext context)
{
TenantContext<AppTenant> tenantContext = null;
// it's just a sample...
var tenant = tenants.FirstOrDefault(t =>
t.Hostnames.Any(h => h.Equals(context.Request.Host.Value.ToLower())));
if (tenant != null)
{
tenantContext = new TenantContext<AppTenant>(tenant);
}
return tenantContext;
}
}
Wiring it up :
public void ConfigureServices(IServiceCollection services)
{
services.AddMultitenancy<AppTenant, AppTenantResolver>();
}
Getting the current tenant (whenever you need it) :
public class HomeController : Controller
{
private AppTenant tenant;
public HomeController(AppTenant tenant)
{
this.tenant = tenant;
}
.
.
.
}
For more info take a look at SaasKit
Building multi-tenant applications with ASP.NET Core (ASP.NET 5)