I have a use case where my function is triggered by a queue. To be able to process the message, it needs some configuration, which I can get via an API.
This configuration changes a couple of times a day, but is relatively static. I can receive an event / message when the configuration changes.
To limit the number of requests to the API and not being fully dependent on the API being up, I would like to cache this configuration.
Since functions are short lived, memory caching didn't seem to be the right option. And each instance of my function should have a copy of the configuration.
Of course, I could consider a Redis cache, but I was hoping to find a more cost efficient solution.
I came across the watchFiles property which can be set in the host.json of an Azure Function. Microsoft docs. What I like about this, is that any changes to these files are picked up by the instances of the function. I was thinking of writing a configuration to a json file which I configure in watchFiles. Each time I receive an event of a configuration change, I can update the content of the file. The change of the file would then be picked up by the other instances of the function as well.
Unfortunately I couldn't find much documentation about this feature.
Would this work for the cache requirements I have? In case someone has some example code, that would be much appreciated.
Btw, I am using C# .NET 6.
In the meantime I got it to work locally.
Add to the host.json the following:
"watchFiles": [ "localCache.json" ]
Somewhere in code, write data to the file. This can simply be done with for example:
var content = "Some content I want to cache.";
File.WriteAllText("localCache.json", content);
Reading works in a similar way:
var content = File.ReadAllText("localCache.json");
In my case I serialized the data to JSON first and stored that to the file.
Each time data was written to the file, the function app instance was restarted.
There was one big showstopper for me: this doesn't work if you are using Zip-deploy. Then the filesystem is read-only and you can't write to the file.
Since I wanted to keep using zip-deploy, I am now considering writing a cache to a storage account I am already interacting with anyway.
Related
In my laravel application, I want to provide the users with the opportunity to download a copy of their stored data in the form of a Word document. I found that certain parts of this can only be accomplished using C#/.NET.
For this, I wrote a C# application alongside a method called GetWordProfile(User user) which returns FileInfo set to the actual path of the output file (this is always within the storage folder of laravel, so laravel has access to it). I only need the path and everything's done and dusted since from this point on, I can manage my laravel application to download this for the user.
However, the question is how do I get there? I must not forget about potential errors which may occur and thus display them (the errors are (inside my C# application) handled by log4net in a file as well as on the console; same goes for all output).
I tried to run my application using shell_exec respectively exec, however, both only returned zero results (null) (despite having set $output for exec) and thus seem not to be suitable. Also, I usually don't want loops (inside PHP/laravel) too much since you're then using a lot of computing power which is unnecessary for this sort of task, also you don't want to let your users wait more than, say, 5 secs, seeing nothing in your browser but the script being executed within a blank page (during the execution there's no content, obviously).
EDIT: I also approached the use of COM which ultimately did not work properly out either.
What is an appropriate approach towards this?
I did something similar with Python + C# a while back using IPC (Inter-process Communication) using named pipes.
EDIT: URL is broken. Here's the question someone asked previously on this topic.
Interprocess Communication using Named Pipes in C# + PHP
I have been reading about blocking Ip addresses from launching attacks against my published service, in this case a Worker Role. I have added a NetworkConfiguration to help achieve this goal and it appears to work just fine.
Now my question is this, how can I add nodes to this file from my running program?
I plan to keep logs and when an IP has sent a number of bad requests I would like my program to update the list of black listed IP addresses. It is not reasonable for me to manually edit this list overtime when I hopefully can blacklist live should my port come under a brute force attack or dns attack.
How can I load the config file in code to use XDocument to edit the
values in that node?
Will these changes propagate out to the other
instances running automatically?
Is there another way to accomplish
this goal?
I believe you can. Changes to ServiceConfig (not Service Defintion) can be updated via Service Management API.
API definition is here: https://msdn.microsoft.com/en-us/library/azure/ee460809.aspx
If you're in .NET you may want to use one of the Nuget packages to assist with this: https://www.nuget.org/packages/Microsoft.WindowsAzure.Management.Compute/
How can I load the config file in code to use XDocument to edit the
values in that node?
For this you would need to get the cloud service properties and extract the configuration settings from the XML returned. You would need to do it by invoking Get Cloud Service Properties Service Management API operation (Please look for appropriate method if you are using Azure Management Library). If you're consuming REST API, then please keep in mind that the config file returned in Bas64 encoded so you would need to convert it into string and then can load the XML.
Will these changes propagate out to the other instances running
automatically?
Once you make the changes, you would need to perform Change Deployment Configuration operation for the change to be applied. Once you apply these changes, they will be applied to all the instances automatically.
Is there another way to accomplish this goal?
AFAIK, this is the only way to accomplish this programmatically.
In my IIS I want to be able to load some variables from an external file.
Reading them from the web.config is a possibility but if I will want to update the variable without restarting, it will require me to edit all the web.config files in the cloud.
Reading them from a centralized db is also an option but some of my apps dont require a connection.
What is a recommended way to manage application variables for IIS.
Thanks.
The recommended way is a Web.config file, that's why there is a built-in appSettings section. Now, as far as updating variables without restarting, you wouldn't have to edit them in the cloud at all. Have a copy of those Web.config files locally, in a testing environment that's like PROD, and then update them there first. Then you can test those changes and upload the entire Web.config file which will cause the application pool to reset automatically.
EDIT
With more information now available because of the OP's comments -another good solution may be to leverage the machine.config instead of the web.config for those settings that needs to be changed across multiple websites on the same server, especially if it's time sensitive that the applications see the change.
Since you don't want to have an application restart (which both web.config and machine.config will do), I would recommend keeping it to custom settings object that can be responsible for updating itself independently. Consider a serializable object with settings that appear in a NameValueCollection (similar to a web.config or app.config file). This would allow you to add settings similarly. Some steps to consider:
Add a shared directory to a common network location so that you only have 1 copy of the file itself for all websites. Something like \\somecommonserver\shared\configuration\.
Add a static application variable to the code base so that each application pool will have a record of the last date/time the file was updated --> Application["CustomConfigLastUpdate"] = configFileInfo.LastWriteTimeUtc; (I recommend using UTC because servers time may differ, but UTC does not)
Each time the settings are accessed may be a bit too heavy, but at some determined interval (each access, every 5/15/60 minutes, whatever) check if the configFileInfo.LastWriteTimeUtc property is greater than what is stored in the application variable, then you need to go get a fresh copy of the settings.
Create your custom object with the [Serializable] attribute and provide it a LoadFromXmlFile method that receives a filename as an argument whose responsibility is to repopulate itself.
I have done this before to achieve a similar goal, but I do not have those code samples here at my office. Here is a good SO question that has a relevant answer with code describing similar behavior: How to Deserialize XML document
I am looking at ways to make our application more extensible and easier to manipulate without having to alter the web.config (or, in our case, application.config files, which contain the appsettings node).
One way I have thought about is keeping the app settings in the database table that has a sqlcachedependancy. This means that:
Any time a setting is changed in the database, the cache is invalidated, and the settings are retrieved again, thus updating the application in realtime without having to alter files and restart the entire app.
We can create a custom tool which allows us to alter the settings.
The cons as I see it are that this may cause serious logic problems in that, if you have something that checks an appsetting at the start of a process, and it then changes halfway through, you could end up unintentionally altering the process flow, as the requirement for a complete application restart is negated.
Is there a way round this?
Is there a better way to manage appsettings, so that you can alter them on the fly remotely for one, several, or all servers in one go?
I think you've nailed the two major players:
either you have access to the file system and you put all your settings in a plethora of *.config files there
OR:
you don't have access (or only very limited access) to the server's file system and thus you're probably better off putting config settings and user preferences in a database, basically leaving nothing but the connection string to the config file on disk
Both approaches have their pros and cons. I've been trying for a long time to find a way to "materialize" a config section from a database field, so that I could basically just use the config XML, but stored in a database field. Unfortunately, the entire .NET 2.0 config system is very much "locked down" and just only assumes data will come from files - there's no way to plug in e.g. a database provider to allow the config system to read its contents from a database field :-( Really too bad!
The only other approach I've seen is a "ConfigurationService" in the StockTrader 2.0 sample app provided by Microsoft, but for my needs, it felt like overkill and like a really complex, really heavy-weight subsystem.
You could use SQLite, which will be a self-contained DB in a single file. Two birds with one stone?
If you reference an external config file that contains appsettings (leaving everything else in the normal app.config) then I believe editing it only reloads those settings, it doesn't force the whole app to restart.
There's a similar question on the subject here:
Nested app.config (web.config) files
WRT the problem of values changing in the middle of program execution, I guess you could locally cache the values, and raise an event when they change, allowing routines to reach a suitable point before using the updated values.
I think in asp.net we sort of get this for free because each page lifecyle is distinct, so the value is simply applied to new page requests only, not in the middle of an execution.
Edit: A little extra info:
Configuration Changes Cause a Restart of the Application Domain
From MSDN:
Changes to configuration settings in Web.config files indirectly cause the application domain to restart. This behavior occurs by design. You can optionally use the configSource attribute to reference external configuration files that do not cause a restart when a change is made. For more information, see configSource in General Attributes Inherited by Section Elements.
More information on the ConfigurationManager class in the System.Configuration namespace which could be used to modify the config files programatically (ie in a custom tool, if relevant disk read permissions can be provided). If you stick to using the built in configuration classes, I think changing the external configs, would not cause application restart, but would raise events (such as property changed) which you could handle, to ensure your code is not caught out by changing settings.
Is there a way at runtime to switch out an applications app.config (current.config to new.config, file for file). I have a backup/restore process which needs to replace its own application.exe.config file. I have seen this post but it does not answer how to do this at runtime.
Turns out I can swap the .config file for the new one and do a ConfigurationManager.RefreshSection(...) for each section. It will update from the new .config file.
Microsoft .NET's app.config is not designed for your scenario, as well as many others. I often encounter a similar need, so I have spent a lot of effort designing a solution.
Redesign to use app.config only as a configuration bootstrap: specify where to find the rest of the real configuration data. This information should almost never change, so there is no need to handle file watching or application restarts.
Pick an alternate location for the real configuration data: a file, a database, perhaps even a web service. I prefer a database most of the time, so I create a configuration table with a simple structure that allows me to store my data.
Implement a simple library to wrap your configuration access so that you have a simple API for the rest of your application (via dependency injection). Hide the usage of app.config as well as your real configuration storage location(s). Since .NET is strongly-typed, make the configuration settings so--convert each string retrieved into the most-specific type available (URL, Int32, FileInfo, etc.).
Determine which configuration settings can be safely changed at runtime versus those that can't. Typically, some settings need to change along with others, or it simply makes no sense to allow them to change at all. If all your configuration data can safely change at runtime, then that makes things easy, but I HIGHLY doubt such a scenario. Hide the changeability and interdependencies of the configuration settings to the extent possible.
Design the response to the unavailability of your real configuration data. I prefer to treat the absence of any configuration setting as a fatal error that aborts the application, unless I can identify a usable default. Likewise, I abort in the absence of the configuration storage container (file, database table, etc.).
Enjoy, and best wishes.
Are you able to restart the application when you detect that you need to switch files? If so, it's just a matter of switching the files and restarting. Now, the tricky bit is if .NET keeps the app.config file open while the program is running. I suspect it doesn't, but if the most obviously approach fails, I suggest you have a second application (cfgswitcher.exe) which waits for the process with a PID specified on the command line to terminate, then switches config files and relaunches the original process. Then your app would just need to launch cfgswitcher.exe (passing in its own PID as a command line argument) and terminate.
As I say though, it's worth trying the more obvious approach first.
EDIT: If you can't restart the application (or even part of it in a new AppDomain) then various aspects of app.config (assembly bindings etc) can't be changed. If you're only interested in your own configuration sections changing, then I suggest you store them in a separate config file and reload them whenever you want to.
Look at the events available to you on the ApplicationSettingsBase class. There are PropertyChanged & SettingChanging that may give you what you need.
You could also watch the file and if it has changed call the reload method to get the new settings.
I don't think it is possible at all to switch the configuration at runtime without restarting, so if you can't apply Jon's approach, you should try to come up with an other approach.
Anyway, maybe it's just me not having enough information about your scenario, but this kind of feels fishy.
Are you sure that swapping the configuration file is the best way to achieve whatever requirement you need to meet? I mean, this is quite an uncommon thing. If I were you, I would try to come up with some other approach.