I've just found an obscure, yet deeply frustrating, bug with CloudConfigurationManager. I'm looking for workarounds, and also (as a side note) tips about the best forum in which to report the bug. I'm guessing it will be a relatively quick fix.
I've got an Azure app service that connects to DocumentDb with config settings called DocumentDB.Endpoint and DocumentDB.Key. These are picked up in F# with
let endpoint = config.ReadConfigSetting<string>("DocumentDB.Endpoint")
let key = config.ReadConfigSetting<string>("DocumentDB.Key")
The ReadConfigSetting method is a convenience method that performs the relevant type conversions and default assignments. Under the covers it uses CloudConfigurationManager.GetSetting. For our purposes, think of the call as
let endpoint = CloudConfigurationManager.GetSetting("DocumentDB.Endpoint")
let key = CloudConfigurationManager.GetSetting("DocumentDB.Key")
I have a webjob that performs cron jobs on my document DB collections. CloudConfigurationManager picks up the setting from the app service settings first, and if the key is not found in the app service settings, it will look at my webjob's app.config.
In my QA environment, my webjob is picking the correct endpoint, but the wrong key. This is because DocumentDb.Endpoint is listed directly in my app.config file, but DocumentDb.Key is in a separate file that is .gitignored. I don't want sensitive keys in the Git repo, even though it is private, and the credentials are only listed in app.config and my external file as a convenience that lets me run the job locally with a debugger.
So here is my setup:
App.config
<appSettings file="keys.config">
<add key="agentUserName" value="<Everyone can read this>" />
<add key="apiHost" value="<and this>" />
<add key="DocumentDB.Endpoint" value="<points to my remote develpment copy of DocumentDB -- looking forward to when I can get a local repo>" />
</appSettings>
keys.config
<appSettings>
<add key="DocumentDB.Key" value="<This is private, so it's in this gitignored file>" />
<add key="agentPassword" value="<I'm not telling you>" />
<add key="TestUserPassword" value="<I'd be an idiot to post this value in a SO question>" />
</appSettings>
You can see what's happening.
Expected behaviour of CloudConfigurationManager when looking up the value of DocumentDB.Key
Look at the underlying app serice settings for a value of DocumentDB.Key
If it exists, use that.
Otherwise, look in App.config.
If it's not there, look in keys.config.
Actual behaviour of CloudConfigurationManager
Is there a value in keys.config?
If so, use that value.
Then look at the app service settings
Then App.config.
The best workaround I have right now is to comment out the value in keys.config when I publish the web job, but that's clunky. Are there any better ways of doing this?
And where is the best place to log this issue?
Have you looked into Azure Key Vault? Here is an intro to Azure Key Vault: https://azure.microsoft.com/en-us/documentation/articles/key-vault-get-started/
If you store DocumentDB secrets in the Azure Key Vault, you can grant the access to the secrets to the application level. Here is another article that shows how to do it inside a web application: https://azure.microsoft.com/en-us/documentation/articles/key-vault-use-from-web-application/
Hope that helps.
Thanks.
Lengning
Related
I am currently trying to activate Azure blob storage for my Web app (I am running NopCommerce 3.70).
I created the storage account on Azure and updated the values of AzureBlobStorage in the Web.config, however nothing seems to happen. The container of the blob is still empty and all thumbs are still being saved under ~/Content/Images/Thumbs.
My config is the following:
<NopConfig>
<AzureBlobStorage ConnectionString="DefaultEndpointsProtocol=https;AccountName=myaccountname;AccountKey=myaccountkey;EndpointSuffix=core.windows.net" ContainerName="images" EndPoint="https://myaccountname.blob.core.windows.net/images" />
</NopConfig>
which is equivalent for Nopcommerce than writing this I reckon:
<appSettings>
<add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=myaccountname;AccountKey=myccountkey;EndpointSuffix=core.windows.net" ContainerName="images" EndPoint="https://myaccountname.blob.core.windows.net/images" />
</appSettings>
Is there anything else to do that I missed?
I'm running a demo site with nopCommerce 3.9 just fine. My config looks similar except the end point part.
Try using this (remove images at the end of endpoint)
<NopConfig>
<AzureBlobStorage ConnectionString="DefaultEndpointsProtocol=https;AccountName=myaccountname;AccountKey=myaccountkey;EndpointSuffix=core.windows.net" ContainerName="images" EndPoint="https://myaccountname.blob.core.windows.net/" />
</NopConfig>
I have been using dotnet core to create an application that runs in a Kubernetes cluster on Linux hosts. As I was testing it noticed getting exceptions when validating the CSRF tokens, that makes sense since I did not edit the machine key to be the same on every instance yet. As i proceeded to set the machine key in web.config i noticed this would no longer work in .Net Core.
As is is now using the DataProtection API, the machine key no longer worked. I tried implementing the api into my application, but when i read i would need to use a network share to exchange the keys between all instances i was stunned. Surely there must be an easier (and better) way to accomplish this without having to rely on a share to be online right?
i tried to set the following in the Startup class in the ConfigureServices method:
services.AddDataProtection().SetApplicationName("DockerTestApplication");
I somehow expected the keys to be generated using the applicationname, but this did not resolve the issue.
I found some interesting docs that all use code that will no longer compile, i guess Microsoft changed up some things:
https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/compatibility/replacing-machinekey
Does anyone know a solution to this problem that will also run on Linux and has the ability to share the tokens over the network between instances?
Thanks in advance!
I've made some tests to back up my comment about copying keys. First I created simple console application with the following code:
var serviceCollection = new ServiceCollection();
serviceCollection.AddDataProtection()
.SetApplicationName("my-app")
.PersistKeysToFileSystem(new DirectoryInfo(#"G:\tmp\so\keys"));
var services = serviceCollection.BuildServiceProvider();
var provider = services.GetService<IDataProtectionProvider>();
var protector = provider.CreateProtector("some_purpose");
Console.WriteLine(Convert.ToBase64String(protector.Protect(Encoding.UTF8.GetBytes("hello world"))));
So, just create DI container, register data protection there with specific folder for keys, resolve and protect something.
This generated the following key file in target folder:
<?xml version="1.0" encoding="utf-8"?>
<key id="e6cbce11-9afd-43e6-94be-3f6057cb8a87" version="1">
<creationDate>2017-04-10T15:28:18.0565235Z</creationDate>
<activationDate>2017-04-10T15:28:18.0144946Z</activationDate>
<expirationDate>2017-07-09T15:28:18.0144946Z</expirationDate>
<descriptor deserializerType="Microsoft.AspNetCore.DataProtection.AuthenticatedEncryption.ConfigurationModel.AuthenticatedEncryptorDescriptorDeserializer, Microsoft.AspNetCore.DataProtection, Version=1.1.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60">
<descriptor>
<encryption algorithm="AES_256_CBC" />
<validation algorithm="HMACSHA256" />
<masterKey p4:requiresEncryption="true" xmlns:p4="http://schemas.asp.net/2015/03/dataProtection">
<!-- Warning: the key below is in an unencrypted form. -->
<value>rVDib1M1BjbCqGctcP+N25zb+Xli9VWX46Y7+9tsoGywGnIg4p9K5QTM+c388i0mC0JBSLaFS2pZBRdR49hsLQ==</value>
</masterKey>
</descriptor>
</descriptor>
</key>
As you see, file is relatively simple. It states creation, activation, expiration dates, algorithms used, reference to deserializer class and of course key itself.
Now I configured asp.net application (so, another application, not that console one) like this:
services.AddDataProtection()
.SetApplicationName("my-app")
.PersistKeysToFileSystem(new DirectoryInfo(#"G:\tmp\so\keys-asp"))
.DisableAutomaticKeyGeneration();
If you now try to run application and do something that requires protection - it will fail, because there no keys and automatic key generation is disabled. However, if I copy keys generated by console app to the target folder - it will
happily use them.
So pay attention to the usual security concerns with copying keys, to expiration time of those keys (configurable with SetDefaultKeyLifetime) and using the same version of Microsoft.AspNetCore.DataProtection in all applications you share keys with (because it's version is specified in key xml file) - and you should be fine. It's better to generate your shared keys in one place and in all other places set DisableAutomaticKeyGeneration.
I'm trying to host my WCF service with a custom ServiceHost on IIS. I found a couple of articles on MSDN like this: Custom Service Host. Here, I'm supposed to add something to my services svc file, but I don't have one and I can't add one in visual studio either. Then I found this article: Configuration-Based Activation in IIS and WAS. This says
"The configuration-based activation feature removes the requirement to have a .svc file and therefore the associated overhead."
so I can just create a serviceHostingEnvironment entry in my Web.config (which I don't have either, but I guess App.config is equivalent since it contains my system.serviceModel configuration). However, I have to specify a relativeAddress for the service activation.
"The relativeAddress attribute must be set to a relative address such as <sub-directory>/service.svc or ~/<sub-directory/service.svc. "
So it should point to my svc file? I'm a bit confused, could you point me to the right direction?
I know documentation on MSDN is little confusing. Here is configuration that you need to put in web.confi/app.config
<serviceHostingEnvironment>
<serviceActivations>
<add relativeAddress="MyNonExistingServiceSVC.svc" service="MyService" factory=”MyServiceHostFactory”/>
</serviceActivations>
</serviceHostingEnvironment>
Here relative address will be just any dummy name. This name will be used to browse your service metadata. Please note that this name can be anything of your choice and it DOES NOT require same physical file to be present on disk. It just needs any name with .SVC extension.
So while accessing service metadata your URL will be
http://myserver/myservice/MyNonExistingServiceSVC.svc
// Set LastRun to now
config.AppSettings.Settings["LastRun"].Value = DateTime.Now.ToString();
// Save all settings
config.Save(ConfigurationSaveMode.Modified);
This code was working fine in my development server but not in my production server. It seems like my program is unable to communicate with my app.config file. I have checked all the "obvious" . . Any ideas ... ?
From your code example, I cannot tell how your config variable is initialized. But, from the comments, you have a web app. Unless you are attempting to load a specific app.config file, the web app will attempt to get AppSettings from web.config.
It's not a good idea to programatically change the values of web.config. Changing web.config will cause an application restart.
If you have a different app.config for storing this type of information, that would be better than trying to change web.config. But you'll have to specifically load the file, something like this:
Configuration config = WebConfigurationManager.OpenWebConfiguration("yourPath\app.config");
ConfigurationManager.OpenExeConfiguration() is intended for use within an executable application not a web app. Try using WebConfigurationManager as shown above.
You find some more information in this SO question/answers.
More information can be found in this SO question/answer.
I have a WCF service that I'm using to expose a data source (connected to via an EF model). My endpoint for some reason just stopped responding.
Up until my last changes I was just pulling the data at startup of my app. I'm now working on adding a subscribe/publish model on top of this so I can throw events when something hits the WCF service that results in the data changing (could do it in the client side but than I wouldn't catch other users changes).
I've been following the example from Microsoft: http://msdn.microsoft.com/en-us/library/ms752254.aspx and added the appropriate decorations to my service interface and implementations and added a subscribe/unsubscribe OperationContract etc. I than try to fire up the service and can't connect.
I than dropped the Service Reference from my client and tried to reattach. Here is the strange thing:
1) The WCF service builds and spins up in the developer host.
2) The Web.Config file wasn't touched.
3) The "Discover Services" option in the add service reference in vs 2012 sees the service but when I try to actually accept the discovered service it throws an error saying "error downloading metadata from the address.
I don't get how it can discover the service and then not be able to add it as a reference. I can see if I added the info myself but it found it.
As part of adding the publishing functionality I added the SessionMode.Required tag to the interface I'm not really sure how WCF handles sessions and haven't ever had to deal with sessions anywhere else in my coding (generally pull the data from the source and than process it locally has been the types of tools I needed to make in the past so no state needed to be persisted server side).
Any ideas?
my service web.config:
<connectionStrings>
<add name="TaskModelContainer" connectionString="metadata=res://*/TaskModel.csdl|res://*/TaskModel.ssdl|res://*/TaskModel.msl;provider=System.Data.SqlClient;provider
connection string="data source=winhacker\sqlexpress;initial
catalog=TaskDB;integrated
security=True;MultipleActiveResultSets=True;App=EntityFramework""
providerName="System.Data.EntityClient"/>
Note: Note sure what is going on with the formating most of the file doesn't show up in the preview but sufficience it to say it is the boilerplate one that gets autogenerated other than I added a connectionStrings section to like to my EF datapoint. If you want to see the file click on "edit" seems to show the source I copied though a bit formatted weird.
In the vast majority of cases where I have encountered this error, the issue has been a serialization issue on the server side; WCF will throw an exception about this during the metadata discovery phase, but VS does not display this exception.
In order to troubleshoot this problem, you need to add a diagnostics section to your web.config to capture the WCF information into a svclog file, then use Microsoft's Service Trace Viewer to examine the log and discover the actual exception being thrown.
The web.config section should look something like this:
<system.diagnostics>
<sources>
<source name="System.ServiceModel"
switchValue="Information, ActivityTracing"
propagateActivity="true">
<listeners>
<add name="traceListener"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData="c:\log\WebTrace.svclog" />
</listeners>
</source>
</sources>
</system.diagnostics>
and be added at the <configuration> level.
Once you have added the configuration and ensure that the specified directory exists, try adding the service reference again. This should cause the specified svclog file to be generated.
If you have VS installed on that machine, you should be able to just double-click the file in Windows Explorer and have the log opened automatically in the tool.
Once the log is opened, look down the list of activities in the left hand pane and click on the first one in red. Then, in the top right pane, select the first one that is red (or shows an exception, I forget which). You can then click on that to see the details, including exception information in the bottom right pane.