I have been using dotnet core to create an application that runs in a Kubernetes cluster on Linux hosts. As I was testing it noticed getting exceptions when validating the CSRF tokens, that makes sense since I did not edit the machine key to be the same on every instance yet. As i proceeded to set the machine key in web.config i noticed this would no longer work in .Net Core.
As is is now using the DataProtection API, the machine key no longer worked. I tried implementing the api into my application, but when i read i would need to use a network share to exchange the keys between all instances i was stunned. Surely there must be an easier (and better) way to accomplish this without having to rely on a share to be online right?
i tried to set the following in the Startup class in the ConfigureServices method:
services.AddDataProtection().SetApplicationName("DockerTestApplication");
I somehow expected the keys to be generated using the applicationname, but this did not resolve the issue.
I found some interesting docs that all use code that will no longer compile, i guess Microsoft changed up some things:
https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/compatibility/replacing-machinekey
Does anyone know a solution to this problem that will also run on Linux and has the ability to share the tokens over the network between instances?
Thanks in advance!
I've made some tests to back up my comment about copying keys. First I created simple console application with the following code:
var serviceCollection = new ServiceCollection();
serviceCollection.AddDataProtection()
.SetApplicationName("my-app")
.PersistKeysToFileSystem(new DirectoryInfo(#"G:\tmp\so\keys"));
var services = serviceCollection.BuildServiceProvider();
var provider = services.GetService<IDataProtectionProvider>();
var protector = provider.CreateProtector("some_purpose");
Console.WriteLine(Convert.ToBase64String(protector.Protect(Encoding.UTF8.GetBytes("hello world"))));
So, just create DI container, register data protection there with specific folder for keys, resolve and protect something.
This generated the following key file in target folder:
<?xml version="1.0" encoding="utf-8"?>
<key id="e6cbce11-9afd-43e6-94be-3f6057cb8a87" version="1">
<creationDate>2017-04-10T15:28:18.0565235Z</creationDate>
<activationDate>2017-04-10T15:28:18.0144946Z</activationDate>
<expirationDate>2017-07-09T15:28:18.0144946Z</expirationDate>
<descriptor deserializerType="Microsoft.AspNetCore.DataProtection.AuthenticatedEncryption.ConfigurationModel.AuthenticatedEncryptorDescriptorDeserializer, Microsoft.AspNetCore.DataProtection, Version=1.1.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60">
<descriptor>
<encryption algorithm="AES_256_CBC" />
<validation algorithm="HMACSHA256" />
<masterKey p4:requiresEncryption="true" xmlns:p4="http://schemas.asp.net/2015/03/dataProtection">
<!-- Warning: the key below is in an unencrypted form. -->
<value>rVDib1M1BjbCqGctcP+N25zb+Xli9VWX46Y7+9tsoGywGnIg4p9K5QTM+c388i0mC0JBSLaFS2pZBRdR49hsLQ==</value>
</masterKey>
</descriptor>
</descriptor>
</key>
As you see, file is relatively simple. It states creation, activation, expiration dates, algorithms used, reference to deserializer class and of course key itself.
Now I configured asp.net application (so, another application, not that console one) like this:
services.AddDataProtection()
.SetApplicationName("my-app")
.PersistKeysToFileSystem(new DirectoryInfo(#"G:\tmp\so\keys-asp"))
.DisableAutomaticKeyGeneration();
If you now try to run application and do something that requires protection - it will fail, because there no keys and automatic key generation is disabled. However, if I copy keys generated by console app to the target folder - it will
happily use them.
So pay attention to the usual security concerns with copying keys, to expiration time of those keys (configurable with SetDefaultKeyLifetime) and using the same version of Microsoft.AspNetCore.DataProtection in all applications you share keys with (because it's version is specified in key xml file) - and you should be fine. It's better to generate your shared keys in one place and in all other places set DisableAutomaticKeyGeneration.
Related
I'm working on an app that uses the IdentityServer6 library. Out of the box this seems to use a pair of in-memory signing keys which I can see in the jwks discovery doc when run locally.
I've now provided a custom implementation of ISigningCredentialStore and IValidationKeysStore to load my own key pair from an external source. These I've registered as per the docs like this:
builder.Services.AddSingleton<ISigningCredentialStore, MySigningCredentialStore>();
builder.Services.AddSingleton<IValidationKeysStore, MyValidationKeysStore>();
However, when I run this I now get the original pair of keys, plus my own two in the discovery doc, so four in total. So how can I stop IdentityServer using its own keys?
OK, I worked this out. Needed to set KeyManagement.Enabled to false in the options for AddIdentityServer.
I am using Microsoft.AspNetCore.DataProtection in ASP.NET Core 2.0 application for data protection. And for the default settings I have added below code in Startup.cs file
public IServiceProvider ConfigureServices(IServiceCollection services)
{
services.AddDataProtection().ProtectKeysWithDpapi();
...
...
}
But that code throws an error:
InvalidOperationException: The 'IXmlRepository' instance could not be found. When an 'IXmlEncryptor' instance is set, a corresponding 'IXmlRepository' instance must also be set.
Did I miss something in this implementation?
You specified how to encrypt the keys (the Windows DPAPI implementation of IXmlEncryptor) but you did not specify where to persist the encryption keys (the IXmlRepository). There are various options to persist the keys, for example the file system, registry or some remote location in the cloud.
Try using PersistKeysToFileSystem() or PersistKeysToRegistry(). I suggest you take a look at the documentation regarding ASP.NET Core Data Protection configuration.
I am running an ASP.NET Core MVC app in a docker container, with an AWS credentials file. I have another service that is putting new keys into the file when the old ones expire, but these new keys don't seem to propagate through to my MVC app and my site crashes. I have seen that normally the solution to get strongly typed configuration to reload is to use IOptionsSnapshot, like:
services.AddDefaultAWSOptions(Configuration.GetAWSOptions())
.AddScoped(config => config.GetService<IOptionsSnapshot<AWSOptions>>().Value)
.AddAWSService<IAmazonS3>();
but this gives an exception:
System.InvalidOperationException: Cannot resolve scoped service 'Amazon.Extensions.NETCore.Setup.AWSOptions' from root provider.
Does anyone have a solution to getting ASP to reload the AWS credentials file? I'd like to continue using the AWS dependency injection extension if possible.
By default, AddAWSService registers the client factory in singleton scope, which means it's one and done for the life of the application. However, AddAWSService has a lifetime param you can utilize to customize this. Essentially, you need a shorter lifetime on the client, so that it will be recreated with the new settings. You can choose either "scoped" (request-scoped) or "transient" (new instance every time it's injected).
Obviously with "scoped", you'll get a connection with the updated settings every request. However, if you do any further operations on the same request after the settings have been changed, it will remain the old connection with the old settings (i.e. you'll still have the same issue, at least for the life of the request).
Using "transient" scope, you'll have have a client with the most updated settings, but you'll end up basically with a client for every use, which may not be ideal.
I'm trying to connect to the Google Datastore on my account with service account credentials file (which I've created according to the documentation), but I'm encountering with authentication error while trying to insert an entity:
Grpc.Core.RpcException: Status(StatusCode=Unauthenticated,
Detail="Exception occured in metadata credentials plugin.")
My code is:
var db = DatastoreDb.Create("myprojectid");
Entity entity = new Entity{
Key = db.CreateKeyFactory("mykindname").CreateIncompleteKey()
};
var keys = await db.InsertAsync(new[] { entity });
The GOOGLE_APPLICATION_CREDENTIALS variable refers to the credentials file and when calling GoogleCredential.GetApplicationDefaultAsync() to see if the credentials object is valid it indeed looks good...
I saw some earlier examples which used the GetApplicationDefaultAsync function togehether with some DatastoreService object - but I couldn't find the DatastoreService object (probably it was there in old versions...) in the latest .Net API: Google.Cloud.Datastore.V1
Notice that I don't want to use the other authenticaiton methods:
1) Using the gcloud cli.
2) Running from Google environment (app engine for example).
Any idea how to solve this?
After the great help of Jon Skeet the issue was solved.
The authentication issues can occur if you don't reference all the required Datastore dlls. Make sure that all the dlls are referenced on the project that are running the calls to the Datastore.
I've added the Google Datastore lib via the NuGet to my test project and everything worked!
Notice that in such cases it is recommended to enable gRPC logging. `(For exmaple: GrpcEnvironment.SetLogger(new ConsoleLogger()), there you'll probably see if there were issues loading several dlls...
Authentication can be broken if your system clock is significantly incorrect. Check your system time, and fix it if necessary, then try authenticating against Datastore again.
I've just found an obscure, yet deeply frustrating, bug with CloudConfigurationManager. I'm looking for workarounds, and also (as a side note) tips about the best forum in which to report the bug. I'm guessing it will be a relatively quick fix.
I've got an Azure app service that connects to DocumentDb with config settings called DocumentDB.Endpoint and DocumentDB.Key. These are picked up in F# with
let endpoint = config.ReadConfigSetting<string>("DocumentDB.Endpoint")
let key = config.ReadConfigSetting<string>("DocumentDB.Key")
The ReadConfigSetting method is a convenience method that performs the relevant type conversions and default assignments. Under the covers it uses CloudConfigurationManager.GetSetting. For our purposes, think of the call as
let endpoint = CloudConfigurationManager.GetSetting("DocumentDB.Endpoint")
let key = CloudConfigurationManager.GetSetting("DocumentDB.Key")
I have a webjob that performs cron jobs on my document DB collections. CloudConfigurationManager picks up the setting from the app service settings first, and if the key is not found in the app service settings, it will look at my webjob's app.config.
In my QA environment, my webjob is picking the correct endpoint, but the wrong key. This is because DocumentDb.Endpoint is listed directly in my app.config file, but DocumentDb.Key is in a separate file that is .gitignored. I don't want sensitive keys in the Git repo, even though it is private, and the credentials are only listed in app.config and my external file as a convenience that lets me run the job locally with a debugger.
So here is my setup:
App.config
<appSettings file="keys.config">
<add key="agentUserName" value="<Everyone can read this>" />
<add key="apiHost" value="<and this>" />
<add key="DocumentDB.Endpoint" value="<points to my remote develpment copy of DocumentDB -- looking forward to when I can get a local repo>" />
</appSettings>
keys.config
<appSettings>
<add key="DocumentDB.Key" value="<This is private, so it's in this gitignored file>" />
<add key="agentPassword" value="<I'm not telling you>" />
<add key="TestUserPassword" value="<I'd be an idiot to post this value in a SO question>" />
</appSettings>
You can see what's happening.
Expected behaviour of CloudConfigurationManager when looking up the value of DocumentDB.Key
Look at the underlying app serice settings for a value of DocumentDB.Key
If it exists, use that.
Otherwise, look in App.config.
If it's not there, look in keys.config.
Actual behaviour of CloudConfigurationManager
Is there a value in keys.config?
If so, use that value.
Then look at the app service settings
Then App.config.
The best workaround I have right now is to comment out the value in keys.config when I publish the web job, but that's clunky. Are there any better ways of doing this?
And where is the best place to log this issue?
Have you looked into Azure Key Vault? Here is an intro to Azure Key Vault: https://azure.microsoft.com/en-us/documentation/articles/key-vault-get-started/
If you store DocumentDB secrets in the Azure Key Vault, you can grant the access to the secrets to the application level. Here is another article that shows how to do it inside a web application: https://azure.microsoft.com/en-us/documentation/articles/key-vault-use-from-web-application/
Hope that helps.
Thanks.
Lengning