We Keep Coding

"WindowsCryptographicException: The system cannot find the path specified" when trying to use DPAPI from LocalSystem account

I use DPAPI with DataProtectionScope.LocalMachine in a Windows Service.
I tested my service by running it directly on my user account. It works.
Also works when run as administrator.
Then I install it as a Windows Service on LocalSystem account.
Then it throws something like this:
Exception Info: Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException: The system cannot find the path specified.
at System.Security.Cryptography.ProtectedData.ProtectOrUnprotect(Byte[] inputData, Byte[] optionalEntropy, DataProtectionScope scope, Boolean protect)
at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
I changed service user to my user account and it works. So - the data protection fails only if the service is run as LocalSystem.
Fragment that throws:
public byte[] Unprotect(byte[] data, DataProtectionScope scope = DataProtectionScope.CurrentUser)
=> ProtectedData.Unprotect(data, null, scope.AsSystemType());
The scope is set to DataProtectionScope.LocalMachine.
I also tried to run the service as "NT Authority\NetworkService" user. Same behavior. Can't access local machine protection keys.
Why is that and how can I fix that?

So, it seems like a bug in Windows DPAPI to me. It requires a user account different from LOCAL SYSTEM and NETWORK SERVICE.
As LOCAL SYSTEM is in Administrator role, I see no reason why the key access is denied for the user.
As a workaround I just use a different API.
Microsoft.AspNetCore.DataProtection - to be exact.
That API requires the new key to be created for the first time.
For practical implementation see:
https://github.com/HTD/Woof/tree/master/Packages/Woof.DataProtection
https://github.com/HTD/Woof/blob/master/Packages/Woof.DataProtection/Api/WindowsLocalSystemKey.cs
You can also check the https://github.com/HTD/Woof/tree/master/Demos/Woof.ServiceInstaller.TestService
for testing of a data protected service configuration.

Intermittent error accessing Azure key vault - Keyset does not exists

I am using Azure Key Vault in my MVC web application. I connect to the Key Vault using certificate. Below is my sample code
AssertionCert = GetCertificate(WebConfigurationManager.AppSettings[KeyVaultConstant.ApplicationID], WebConfigurationManager.AppSettings[KeyVaultConstant.CertificateThumbprint], false);
var keyClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback((authority, resource, scope) => GetToken(authority, resource, scope, AssertionCert)));
var secrets = keyClient.GetSecretsAsync(WebConfigurationManager.AppSettings["VaultUri"]).GetAwaiter().GetResult();
On some instances I get an error stating
Cryptographic Exception: Keyset does not exists
On research I found that I have to give IIS_IUser permission to the folder where primary key is stored. I did that, Yet once in a while I get this error. This does NOT happen everytime.

You are right about the permission issue . Here are the troubleshooting steps for the same:
Troubleshooting Steps:
First check the service account which is running CRM App pool or ADFS App pool
Now open certificate manager, in personal store, locate your certificate. Right click on the certificate and select "Managed Private Keys" option
In the permission window, check if your app pool service account is given appropriate permission (Read permission should be fine, otherwise you can give Full control)
In below screenshot I have given full permission on my wildcard certificate *.test.local to my App Pool account – "test\AppPoolSvc"
Once the permission is given, perform an IISRESET and try accessing.
For further reference , please check. Also please try to restart the machine to see if it works.

Azure web app sporadically throws CryptographicException: Keyset does not exist

I have a Identity Server web app hosted on Azure. It has a .pfx file in it's root directory for signing. The problem is that when newly published it works perfectly fine but after some time it starts throwing CryptographicException: Keyset does not exist.
Based on CryptographicException KeySet does not exists I would assume that it is a file access issue, but why out of the sudden azure is messing up with file access.

I've been seing the same exception sporadically. In my case, it was caused by Data Protection keys changing when I swap between deployment slots. When using services.AddDataProtection() with the default configuration and hosting the app on Azure App Service, the data protection keys are stored in %HOME%\ASP.NET\DataProtection-keys. This directory is backed by a network share to make sure the keys are available on all your app instances, BUT this is not the case for deployment slots. So when you switch from one deployment slot to another, the keys will change and you will see CryptographicException: Keyset does not exist when your app tries to unprotect a payload.
To make sure your app is always using the same key, you need to configure a different key storage provider. I.e. use Redis or Azure Blob Storage as backing store.
You can find more information about how to configure this in the official documentation.
I've also described the issue I had on my blog.

I recommend that you do not retrieve your signing cert from disk firstly. Rather upload the certificate to the associated web app in Azure Portal.
and retrieve the certificate on application startup
something like.
X509Certificate2 Certificate = null;
var certStore = new X509Store(StoreName.My, StoreLocation.CurrentUser);
certStore.Open(OpenFlags.ReadOnly | OpenFlags.OpenExistingOnly);
var certCollection = certStore.Certificates.Find(X509FindType.FindByThumbprint,"CERTIFICATE_THUMBPRINT_HERE",false);
Certificate = certCollection[0];

.Net Prevents API Web Service Connection

I have created an app in C# (.exe output) which connects to an API WebService that requires a Self-Signed certificate.
For debugging, I have add the certificate to Trusted Root Certification Authorities certificate store, and It has been working perfectly.
Running it from command line console, I do not get any error neither, (I guess it is because of command line also checks if the certificate is there.)
The problems comes when the applications is runn from another process, then is when I get the following error.
"The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel."
My questions are:
Why can I run it without any problem from comand line and from an automated process dont??
What should I do in order to that my application would use the self-signed certificate?
I have found the solution.
System.Net.WebException thrown when consuming a web service over HTTPS

You don't mention it, but is the process running as another user?
Unless you specifically select the computer account certificate store, you'll add certificates to your personal certificate store (your user account). These are not available when running as another user.
To make a certificate available for all users, add it to the computer certificate store:
Start mmc.exe
Press CTRL+M (Add/remove snap-in)
Select Certificates, click Add
Select Computer account, click Finish
Import your certificate
You can also select Service account and select the service which needs access to the certificate if you prefer.
This article on MSDN covers the same steps and is a bit more verbose.
Edit:
Adding a certificate to the machine store requires elevated permissions. If you're not bundling this in a MSI installer, I suggest you write a separate InstallCertificate.exe (console or windows app) which performs the installation.
In Visual Studio:
Create a new console/windows application and add an Application Manifest File (Add Item / CTRL+SHIFT+A). In the app.manifest file just added, change this line
<requestedExecutionLevel level="asInvoker" uiAccess="false" />
into this
<requestedExecutionLevel level="requireAdministrator" uiAccess="false" />
Place this code in Program.cs (add some logging and error handling before pushing this to production!):
using System;
using System.Security.Cryptography.X509Certificates;
namespace InstallCertificate
{
class Program
{
static void Main(string[] args)
{
string cerFileName = args[0];
X509Certificate2 certificate = new X509Certificate2(cerFileName);
X509Store store = new X509Store(StoreName.Root, StoreLocation.LocalMachine);
store.Open(OpenFlags.ReadWrite);
store.Add(certificate);
store.Close();
}
}
}
Run the application. You should get the UAC prompt and the certificate should be installed in your machine's root cert store.

Using a certificate for Azure subscription logs - 403 forbidden

Background:
I have a Windows Service which polls Azure subscription logs (API: http://msdn.microsoft.com/en-us/library/windowsazure/gg715318.aspx)
On my local development machine the service is set to log on as my account. The X509 certificate was imported under CurrentUser\Personal and in the source code where I check the cert store I have:
X509Store certStore = new X509Store(StoreName.My, StoreLocation.CurrentUser);
Issue:
The service works fine on my dev machine, it can retrieve data from the API.
On the testing machine I get this error:
The remote server returned an error: (403) Forbidden.
The service is set to log on as a specific user, dmz\aaseclg1 and the current user\personal cert store has the required certificate.
Any ideas?
Thanks in advance.
Edit: image of password prompt:

I have seen this error when I export the certificate from the machine on which it is created and while exporting, I choose to import it in .cer format (i.e. without exporting private keys). Can you try by exporting the certificate from your dev box in pfx format and then copy the file on your test box and import it again in your test box by selecting the file and installing the certificate?
UPDATE
I was able to reproduce this. When you import the certificate, please make sure that you have unchecked the checkbox which reads "Enable strong private key protection" as shown in the screenshot below.
When I check this checkbox, every time I use this certificate it prompts me to enter a password. Now I was using a GUI application so I could see that box. In your case since you are consuming the certificate through a Windows Service (a non UI thingie), this box never shows and you think the service is hanging.