We Keep Coding

VSTS Build Pipeline: Test fails connecting to Azure Key Vault

I am trying to use VSTS (now Azure DevOps) to do a CI/CD pipeline. For my build pipeline, I have a very basic setup involving doing a restore, build, test, and publish steps.
For my test step, I have it setup to run two test projects - one unit test project and one integration test project. I have my Key Vault access policy setup to provide access to both myself and Azure Devops. When I run my tests locally using visual studio, as I am logged into the same account which has access to azure key vault, I can run the tests without any errors.
My application is configured to access key vault using below setup:
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((ctx, builder) =>
{
var keyVaultEndpoint = GetKeyVaultEndpoint();
if (!string.IsNullOrEmpty(keyVaultEndpoint))
{
var azureServiceTokenProvider = new AzureServiceTokenProvider();
var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
builder.AddAzureKeyVault(keyVaultEndpoint, keyVaultClient, new DefaultKeyVaultSecretManager());
}
}
)
.UseStartup<Startup>();
When I run the build pipeline, I am using a Hosted VS2017 instance to build my project. Everything is running except the integration tests which try and access the key vault fail. I am using the following packages:
Microsoft.Azure.Services.AppAuthentication - makes it easy to fetch
access tokens for Service-to-Azure-Service authentication scenarios.
Microsoft.Azure.KeyVault - contains methods for interacting with Key Vault.
Microsoft.Extensions.Configuration.AzureKeyVault - contains
IConfiguration extensions for Azure Key Vault
I followed this tutorial https://learn.microsoft.com/en-us/azure/key-vault/tutorial-web-application-keyvault to setup the key vault and integrate it into my app.
I am merely trying to get my build to work by making sure both the unit and integration tests pass. I am not deploying it to an app service yet. The unit tests run without any issues as I am mocking the various services. My integration test is failing with below error messages. How do I get my test access to the key vault? Do I need to add any special access policies to my key vault for the hosted VS2017 build? Not sure what to do as I don't see anything that stands out.
Below is the stack trace for the error:
2018-10-16T00:37:04.6202055Z Test run for D:\a\1\s\SGIntegrationTests\bin\Release\netcoreapp2.1\SGIntegrationTests.dll(.NETCoreApp,Version=v2.1)
2018-10-16T00:37:05.3640674Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-10-16T00:37:05.3641588Z Copyright (c) Microsoft Corporation. All rights reserved.
2018-10-16T00:37:05.3641723Z
2018-10-16T00:37:06.8873531Z Starting test execution, please wait...
2018-10-16T00:37:51.9955035Z [xUnit.net 00:00:40.80] SGIntegrationTests.HomeControllerShould.IndexContentTypeIsTextHtml [FAIL]
2018-10-16T00:37:52.0883568Z Failed SGIntegrationTests.HomeControllerShould.IndexContentTypeIsTextHtml
2018-10-16T00:37:52.0884088Z Error Message:
2018-10-16T00:37:52.0884378Z Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProviderException : Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/63cd8468-5bc3-4c0a-a6f8-1e314d696937. Exception Message: Tried the following 3 methods to get an access token, but none of them worked.
2018-10-16T00:37:52.0884737Z Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/63cd8468-5bc3-4c0a-a6f8-1e314d696937. Exception Message: Tried to get token using Managed Service Identity. Access token could not be acquired. MSI ResponseCode: BadRequest, Response: {"error":"invalid_request","error_description":"Identity not found"}
2018-10-16T00:37:52.0884899Z Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/63cd8468-5bc3-4c0a-a6f8-1e314d696937. Exception Message: Tried to get token using Visual Studio. Access token could not be acquired. Visual Studio Token provider file not found at "C:\Users\VssAdministrator\AppData\Local\.IdentityService\AzureServiceAuth\tokenprovider.json"
2018-10-16T00:37:52.0885142Z Parameters: Connection String: [No connection string specified], Resource: https://vault.azure.net, Authority: https://login.windows.net/63cd8468-5bc3-4c0a-a6f8-1e314d696937. Exception Message: Tried to get token using Azure CLI. Access token could not be acquired. Process took too long to return the token.
2018-10-16T00:37:52.0885221Z
2018-10-16T00:37:52.0885284Z Stack Trace:
2018-10-16T00:37:52.0885349Z at Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider.GetAccessTokenAsyncImpl(String authority, String resource, String scope)
2018-10-16T00:37:52.0885428Z at Microsoft.Azure.KeyVault.KeyVaultCredential.PostAuthenticate(HttpResponseMessage response)
2018-10-16T00:37:52.0885502Z at Microsoft.Azure.KeyVault.KeyVaultCredential.ProcessHttpRequestAsync(HttpRequestMessage request, CancellationToken cancellationToken)
2018-10-16T00:37:52.0886831Z at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretsWithHttpMessagesAsync(String vaultBaseUrl, Nullable`1 maxresults, Dictionary`2 customHeaders, CancellationToken cancellationToken)
2018-10-16T00:37:52.0886887Z at Microsoft.Azure.KeyVault.KeyVaultClientExtensions.GetSecretsAsync(IKeyVaultClient operations, String vaultBaseUrl, Nullable`1 maxresults, CancellationToken cancellationToken)
2018-10-16T00:37:52.0886935Z at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.LoadAsync()
2018-10-16T00:37:52.0887000Z at Microsoft.Extensions.Configuration.AzureKeyVault.AzureKeyVaultConfigurationProvider.Load()
2018-10-16T00:37:52.0887045Z at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
2018-10-16T00:37:52.0887090Z at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
2018-10-16T00:37:52.0887269Z at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
2018-10-16T00:37:52.0887324Z at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
2018-10-16T00:37:52.0887371Z at Microsoft.AspNetCore.TestHost.TestServer..ctor(IWebHostBuilder builder, IFeatureCollection featureCollection)
2018-10-16T00:37:52.0887433Z at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateServer(IWebHostBuilder builder)
2018-10-16T00:37:52.0887477Z at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
2018-10-16T00:37:52.0887525Z at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
Update
I have found only 1 related post to this issue: https://social.msdn.microsoft.com/Forums/en-US/0bac778a-283a-4be1-bc75-605e776adac0/managed-service-identity-issue?forum=windowsazurewebsitespreview. But the post is related to deploying an application into an azure slot. I am merely trying to build my application in a build pipeline.
I am still trying to solve this issue and am not sure what the best way to provide the required access is.
Update 2
I have still not found a solution for this. I am lost on how to get my pipeline to run my test without issues. I saw that the release pipeline you have the options of running tests too. But these seem to take .dll files and my build pipeline drop file only has the web app (I don't see any of the test projects published drop file). Not sure if that is even a possibility.
Update 3
I managed to get it to work by using the last option provided here: https://learn.microsoft.com/en-us/azure/key-vault/service-to-service-authentication#connection-string-support
I tried the other ways of using a certificate but anytime {CurrentUser} is provided in a connection string, the build pipeline fails. It works on my local machine but not in the build pipeline.
To get it to work, I had to do three things:
Log in to Azure. Setup a new app registration in Azure AD
In your new AD app registration, create a new client secret
Provide your new AD App access to your key vault. Go into your key vault access policies and add the app that you created in your AD with read access to your secrets.
Modified my call to AzureServiceTokenProvier() in my Program.cs file as follows:
var azureServiceTokenProvider = new AzureServiceTokenProvider("connectionString={your key vault endpoint};RunAs=App;AppId={your app id that you setup in Azure AD};TenantId={your azure subscription};AppKey={your client secret key}")
Note that your client secret has to be formatted correctly. The app registrations (preview) generates a random secret key. Sometimes this key does not work in the connection string (throws an error as incorrectly formatted). Either try generating your own key in the non-preview version of app registration or generate a new key and try again.
After that I was able to run my integration test in my build pipeline successfully and create a release to my web app in Azure. I'm not satisfied with this approach because although it works, its exposing a secret value in the code itself. Manages service identity does not need to be turned on due to above approach. I feel that this is extremely bad in that regard.
There has to be a better way than this. One option is not to run the integration test in the build pipeline. Not sure if this is the correct approach. I'm still hoping someone will be able to provide a better approach to this or explain if my approach is okay to use.

Use the Azure CLI pipeline task to run integration tests that need KeyVault secrets successfully, without exposing any secrets in source control:
Create a Service Principal service connection in your Azure
DevOps project.
Give the principal Get and List permissions to the Vault in Azure.
Run your integration tests inside an Azure CLI task:
- task: AzureCLI#1
inputs:
azureSubscription: 'Your Service Connection Name'
scriptLocation: 'inlineScript'
inlineScript: 'dotnet test --configuration $(buildConfiguration) --logger trx'
This works because the tests will run in the context of azure cli, which is where AzureServiceTokenProvider tries fetching a token from before it fails. Azure CLI handles the authentication and cleans up when the task is done.

You should not do the integration test of authentication to Azure KeyVault within Azure DevOps Pipelines build, because you are using Azure DevOps default hosted agents.
By default, the Azure DevOps Pipelines are using basic default hosted agents, and these hosted agents are not accessible from your Azure subscription. These are not surprising, because these hosted agents are common agents for all common build needs, including build/compile, running unit tests, getting test coverages, and all of these tasks has no other additional features such as having ActiveDirectory, database, and other actual authentication/requests to other party such as authentication to any Azure Keyvault. Therefore these agents by default are not registered in your Azure subscription.
If you want to have successful integration tests for these special needs, you have to create your own agents for Azure DevOps Pipelines build and release. Therefore, there is no other way to force Azure DevOps default agent to run your KeyVault authentication tests, other than creating your own agents and configure your Azure DevOps to use your own agents.
To create your own agents, consult this documentation from Microsoft:
https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=vsts#install
UPDATE 29th October, 2018:
For more clarity, I also reply for your "Update 3" workaround. There is no guarantee that your workaround will work nicely when Microsoft updates the Azure DevOps' default hosted agent.
Therefore I also need to add more point: it's not a good practice to have integration test that relies on other party beyond the realm of your Azure DevOps Pipelines build such as connecting to a database server or using external authentications (even on Azure KeyVault) within your CI, especially if you are using Microsoft's default hosted agents.
Not just it will be error-prone due to invalid authentication configuration, but there's no guarantee that the further updates on the default hosted-agents would guarantee your third-party logic test will work.

Running into the exact same issue myself. I did get a little further by modifying the code by adding a connection string to the AzureServiceTokenProvider (The default parameter passed is null). I still didn’t get it to fully work though, maybe since the Azure DevOps user may or may not have the required access to the KeyVault, but I did not get an opportunity to dig in further.
Hoping there is a better solution posted here.
Update
We added the Build user into the Azure AD and then added it to the Access Policies within the KeyVault to the user. Granting it only Get Access (Our test was only testing whether it could gather the secret). Tests pass successfully now.

An easier solution would be to use Azure DevOps Variable Groups.
Someone with read permissions and contributor on the DevOps project can create the variable group, link it to the key vault and select the desired secrets.
The variable group can now be linked to any of your pipelines.
However to make it available to any code running in the pipeline you must first export the secrets using this method.
You need to do this via a task (Azure Powershell or Bash) but it must be done via an inline script. You cannot export the keyvault variables in a script in a file in the t ask. So in the first task export all your variables and all the subsequent tasks and referenced scripts can consume them.
PowerShell:
Write-Host "##vso[task.setvariable variable=mysecretexported]$(mysecret1)"
Bash
#echo ##vso[task.setvariable variable=mysecretexported]$(mysecret1)"
You can then refer to the secret using the exported variable
Powershell
Write-Host No problem reading "$env:MYSECRETEXPORTED"
Batch:
#echo No problem reading %mysecretexported%
Bash works similar:
#!/bin/bash
echo "No problem reading $MYSECRETEXPORTED"
This is also supported in YAML
The nice part is that these variables will be masked in your logs so your secrets stay secret.

Error executing command from .NET: The supplied kernel information version is invalid

I'm trying to execute a command from inside a .NET process. The command works just fine in Powershell or a regular command prompt executed by the same user as used for the .NET process.
I am trying to update the RRAS SSTP cert on Windows server 2016 with the following command:
netsh ras set sstp-ssl-cert name=bla.domain.foo
The error I'm getting is the following:
The supplied kernel information version is invalid.
This is the code I'm using to execute from inside a simple .NET console application. Outputting logic omitted for brevity.
using (var powerShellInstance = PowerShell.Create())
{
powerShellInstance.AddScript(command);
powerShellInstance.Invoke();
}
I'm using the .NET framework 4.6.1, I have tried adding a manifest to up the required execution rights but neither that or running as admin manually changes anything.
The application is executed by a scheduled task and if I manually add the required script as a step directly executing with cmd.exe it works like a charm. This is not very clean however, and requires me knowing the cert name when creating the task. There are a million ways I can overcome this issue but for the life of me I don't understand why it doesn't just work from .NET while all other commands I require are working fine.

So, this is not really a PoSH specific thing natively. That error message is more general than specific to .Net use case.
This error message is not unique to what you at doing, especially most recently on Win10. This error has been reported, since the 1703 release, even just when doing things on the file system, like WDS captures, folder creation etc.
Several reasons have been postulated, permissions and the like, but all had to just come up with a workaround (permission fixes, etc.) as you have indicated the you have a million ways to address your use case.
But to get to potential root cause, you need to look deeper at the other Windows event log information, Application, Security, System and PowerShell logs from when you are doing your .Net effort.

C# EventLog.Delete Access Denied

I have read other questions on SO in regards to security and registry keys, nothing has helped me solve my particular use case scenario.
Here's my scenario:
What I'm Trying To Do
I want to, in code, delete a windows event log.
The Problem
When executing the function, I receive a System.ComponentModel.Win32Exception. The exception message is "Access is denied".
How I Am Doing It Currently
I am using an impersonator function that I wrote which wraps around the EventLog.Delete function, it drops me into a user context that has full access to the EventLog Registry Hive. Subsequently the logs I am interested in also have full access for this particular user.
My Question
Why do I receive a "Access Is Denied" if the user I am running under (through impersonation) has full access to the log in question? I've tested my Impersonation function and it works as expected for other code I've written. I don't get why I would get access denied for this.
In another scenario with my impersonation function it works just fine, for example if I tried to write to a file that the user context that is running the program does not have write access to, then I would not be able to write to the text file, however if I use my impersonation to drop into a user context that does have write access then it works just fine (I can write to the file). So I just don't understand why the same concept can't be applied to registry keys.
What am I missing here?
The Code
Exception Message
My Test
Where sw-test is a user I created for testing purposes, it has full access permissions to the registry we are trying to delete.
[TestMethod]
public void DeleteEventLog_ValidatedUser_DeleteLog()
{
using (new Impersonator(Environment.UserDomainName, "sw-test", "pswd"))
{
Logging logging = new Logging();
logging.DeleteEventLog("testLog");
}
}

Okay I eventually got around to figuring this out, there were two issues at play here that were causing the mentioned exception being thrown, they are as follows:
1. Visual Studio was NOT running in administrator mode.
Not running visual studio in administrator mode was one part of the problem, this seems to be associated with access tokens in the windows OS. According to a source I read, if I run a program without UAC on (which is my scenario, I have it off), then the program being run gets a copy of my access token. However if I have UAC enabled, the program gets a copy of my access token but it is a restricted access token. (see: What precisely does 'Run as administrator' do?) - To be honest this doesn't really make sense in my case, why do I have to run as admin if I have UAC off? Shouldn't visual studio have an unrestricted copy of my access token? I am in the administrator group with UAC off...
2. Not Specifying NewCredentials As a Logon32Type In Impersonation
I don't really understand it but as soon as I specified this for my impersonation everything started working perfectly, I read a blog about it, it talks about how it was introduced in the VISTA days and how it was mainly used to specify credentials to outbound network connections to servers, and was mainly used to remedy security-related issues server-side. Don't see how it correlates to interfacing with local event logs though. (see: https://blogs.msdn.microsoft.com/winsdk/2015/08/25/logonuser-logon32_logon_new_credentials-what-is-this-flag-used-for/)
Code
using (new Impersonator(Environment.UserDomainName, "sw-test", "pswd", Advapi32.Logon32Type.NewCredentials))
{
EventLog.CreateEventSource("testSource", "testLog");
EventLog.Delete("testLog");
}
Where the NewCredentials is an int 9

Converting Microsoft EWS StreamingNotification Example to a service

I've been working to try and convert Microsoft's EWS Streaming Notification Example to a service
( MS source http://www.microsoft.com/en-us/download/details.aspx?id=27154).
I tested it as a console app. I then used a generic service template and got it to the point it would compile, install, and start. It stops after about 10 seconds with the ubiquitous "the service on local computer started and then stopped."
So I went back in and upgraded to C# 2013 express and used NLog to put a bunch of log trace commands to so I could see where it was when it exited.
The last place I can find it is in the example code, SynchronizationChanges function,
public static void SynchronizeChanges(FolderId folderId)
{
logger.Trace("Entering SynchronizeChanges");
bool moreChangesAvailable;
do
{
logger.Trace("Synchronizing changes...");
//Console.WriteLine("Synchronizing changes...");
// Get all changes since the last call. The synchronization cookie is stored in the
// _SynchronizationState field.
// Only the the ids are requested. Additional properties should be fetched via GetItem
//calls.
logger.Trace("Getting changes into var changes.");
var changes = _ExchangeService.SyncFolderItems(folderId, PropertySet.IdOnly, null, 512,
SyncFolderItemsScope.NormalItems,
_SynchronizationState);
// Update the synchronization cookie
logger.Trace("Updating _SynchronizationState");
the log file shows the trace message ""Getting changes into var changes." but not the "Updating _SynchronizationState" message.
so it never gets past var changes = _ExchangeService.SyncFolderItems
I cannot for the life figure out why its just exiting. There are many examples of EWS streaming notifications. I have 3 that compile and run just fine but nobody as far as I can tell has posted an example of it done as a service.

If you don't see the "Updating..." message it's likely the sync threw an exception. Wrap it in a try/catch.
OK, so now that I see the error, this looks like your garden-variety permissions problem. When you ran this as a console app, you likely presented the default credentials to Exchange, which were for your login ID. For a Windows service, if you're running the service with one of the built-in accounts (e.g. Local System), your default credentials will not have access to Exchange.
To rectify, either (1) run the service under the account you did the console app with, or (2) add those credentials to the Exchange Service object.

Allowing connection to .NET COM server with mismatching integrity level

I'm having an issue with a COM based client-server setup. The COM server is written in C# (.NET 4.0) and runs as a (registered) local server.
Depending on which application connects to the server, other clients will receive a Server execution failed (Exception from HRESULT: 0x80080005 (CO_E_SERVER_EXEC_FAILURE)
The underlying issue is explained here (in the section COM is integrity aware). The way I understand it, it is being caused by the fact that an elevated application creates the server with a higher integrity level. When another non-elevated application then connects, it is not allowed to connect to the same instance. The same happens when a non-elevated application creates the process, followed an elevated application connecting.
I've tried to implement the solution described on the page: modifying the registry to set a security descriptor that should allow all clients to connect. There is a code sample in C++, but this does effectively the same thing in .NET:
// Security Descriptor with NO_EXECUTE_UP
var sd = new RawSecurityDescriptor("O:BAG:BAD:(A;;0xb;;;WD)S:(ML;;NX;;;LW)");
byte[] securityDescriptor = new Byte[sd.BinaryLength];
sd.GetBinaryForm(securityDescriptor, 0);
RegistryKey key = Registry.ClassesRoot.OpenSubKey("AppID\\{APP-ID-GUID}", true);
if (key == null)
{
key = Registry.ClassesRoot.CreateSubKey("AppID\\{APP-ID-GUID}");
}
using (key)
{
key.SetValue("LaunchPermission", securityDescriptor, RegistryValueKind.Binary);
}
However, this does not have the desired effect. When the second client tries to create an instance of the object in question, Windows tries to launch a separate instance of my COM Server, but the server prevents two instances from running as the same user. Given the permissions I've set, I would not expect a second instance to launch in the first place.
Since one of the client applications is running in Medium IL and the other in High IL, I also experimented with variants on the mandatory label, like:
O:BAG:BAD:(A;;0xb;;;WD)S:(ML;;NX;;;ME)
O:BAG:BAD:(A;;0xb;;;WD)S:(ML;;NX;;;LW)(ML;;NX;;;ME)(ML;;NX;;;HI)
I've also tried setting the ROTFlags registry key to 0x1 (ROTFLAGS_ALLOWANYCLIENT) as suggested on the page, still no change in behavior.
I've established that the LaunchPermission registry value is being used in some way. I cannot discover where it's being read using Process Monitor, but when I use the dcomcnfg.exe tool to set the same key, I can force the server to fail loading by denying launch permissions.
I would like to point out that my server process does not need elevation. How do I make both elevated and non-elevated processes capable of connecting to a single server instance?

According to Windows Vista Security Model Analysis you will need to use shared objects such as a named pipe to go between the different IL. Also, the shared object should have an IL equivalent to your lowest IL being used.

you have to Set Debug option to Any cpu in VS.