TF.exe and TfSecurity.exe authentication on AzureDevOps - c#

I have a piece of legacy c# code (console application) that runs as a nightly batch and fires off TF.exe and TFSSecurity.exe commands at AzureDevOps.
The commands are built by the c# code and then executed by means of launching PowerShell and executing them.
All have worked fine until now, but lately, it started failing for tf30063 authentication errors.
TF30063: You are not authorized to access https://dev.azure.com/
As part of my troubleshooting I have picked a few of these commands that are being built and executed them in an interactive PowerShell session.
tf permission /recursive $/<tfs_project_name>/ /collection:https://dev.azure.com/<organization_name>/
TFSSecurity /imx adm: /collection:https://dev.azure.com/<organization_name>/
It produces the same result - TF30063: You are not authorized to access https://dev.azure.com/.
The logged-on user (where the commands are run) is able to access this AzureDevOps URL via a browser.
Digging a bit deeper I ran the command: tf settings connections help which returned the following output:
Server Url : https://<organization_name>.visualstudio.com/
User :
I was actually expecting the passed URL here: https://dev.azure.com/
Not sure how this URL got there, or how to get it out - but, be that as it may, the empty user field arouses some suspicion.
I am really trying to figure out what the authentication flow/procedure is when executing these commands (TF and TFSSecurity) in Powershell.
I have been prompted for authentication by an AzureDevOps dialog once, but where do these provided credentials stored? And for how long?
I have been snooping around the Windows credentials in the Credential Manager, here I found some bits and pieces - but nothing conclusive.
Question:
Does anybody perhaps know how these 2 applications (tf.exe and TfSecurity.exe) handles authentication and storing of credentials?
System specs:
OS: Windows Server 2016
Powershell Version: 5.1.14393.3053
Location (version) of the tf.exe and TfSecurity.exe
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\
Team Explorer>

The cached credentials that tf.exe referenced might get corrupted. We donot know what caused this issue, nor can we give a certain method to fix this. You have to try below possible solutions to narrow down the fix.
1,
Using the browser from within your Visual Studio, View->Other Windows->Web Browser and navigate to the https://dev.azure.com/. Then check if it is logged in the wrong account, log out and relog in if wrong account is logged in.
2,
Running the following command from the Developer Command Prompt for VS:
tf workspaces /collection:https://dev.azure.com/<organization_name>
3,
Go to Team Explorer > Manage Connections (Little Plug next to Home button) > Right Clicked on Project > Connect. Then reenter in your credentials.
4, To clear all the caches
Close all Visual Studio instances, delete %LOCALAPPDATA%.IdentityService as you did.
Clear TFS caches %LOCALAPPDATA%\Microsoft\Team Foundation\7.0\Cache
Clear all the browser caches especially for the stored password
5, Run Visual Studio as another user:
cd C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE
runas /netonly /user: devenv.exe
Enter the user possword, then Team Explorer > Manage Connections
If none of the obove method fixes this issue. Please check here and here for more possible solutions.
You can also report a problem to Micrsoft Develop community(Report a problem > Azure Devops) if above issue persist.
For Server Url : https://.visualstudio.com/. It is the old version dev.azure.com domain name. The official document says it can be used as usual.
Update:
TF30063 error for TFSSecurity
The cached credentials for TFSSecurity is stored in the registry. You can delete it.
HKEY_CURRENT_USER\Software\Microsoft\VSCommon\14.0\ClientServices\TokenStorage\VisualStudio\VssApp
After you deleted the cached credentials in above registry. It will prompt you to re-enter the credential when you run tfssecurity.exe command again.

Related

DefaultAzureCredetials can't authenticate via Visual Studio - Can't find AzureServiceAuth\tokenProvider.json

I'm attempting to connect to an Azure Key Vault instance from a .NET 4.7 application running locally under IIS and the debugger (Visual Studio 2022 17.4.4) but am encountering the below exception(s) from the Azure.Identity package when it attempts to retrieve a token to authenticate to Azure when calling to perform a KeyVault action such as GetSecretAsync().
DefaultAzureCredential failed to retrieve a token from the included
credentials. See the troubleshooting guide for more information.
https://aka.ms/azsdk/net/identity/defaultazurecredential/troubleshoot
ManagedIdentityCredential authentication unavailable. Multiple attempts failed to obtain a token from the managed identity endpoint.
Visual Studio Token provider can't be accessed at C:\WINDOWS\system32\config\systemprofile\AppData\Local.IdentityService\AzureServiceAuth\tokenprovider.json
I need to connect to the KeyVault instance via a User Assigned Managed Identity in cloud environments such as production, whereas in development environments, we therefore need to connect via the developer's Visual Studio account to authenticate them to access the service, similarly. Perhaps I have misunderstood, but I believed this is possible via the DefaultAzureCredential option, which will try various methods of authentication in order (such as environment variables, managed identities, then Visual Studio credentials, etc) until one succeeds.
When inspecting the inner exception(s) relating to the Visual Studio Credentials flow, I see the System.Exception {System.IO.DirectoryNotFoundException} exception message states...
"Could not find a part of the path 'C:\WINDOWS\system32\config\systemprofile\AppData\Local.IdentityService\AzureServiceAuth\tokenprovider.json'.
Previously, this message had stated the below message (which I understand to be the more recent location for this file), until I attempted to run under Visual Studio 2019 for comparison, at which point, it changed to the above message.
"Could not find a part of the path C:\Users[AppPoolName]\AppData\Local.IdentityService\AzureServiceAuth\tokenProvider.json".
At first, I noticed the path didn't exist from .IdentityService onward, and so followed the suggestion on this MSFT forum post to restore the AppAuthentification extension from VS2019 into VS2022's configuration to restore the C:\Users\<AppPoolName>\AppData\Local\.IdentityService\AzureServiceAuth\tokenprovider.json file and providers the TokenProviders as a path to C:\Program Files (x86)\Microsoft Visual Studio\<version>\Enterprise\Common7\IDE\Extensions\<random dir name>\TokenService\Microsoft.Asal.TokenService.exe. On the next build, I noticed .IdentityService had been created, but not the proceeding directory or file.
I then tried logging out and into Visual Studio a number of times, but this did not seem to create the remaining missing directory and file. Creating the directory and file manually of course resolve the System.IO.DirectoryNotFoundException, but the error message then informs me that the file schema is incorrect. I'm unable to find an example with the correct schema and values.
In terms of client configuration options, I've been explicitly limiting the modes of authentication flow to just ManagedIdentity and VisualStudioCredential for simplicity after noticing other methods (e.g. AzureCLI and Azure PowerShell Module` also failed, despite being logged in to them).
_client = new SecretClient(new Uri(options.KeyVaultUri), new DefaultAzureCredential(
new DefaultAzureCredentialOptions
{
ExcludeManagedIdentityCredential = false,
ExcludeVisualStudioCredential = false,
ExcludeInteractiveBrowserCredential = true,
ExcludeAzurePowerShellCredential = true,
ExcludeAzureCliCredential = true,
ExcludeEnvironmentCredential = true,
ExcludeVisualStudioCodeCredential = true,
ExcludeSharedTokenCacheCredential = true,
ManagedIdentityClientId = options.ManagementIdentityClientId
}
));
I've also tried the suggestions on Azure SDK GitHub Issue #4590 of settings setProfileEnvironment and loadUserProfile to true in case it's an IIS permissions issue, but this made no difference - the same errors continue.
Finally, the only other reference I've found to the tokenProvider.json file is in Microsoft's documentation for App Authentication, but the re-authenticate button doesn't exist in the Tools > Options > Azure Service Authentication window as suggested.
"If you run into problems using Visual Studio, such as errors that
involve the token provider file, carefully review the preceding steps.
You may need to reauthenticate your developer token. To do so, select
Tools > Options, and then select Azure Service Authentication. Look
for a Re-authenticate link under the selected account. Select it to
authenticate."
As I'm able to locate C:\Program Files (x86)\Microsoft Visual Studio\<version>\Enterprise\Common7\IDE\Extensions\<random dir name>\TokenService\Microsoft.Asal.TokenService.exe and its related configuration file, I suspect it's the missing tokenProvider.json file that's the issue, but I'm not aware of what is responsible for creating that, nor what it should contain.
Any insight or pointers would be appreciated.
Notable packages and their versions in use:
Azure.Identity # v1.8.0
Azure.Security.KeyVault.Secrets # v4.4.0
Edit (1)
As one might expect, I'm able to configure an alternative flow to work by granting an RBAC record upon the Key Vault for an Azure AD Application Registration and then using the ClientSecretCredential flow in place of the DefaultAzureCredentials flow (as below). But this doesn't solve the problem in the best way so I'd be interested if anyone can spot where I'm going wrong with the DefaultAzureCredentials flow, if at all.
_client = new SecretClient(new Uri(options.KeyVaultUri),
new ClientSecretCredential(options.TenantId, options.ClientId, options.Secret)
);
Having found this question on SO, I found that tokenProvider.json existed in the same directory under C:\Users\<local user account> (via %LOCALAPPDATA%.IdentityService\AzureServiceAuth\) and was able to analyse it for reference and duplicate it to the IIS.
As suggested by Johnny5, it seems VisualStudioCredentials executes as the signed-in domain user, but IIS is running as the ApplicationPoolIdentity, hence it doesn't access the file under the domain user location and doesn't create one as it's not signed into Visual Studio. After a little research on how to alter this, I was able to set the IIS Application Pool identity as my domain user which matched the signed-in Visual Studio account.
To do this, follow these steps
Open IIS Manager > Go to Application Pools
Right-click the relevant pool, then click Advanced Settings...
Click the 3-dot button next to the Identity settings (likely ApplicationPoolIdentity)
Select Custom Account and enter your credentials (including any domain prefix - e.g. DOMAIN\MyUser)
If you aren't sure of your domain, open a command prompt and enter echo %USERDOMAIN% to find it.
I then set the SecretClient authentication flow back to utilise DefaultAzureCredential like so, re-tested locally, and success - secrets retrieved.
_client = new SecretClient(new Uri(options.KeyVaultUri), new DefaultAzureCredential(
new DefaultAzureCredentialOptions()
{
ExcludeEnvironmentCredential = true,
ExcludeVisualStudioCodeCredential = true,
ExcludeAzureCliCredential = true,
ExcludeAzurePowerShellCredential = true,
ExcludeSharedTokenCacheCredential = true,
ExcludeInteractiveBrowserCredential = true,
ExcludeManagedIdentityCredential = false,
ExcludeVisualStudioCredential = false,
ManagedIdentityClientId = options.ManagementIdentityClientId,
}));

Selenium IE 11 test on Jenkins Master Server Using nunit3-console.exe

I have a Selenium IE 11 test written in C# that runs perfectly when run locally in Debug or Release. this code is deployed to a Win 10 box with Jenkins (NO SLAVES). The Jenkins "Build" is configured to build the code, copy the test.dll to a folder and then call nunit3-console.exe to run the test. the Jenkins Service is also configured with a Domain User Account as a Service Logon Account.
I can logon to the win 10 box as the (Jenkins) domain user, and open a cmd window and run the following with no problem at all
C:\Program Files (x86)\Jenkins\workspace\Prod Login>"C:\Program Files
(x86)\NUnit.org\nunit-console\nunit3-console.exe" C:\Prod
Login\Tests\bin\Debug\Tests.dll
but if you try to "Build" the solution via Jenkins Web UI, it has a problem during the Nunit test finding some elements after it does a few clicks and a submit.
I know the IE configuration is rock solid like I said the test runs fine in a command window when logged in as the (Jenkins) Domain account
here is the error I get:
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException :
Assert.Fail failed. ProcessError* By.XPath:
//input[#placeholder='Username'], Following element not found:
By.XPath: //input[#placeholder='Username'], Timed out after 75
seconds;
again I know its not the locator, it works with By.Id or By.Xpath in the command window, AND my Chrome, and Firefox test with the SAME code base just a different WebDriver all work.
I can't help but think it has something to do with the Identity that's being used by Jenkins and or Nunit
any help is greatly appreciated!!!
** UPDATE
I just tried to configure a Jenkins "Slave" with the service account
running as the Jenkins Domain user, no luck still...
I am unable to post just a comment, but I have a debug suggestion. I know what you are saying with "the selector works fine", so I am not suggesting that the selector is wrong in any way. However, I do suggest having the test spew the page source just before the failing command. Driver.PageSource. And post it somewhere you can get to it.
That way, you can identify that the driver definitely sees the elements in the DOM that it has available to it.
This will help eliminate a whole slew of potential problems. There is always the possibility that the driver is simply not seeing the html that these selectors would point to, even though we can see it with our own two eyes.

SQL Server Agent Job - SSIS - C# - "Access denied. " when trying to delete files

One of our Jobs that runs an SSIS package for deleting files with a script task (c# File.Delete) behaves in a strange way as below. Could anyone help us understand the reason for it?
Below are the basic conditions:
1. SQL Server Agent Service's Logon Account: DomainA\AAA
2. Owner of the Job: DomainA\AAA
3. DomainA\AAA is a member of local "Administrators" group
What we found strange is:
The job fails with the message "Access to the path E:\XXXX\pp.csv denied" when Full control access is given only to "Administrators" windows group and successes when Full control access is given directly to "DomainA\AAA" windows user.
Before the error message above, it says "The step was executed as: DomainA\AAA".
Version Info:
SQL Server 2008 SP2(10.0.4000)
Windows 2003 R2 x64 SP2
Note:
1. English messages above are my own translation from our language and it wouldn't be accurately equal to the ones in English version.
2. The SSIS package is simplified to have this one script task only for testing.
Can you check weather on the folder you have given all the permission(Read and write) for Administrators group. If this doesn't works please change the folder and try once.
Try this blog to get information about the permission on this directory and file. Hope, this code will help you to find out which permission you need to assign for this user.
http://craigot.blogspot.com/2012/09/ssis-checking-filefolder-permissions.html

Does TFS API require Visual Studio?

I am trying to check in, check out, GetLatest, etc. with the TFS API using C#.
For the development PC and an Admin user this works flawlessly.
however, on a dedicated maschine without VS 2010 installed and the TFS user being a non-Admin this does nto work.
I get the following error:
*
Access to the registry key
'HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0' is denied.
2012-10-08 14:58:30 [...] error : at
Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)
at Microsoft.Win32.RegistryKey.CreateSubKey(String subkey,
RegistryKeyPermissionCheck permissionCheck, RegistrySecurity
registrySecurity) at
Microsoft.TeamFoundation.Client.UIHost.get_UserRegistryRoot() at
Microsoft.TeamFoundation.VersionControl.Client.Workstation.get_GetLatestOnCheckout()
*
I wonder if this has to do with Administrator rights?
The TFS "user" is actually a website running under that under account, so it is and is not supposed to be an Admin.
I tried running a test console app with my own credentials on the same maschine, and it works. So this is a credentials.
Can anyone help?
Is it enough to make the website user account an Admin?
And: what does it want to access the registry when creating the local workspace?
We've seen this kind of error when you are running a website under the application pool identity but the application pool identity is set to not load the user profile (and therefore has no HKCU registry access).
In IIS, under the advanced settings, set "Load User Profile"=True
See http://geekswithblogs.net/ProjectLawson/archive/2009/05/05/iis-system.web.aspnethostingpermission-exception-on-windows-7-rc.aspx
.. for more info.

'Run As...' doesn't correctly create user environment

My C# application crashes under some circumstances when run with a non-admin user.
I'm experiencing a problem with Windows Server 2003 and I'm trying to find more information about it. It may be a problem on other Windows OS's.
It seems that if I create a non-admin user, and then run my application under this user with the 'Run as...' command, the users environment doesn't get set up correctly, and the TEMP environment variable points at C:\Windows\Temp instead of the users having their own Temp folder in the Documents and Settings profile. The user doesn't have permissions to this folder, so the application crashes with the .Net JIT compiler tries to write/read to this folder.
If I log on as this user, the situation is still wrong. I don't get the Environment being prepared thingy you normally get when logging on a new user, and my app still won't run without crashing during startup. Infact I've realized the user can't run calc.exe or other programs in the Windows folder. It appears that their environment is permanently messed up and I guess the only way forward is to delete their profile.
If I create a non-admin user, and log on as them before doing a 'Run as..', they're environment gets set up ok, and my application works.
I can't find any information on this problem or notes on whether Microsoft acknowledge it. Have you experienced this, or do you know where I can look to find more about it?
Consider using runas with a profile for the user if you are not.
C:\temp>runas RUNAS USAGE:
RUNAS [ [/noprofile | /profile] [/env] [/netonly] ]
/user: program
RUNAS [ [/noprofile | /profile] [/env] [/netonly] ]
/smartcard [/user:] program
/noprofile specifies that
the user's profile should not be
loaded.
This causes the application to load more quickly, but
can cause some applications to malfunction.
/profile specifies that the
user's profile should be loaded.
This is the default. /env to use
current environment instead of user's.
/netonly use if the
credentials specified are for remote
access only. /savecred to use credentials
previously saved by the user.
This option is not available on Windows XP Home
Edition
and will be ignored. /smartcard use if
the credentials are to be supplied
from a
smartcard. /user should be
in form USER#DOMAIN or DOMAIN\USER
program command line for EXE.
See below for examples
Examples:
runas /noprofile /user:mymachine\administrator cmd
runas /profile /env /user:mydomain\admin "mmc %windir%\system32\dsa.msc"
runas /env /user:user#domain.microsoft.com "notepad \"my file.txt\""
NOTE: Enter user's password only when
prompted. NOTE: USER#DOMAIN is not
compatible with /netonly. NOTE:
/profile is not compatible with
/netonly.

Categories