We Keep Coding

LibGit2Sharp: How to push a local repo commit to Azure DevOps remote repo using a Personal Access Token inside a custom HTTP authentication header?

I am trying to push a commit I made on my local repository to a remote counterpart, hosted on a private Azure DevOps server, using LibGit2Sharp programmatically.
As per the Azure documentation, the HTTPS OAuth enabled Personal Access Token needs to sent with the request in a custom Authentication header as 'Basic' with the Base64 encoded token:
var personalaccesstoken = "PATFROMWEB";
using (HttpClient client = new HttpClient()) {
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(Encoding.ASCII.GetBytes($":{personalaccesstoken}")));
using (HttpResponseMessage response = client.GetAsync(
"https://dev.azure.com/{organization}/{project}/_apis/build/builds?api-version=5.0").Result) {
response.EnsureSuccessStatusCode();
}
}
The LibGit2Sharp.CloneOptions class has a FetchOptions field which in turn has a CustomHeaders array that can be used to inject the authentication header during the clone operation, like the following (as mentioned in this issue):
CloneOptions cloneOptions = new() {
CredentialsProvider = (url, usernameFromUrl, types) => new UsernamePasswordCredentials {
Username = $"{USERNAME}",
Password = $"{ACCESSTOKEN}"
},
FetchOptions = new FetchOptions {
CustomHeaders = new[] {
$"Authorization: Basic {encodedToken}"
}
}
};
Repository.Clone(AzureUrl, LocalDirectory, cloneOptions);
And the clone process succeeds (I tested it as well as checked the source code :) )
However, the LibGit2Sharp.PushOptions does not have any such mechanism to inject authentication headers. I am limited to the following code:
PushOptions pushOptions = new()
{
CredentialsProvider = (url, usernameFromUrl, types) => new UsernamePasswordCredentials
{
Username = $"{USERNAME}",
Password = $"{PASSWORD}"
}
};
This is making my push operation fail with the following message:
Too many redirects or authentication replays
I checked the source code for Repository.Network.Push() on Github.
public virtual void Push(Remote remote, IEnumerable<string> pushRefSpecs, PushOptions pushOptions)
{
Ensure.ArgumentNotNull(remote, "remote");
Ensure.ArgumentNotNull(pushRefSpecs, "pushRefSpecs");
// Return early if there is nothing to push.
if (!pushRefSpecs.Any())
{
return;
}
if (pushOptions == null)
{
pushOptions = new PushOptions();
}
// Load the remote.
using (RemoteHandle remoteHandle = Proxy.git_remote_lookup(repository.Handle, remote.Name, true))
{
var callbacks = new RemoteCallbacks(pushOptions);
GitRemoteCallbacks gitCallbacks = callbacks.GenerateCallbacks();
Proxy.git_remote_push(remoteHandle,
pushRefSpecs,
new GitPushOptions()
{
PackbuilderDegreeOfParallelism = pushOptions.PackbuilderDegreeOfParallelism,
RemoteCallbacks = gitCallbacks,
ProxyOptions = new GitProxyOptions { Version = 1 },
});
}
}
As we can see above, the Proxy.git_remote_push method call inside the Push() method is passing a new GitPushOptions object, which indeed seems to have a CustomHeaders field implemented. But it is not exposed to a consumer application and is being instantiated in the library code directly!
It is an absolute necessity for me to use the LibGit2Sharp API, and our end-to-end testing needs to be done on Azure DevOps repositories, so this issue is blocking me from progressing further.
My questions are:
Is it possible to use some other way to authenticate a push operation on Azure from LibGit2Sharp? Can we leverage the PushOptions.CredentialsProvider handler so that it is compatible with the auth-n method that Azure insists on?
Can we cache the credentials by calling Commands.Fetch by injecting the header in a FetchOptions object before carrying out the Push command? I tried it but it fails with the same error.
To address the issue, is there a modification required on the library to make it compatible with Azure Repos? If yes, then I can step up and contribute if someone could give me pointers on how the binding to the native code is made :)

I will provide an answer to my own question as we have fixed the problem.
The solution to this is really simple; I just needed to remove the CredentialsProvider delegate from the PushOptions object, that is:
var pushOptions = new PushOptions();
instead of,
PushOptions pushOptions = new()
{
CredentialsProvider = (url, usernameFromUrl, types) => new UsernamePasswordCredentials
{
Username = $"{USERNAME}",
Password = $"{PASSWORD}"
}
};
¯\(ツ)/¯
I don't know why it works, but it does. (Maybe some folks from Azure can clarify it to us.)

It turns out that this works on windows (push options with no credentials provider). Perhaps because somewhere a native call the OS resolves the credentials using some other means. But in Linux / container environment, the issue persists.
"There was a problem pushing the repo: remote authentication required but no callback set"
I think as you mentioned, minimally the CustomHeaders implementation must be exposed for this to work.
Image of error on console

Azure Blob how to properly create and consume a SAS token

Before I begin, allow me to say that I have scoured MSFTs docs, everything seems to imply that I need to manually handroll the GET request? Which seems wrong, given that the SDK handles that for us.
What I am experiencing from my Xamarin app is the following 403 error when I try to run my code to get a list of blobs.
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of the Authorization header is formed correctly including the signature.</Message></Error>
The way my workflow goes is as follows:
App Makes request to API to get Azure SAS token
API Responds using the following code to return the SAS token (blobServiceClient is defined using the default emulator storage connection string):
try
{
var client = blobServiceClient.GetBlobContainerClient(request.VenueId);
var permissions = BlobContainerSasPermissions.Read | BlobContainerSasPermissions.List;
var sas = client.GenerateSasUri(permissions, DateTimeOffset.Now.AddHours(1));
var containerUri = "";
#if DEBUG
var temp = sas.AbsoluteUri;
var replaced = temp.Replace("http://127.0.0.1:10000/", "http://myngrokproxy.ngrok.io/");
containerUri = replaced;
#else
containerUri = sas.AbsoluteUri;
#endif
//var sas = containerUri + container.GetSharedAccessSignature(accessPolicy);
return new AzureSASResponse
{
SAS = containerUri
};
} catch (Exception e)
{
return null;
}
As you can see, the replace is there since localhost URL is meaningless for the emulator.
App side I try to consume it as follows:
try
{
var uri = new Uri(SAS);
var containerClient = new BlobContainerClient(uri);
var blobs = containerClient.GetBlobs().AsPages();
foreach(var blob in blobs)
{
Console.WriteLine(blob);
}
} catch (Exception e)
{
Console.WriteLine(e);
}
My resulting SAS token looks like so: "http://myngrokproxy.ngrok.io/devstoreaccount1/8dc9e4831d634629b386680ad7c9a324?sv=2020-08-04&se=2021-10-21T21%3A43%3A16Z&sr=c&sp=rl&sig=oncjUlSLMsOS3WbxUWqjXDp28WACYxxVqUElrK%2BYNlY%3D"
How can I go about A) setting the auth header on it, even the GET request that fails is the .GetBlobs method in the Xamarin app?

After much trial and error my ways to fix it were as follows:
Use latest version of azurite from Microsoft, I used the original old one (Arafuto/azurite)
change code to look as follows;
var sasBuilder = new BlobSasBuilder()
{
BlobContainerName = containerClient.Name,
Resource = "c",
StartsOn = DateTimeOffset.UtcNow.AddMinutes(-15),
ExpiresOn = DateTimeOffset.UtcNow.AddDays(7)
};
sasBuilder.SetPermissions(BlobSasPermissions.Read | BlobSasPermissions.List);
var client = blobServiceClient.GetBlobContainerClient(request.VenueId);
var permissions = BlobContainerSasPermissions.Read | BlobContainerSasPermissions.List;
var sas = client.GenerateSasUri(sasBuilder);
var containerUri = "";
#if DEBUG
var temp = sas.AbsoluteUri;
var replaced = temp.Replace("http://127.0.0.1:10000/", "http://myngrokproxy.ngrok.io/");
containerUri = replaced;
#else
containerUri = sas.AbsoluteUri;
#endif
return new AzureSASResponse
{
SAS = containerUri
};
The inspiration for the BlobSasBuilder came from this document: https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-user-delegation-sas-create-dotnet#get-a-user-delegation-sas-for-a-container

OAuth 2.0 Authorization for windows desktop application using HttpListener

I am writing a windows desktop application with External Authentication(Google, Facebook) in C#.
I'm using HttpListener to allow a user to get Barer token by External Authentication Service with ASP.NET Web API, but administrator privileges are required for that and I want run without admin mode.
My reference was Sample Desktop Application for Windows.
Is this the best practice for external authentication provider from C#? Or is there another way to do that?
This is my code to get Barer token by external provider:
public static async Task<string> RequestExternalAccessToken(string provider)
{
// Creates a redirect URI using an available port on the loopback address.
string redirectURI = string.Format("http://{0}:{1}/", IPAddress.Loopback, GetRandomUnusedPort());
// Creates an HttpListener to listen for requests on that redirect URI.
var http = new HttpListener();
http.Prefixes.Add(redirectURI);
http.Start();
// Creates the OAuth 2.0 authorization request.
string authorizationRequest = Properties.Settings.Default.Server
+ "/api/Account/ExternalLogin?provider="
+ provider
+ "&response_type=token&client_id=desktop"
+ "&redirect_uri="
+ redirectURI + "?";
// Opens request in the browser.
System.Diagnostics.Process.Start(authorizationRequest);
// Waits for the OAuth authorization response.
var context = await http.GetContextAsync();
// Sends an HTTP response to the browser.
var response = context.Response;
string responseString = string.Format("<html><head></head><body></body></html>");
var buffer = System.Text.Encoding.UTF8.GetBytes(responseString);
response.ContentLength64 = buffer.Length;
var responseOutput = response.OutputStream;
Task responseTask = responseOutput.WriteAsync(buffer, 0, buffer.Length).ContinueWith((task) =>
{
responseOutput.Close();
http.Stop();
Console.WriteLine("HTTP server stopped.");
});
// Checks for errors.
if (context.Request.QueryString.Get("access_token") == null)
{
throw new ApplicationException("Error connecting to server");
}
var externalToken = context.Request.QueryString.Get("access_token");
var path = "/api/Account/GetAccessToken";
var client = new RestClient(Properties.Settings.Default.Server + path);
RestRequest request = new RestRequest() { Method = Method.GET };
request.AddParameter("provider", provider);
request.AddParameter("AccessToken", externalToken);
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
var clientResponse = client.Execute(request);
if (clientResponse.StatusCode == HttpStatusCode.OK)
{
var responseObject = JsonConvert.DeserializeObject<dynamic>(clientResponse.Content);
return responseObject.access_token;
}
else
{
throw new ApplicationException("Error connecting to server", clientResponse.ErrorException);
}
}

I don't know about Facebook, but usually (I am experienced with Google OAuth2 and Azure AD as well as Azure AD B2C), the authentication provider allows you to use a custom URI scheme for the authentication callback, something like badcompany://auth
To acquire an authentication token I ended up implementing the following scheme (All code is presented without warranty and not to be copied thoughtlessly.)
1. Register an URI-handler when the app is started
You can register an URI-Handler by creating a key in the HKEY_CURRENT_USER/Software/Classes (hence no admin privileges needed) key in the Windows registry
The name of the key equals the URI prefix, badcompany in our case
The key contains an empty string value named URL Protocol
The key contains a subkey DefaultIcon for the icon (actually I do not know whether this is necessary), I used the path of the current executable
There is a subkey shell/open/command, whose default value determines the path of the command to execute when the URI is tried to be opened, **please note*, that the "%1" is necessary to pass the URI to the executable
this.EnsureKeyExists(Registry.CurrentUser, "Software/Classes/badcompany", "URL:BadCo Applications");
this.SetValue(Registry.CurrentUser, "Software/Classes/badcompany", "URL Protocol", string.Empty);
this.EnsureKeyExists(Registry.CurrentUser, "Software/Classes/badcompany/DefaultIcon", $"{location},1");
this.EnsureKeyExists(Registry.CurrentUser, "Software/Classes/badcompany/shell/open/command", $"\"{location}\" \"%1\"");
// ...
private void SetValue(RegistryKey rootKey, string keys, string valueName, string value)
{
var key = this.EnsureKeyExists(rootKey, keys);
key.SetValue(valueName, value);
}
private RegistryKey EnsureKeyExists(RegistryKey rootKey, string keys, string defaultValue = null)
{
if (rootKey == null)
{
throw new Exception("Root key is (null)");
}
var currentKey = rootKey;
foreach (var key in keys.Split('/'))
{
currentKey = currentKey.OpenSubKey(key, RegistryKeyPermissionCheck.ReadWriteSubTree)
?? currentKey.CreateSubKey(key, RegistryKeyPermissionCheck.ReadWriteSubTree);
if (currentKey == null)
{
throw new Exception("Could not get or create key");
}
}
if (defaultValue != null)
{
currentKey.SetValue(string.Empty, defaultValue);
}
return currentKey;
}
2. Open a pipe for IPC
Since you'll have to pass messages from one instance of your program to another, you'll have to open a named pipe that can be used for that purpose.
I called this code in a loop in a background Task
private async Task<string> ReceiveTextFromPipe(CancellationToken cancellationToken)
{
string receivedText;
PipeSecurity ps = new PipeSecurity();
System.Security.Principal.SecurityIdentifier sid = new System.Security.Principal.SecurityIdentifier(System.Security.Principal.WellKnownSidType.WorldSid, null);
PipeAccessRule par = new PipeAccessRule(sid, PipeAccessRights.ReadWrite, System.Security.AccessControl.AccessControlType.Allow);
ps.AddAccessRule(par);
using (var pipeStream = new NamedPipeServerStream(this._pipeName, PipeDirection.InOut, 1, PipeTransmissionMode.Message, PipeOptions.Asynchronous, 4096, 4096, ps))
{
await pipeStream.WaitForConnectionAsync(cancellationToken);
using (var streamReader = new StreamReader(pipeStream))
{
receivedText = await streamReader.ReadToEndAsync();
}
}
return receivedText;
}
3. Make sure that the application is started only once
This can be acquired using a Mutex.
internal class SingleInstanceChecker
{
private static Mutex Mutex { get; set; }
public static async Task EnsureIsSingleInstance(string id, Action onIsSingleInstance, Func<Task> onIsSecondaryInstance)
{
SingleInstanceChecker.Mutex = new Mutex(true, id, out var isOnlyInstance);
if (!isOnlyInstance)
{
await onIsSecondaryInstance();
Application.Current.Shutdown(0);
}
else
{
onIsSingleInstance();
}
}
}
When the mutex has been acquired by another instance, the application is not fully started, but
4. Handle being called with the authentication redirect URI
If it's the only (first) instance, it may handle the authentication redirect URI itself
Extract the token from the URI
Store the token (if necessary and/or wanted)
Use the token for requests
If it's a further instance
Pass the redirect URI to the first instance by using pipes
The first instance now performs the steps under 1.
Close the second instance
The URI is sent to the first instance with
using (var client = new NamedPipeClientStream(this._pipeName))
{
try
{
var millisecondsTimeout = 2000;
await client.ConnectAsync(millisecondsTimeout);
}
catch (Exception)
{
onSendFailed();
return;
}
if (!client.IsConnected)
{
onSendFailed();
}
using (StreamWriter writer = new StreamWriter(client))
{
writer.Write(stringToSend);
writer.Flush();
}
}

To add to Paul's excellent answer:
Identity Model Libraries are worth looking at - one of the things they'll do for you is Authorization Code Flow (PKCE) which is recommended for native apps
My preference is the same as Paul's - to use custom URI schemes - usability is better I think
Having said that, a loopback solution should work without admin rights for ports greater than 1024
If it helps there is some stuff on my blog about this - including a Nodejs / Electron sample you can run from here to see what a finished solution looks like.

Why Glav Web Cache does not have the data I saved when called from other instance?

I've tried searching for my problems but nothing seems to ask what I wanted to ask.
I'm working on a web service that generates and sends a kind of token from server to client, currently I'm using Glav CacheAdapter (the web cache kind)
When someone requested a data call, the server should generate a token then saves it to a cache, then sends the key to the client, the client then should send the same token to the server and it should be checked with the one in the cache, but somehow when the server generates the key it successfully creates and saves one (I tested when debugging), but when the client call sends the token (it is the same one) but somehow the cache does not contain any data.
>>>> Project A
>> Service
public string Generate()
{
AppServices.Cache.InnerCache.Add($"AuthenticationTokenCache:{xxx}", DateTime.Now.AddDays(1), new StringValue() { Value = xxx });
return key;
}
public bool Validate(string token)
{
return AppServices.Cache.InnerCache.Get<StringValue>($"AuthenticationTokenCache:{xxx}") != null;
}
>> WebAPI
public bool CallValidate(string token)
{
var xService = new Service();
return xService.Validate(token);
}
>>>> Project B
>> WebAPI
protected override bool RequestValidation(string token)
{
var client = new HttpClient();
var authURL = $"/api/CallValidate?token={token}";
var response = client.GetAsync(authURL).Result.Content;
string jsonContent = response.ReadAsStringAsync().Result;
var authResult = JsonConvert.DeserializeObject<bool>(jsonContent);
if(authResult)
{
return true;
}
}
Is the cache type I use wrong, or maybe there's something wrong that I don't realize is wrong?
And when I create new instance of the same class does the cache gets shared between those object or not?
I'm not really sure as to how the details of how caching works, any pointer as to reference reading material would be helpful too.
Thank you.

Authenticating against ReportExecution2005.asmx in .NET Core

I'm trying to execute an SSRS report in .NET Core.
Since .NET Core doesn't let you add service references, you have to use the WCF Connected Service to add a reference to the WSDL so it can generate .NET Core compatible code. This is what I did for ReportExecution2005.asmx (SQL Server 2016 if it matters).
I tried using the following to authenticate against the service:
var rsExec = new ReportExecutionServiceSoapClient(ReportExecutionServiceSoapClient.EndpointConfiguration.ReportExecutionServiceSoap,
new EndpointAddress("http://server/ReportServer/ReportExecution2005.asmx"))
{
ClientCredentials =
{
Windows =
{
AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation,
ClientCredential = new NetworkCredential("username", "password")
}
}
};
Also tried setting the Username object instead of Windows object, but either way the result is the following error:
MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was 'NTLM'.
Looking at Fiddler, the code isn't passing the credentials along.
This is the code that got generated off the WSDL
public ReportExecutionServiceSoapClient(EndpointConfiguration endpointConfiguration, System.ServiceModel.EndpointAddress remoteAddress)
: base(ReportExecutionServiceSoapClient.GetBindingForEndpoint(endpointConfiguration), remoteAddress)
{
this.Endpoint.Name = endpointConfiguration.ToString();
ConfigureEndpoint(this.Endpoint, this.ClientCredentials);
}
static partial void ConfigureEndpoint(System.ServiceModel.Description.ServiceEndpoint serviceEndpoint, System.ServiceModel.Description.ClientCredentials clientCredentials);
I may be mistaken, but isn't this calling the private method ConfigureEndpoint with the ClientCredentials object before the ClientCredentials object has even been set?
I'm not seeing any other way to configure the ClientCredentials or call ConfigureEndpoint, so how exactly are you supposed to authenticate? The other constructors are basically the same thing, except for one which takes in a Binding instead of an EndpointConfiguration. Any ideas?

After fighting with this for a day, I found an approach that seems to work, by using the only constructor that does not immediately call ConfigureEndpoint as pointed out in the question. If I create a binding that specifies NTLM, and I pass that binding along with a manually created endpoint, it works:
var binding = new BasicHttpBinding(BasicHttpSecurityMode.TransportCredentialOnly)
{
Security =
{
Transport = new HttpTransportSecurity {ClientCredentialType = HttpClientCredentialType.Ntlm}
}
};
var reportService = new CssbiReportService.ReportExecutionServiceSoapClient(binding,
new EndpointAddress("http://myserver/ReportServer/ReportExecution2005.asmx"));
This is working for me in .NET Core.

Edit: update the code for .NET Core
Unfortunately, I don't have SSRS here to test the code right now.
But, try this code (no error check):
// parameters of report (if any)
ParameterValue[] parameters = {new ParameterValue {Name = "ApontamentoID", Value = "364"}};
// connect to the service
ReportExecutionServiceSoapClient webServiceProxy =
new ReportExecutionServiceSoapClient(
ReportExecutionServiceSoapClient.EndpointConfiguration.ReportExecutionServiceSoap,
"http://report_server_url/ReportExecution2005.asmx?wsdl");
// logon the user
await webServiceProxy.LogonUserAsync("username", "password", null);
// ask for the report
await webServiceProxy.LoadReportAsync("/report_path", null);
await webServiceProxy.SetExecutionParametersAsync(parameters, null);
// you can use RenderStreamRequest too
RenderRequest request = new RenderRequest("pdf", null);
RenderResponse response = await webServiceProxy.RenderAsync(request);
// save to the disk
System.IO.File.WriteAllBytes(#"c:\temp\output.pdf", response.Result);
// logoff the user
await webServiceProxy.LogoffAsync();
// close
await webServiceProxy.CloseAsync();

var binding = new BasicHttpBinding(BasicHttpSecurityMode.TransportCredentialOnly)
{
Security =
{
Transport = new HttpTransportSecurity {
ClientCredentialType = HttpClientCredentialType.Ntlm
}
}
};
yourClient = ReportExecutionServiceSoapClient(rsBinding, rsEndpointAddress) {
ClientCredentials =
{ ...
^^^ This for NTLM.
Also, I was getting read-only errors trying to set some properties on the client after it had been created. In case it helps someone, properties must all be set at client-creation time to avoid this as per "yourClient" above.

I had the same problem, for me the following addition was helpful:
ReportExecutionServiceSoapClient rsClient = new ReportExecutionServiceSoapClient(rsBinding, rsEndpointAddress);
rsClient.ClientCredentials.Windows.AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;