We Keep Coding

LibGit2Sharp: How to push a local repo commit to Azure DevOps remote repo using a Personal Access Token inside a custom HTTP authentication header?

I am trying to push a commit I made on my local repository to a remote counterpart, hosted on a private Azure DevOps server, using LibGit2Sharp programmatically.
As per the Azure documentation, the HTTPS OAuth enabled Personal Access Token needs to sent with the request in a custom Authentication header as 'Basic' with the Base64 encoded token:
var personalaccesstoken = "PATFROMWEB";
using (HttpClient client = new HttpClient()) {
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(Encoding.ASCII.GetBytes($":{personalaccesstoken}")));
using (HttpResponseMessage response = client.GetAsync(
"https://dev.azure.com/{organization}/{project}/_apis/build/builds?api-version=5.0").Result) {
response.EnsureSuccessStatusCode();
}
}
The LibGit2Sharp.CloneOptions class has a FetchOptions field which in turn has a CustomHeaders array that can be used to inject the authentication header during the clone operation, like the following (as mentioned in this issue):
CloneOptions cloneOptions = new() {
CredentialsProvider = (url, usernameFromUrl, types) => new UsernamePasswordCredentials {
Username = $"{USERNAME}",
Password = $"{ACCESSTOKEN}"
},
FetchOptions = new FetchOptions {
CustomHeaders = new[] {
$"Authorization: Basic {encodedToken}"
}
}
};
Repository.Clone(AzureUrl, LocalDirectory, cloneOptions);
And the clone process succeeds (I tested it as well as checked the source code :) )
However, the LibGit2Sharp.PushOptions does not have any such mechanism to inject authentication headers. I am limited to the following code:
PushOptions pushOptions = new()
{
CredentialsProvider = (url, usernameFromUrl, types) => new UsernamePasswordCredentials
{
Username = $"{USERNAME}",
Password = $"{PASSWORD}"
}
};
This is making my push operation fail with the following message:
Too many redirects or authentication replays
I checked the source code for Repository.Network.Push() on Github.
public virtual void Push(Remote remote, IEnumerable<string> pushRefSpecs, PushOptions pushOptions)
{
Ensure.ArgumentNotNull(remote, "remote");
Ensure.ArgumentNotNull(pushRefSpecs, "pushRefSpecs");
// Return early if there is nothing to push.
if (!pushRefSpecs.Any())
{
return;
}
if (pushOptions == null)
{
pushOptions = new PushOptions();
}
// Load the remote.
using (RemoteHandle remoteHandle = Proxy.git_remote_lookup(repository.Handle, remote.Name, true))
{
var callbacks = new RemoteCallbacks(pushOptions);
GitRemoteCallbacks gitCallbacks = callbacks.GenerateCallbacks();
Proxy.git_remote_push(remoteHandle,
pushRefSpecs,
new GitPushOptions()
{
PackbuilderDegreeOfParallelism = pushOptions.PackbuilderDegreeOfParallelism,
RemoteCallbacks = gitCallbacks,
ProxyOptions = new GitProxyOptions { Version = 1 },
});
}
}
As we can see above, the Proxy.git_remote_push method call inside the Push() method is passing a new GitPushOptions object, which indeed seems to have a CustomHeaders field implemented. But it is not exposed to a consumer application and is being instantiated in the library code directly!
It is an absolute necessity for me to use the LibGit2Sharp API, and our end-to-end testing needs to be done on Azure DevOps repositories, so this issue is blocking me from progressing further.
My questions are:
Is it possible to use some other way to authenticate a push operation on Azure from LibGit2Sharp? Can we leverage the PushOptions.CredentialsProvider handler so that it is compatible with the auth-n method that Azure insists on?
Can we cache the credentials by calling Commands.Fetch by injecting the header in a FetchOptions object before carrying out the Push command? I tried it but it fails with the same error.
To address the issue, is there a modification required on the library to make it compatible with Azure Repos? If yes, then I can step up and contribute if someone could give me pointers on how the binding to the native code is made :)

I will provide an answer to my own question as we have fixed the problem.
The solution to this is really simple; I just needed to remove the CredentialsProvider delegate from the PushOptions object, that is:
var pushOptions = new PushOptions();
instead of,
PushOptions pushOptions = new()
{
CredentialsProvider = (url, usernameFromUrl, types) => new UsernamePasswordCredentials
{
Username = $"{USERNAME}",
Password = $"{PASSWORD}"
}
};
¯\(ツ)/¯
I don't know why it works, but it does. (Maybe some folks from Azure can clarify it to us.)

It turns out that this works on windows (push options with no credentials provider). Perhaps because somewhere a native call the OS resolves the credentials using some other means. But in Linux / container environment, the issue persists.
"There was a problem pushing the repo: remote authentication required but no callback set"
I think as you mentioned, minimally the CustomHeaders implementation must be exposed for this to work.
Image of error on console

Authenticating against ReportExecution2005.asmx in .NET Core

I'm trying to execute an SSRS report in .NET Core.
Since .NET Core doesn't let you add service references, you have to use the WCF Connected Service to add a reference to the WSDL so it can generate .NET Core compatible code. This is what I did for ReportExecution2005.asmx (SQL Server 2016 if it matters).
I tried using the following to authenticate against the service:
var rsExec = new ReportExecutionServiceSoapClient(ReportExecutionServiceSoapClient.EndpointConfiguration.ReportExecutionServiceSoap,
new EndpointAddress("http://server/ReportServer/ReportExecution2005.asmx"))
{
ClientCredentials =
{
Windows =
{
AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation,
ClientCredential = new NetworkCredential("username", "password")
}
}
};
Also tried setting the Username object instead of Windows object, but either way the result is the following error:
MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was 'NTLM'.
Looking at Fiddler, the code isn't passing the credentials along.
This is the code that got generated off the WSDL
public ReportExecutionServiceSoapClient(EndpointConfiguration endpointConfiguration, System.ServiceModel.EndpointAddress remoteAddress)
: base(ReportExecutionServiceSoapClient.GetBindingForEndpoint(endpointConfiguration), remoteAddress)
{
this.Endpoint.Name = endpointConfiguration.ToString();
ConfigureEndpoint(this.Endpoint, this.ClientCredentials);
}
static partial void ConfigureEndpoint(System.ServiceModel.Description.ServiceEndpoint serviceEndpoint, System.ServiceModel.Description.ClientCredentials clientCredentials);
I may be mistaken, but isn't this calling the private method ConfigureEndpoint with the ClientCredentials object before the ClientCredentials object has even been set?
I'm not seeing any other way to configure the ClientCredentials or call ConfigureEndpoint, so how exactly are you supposed to authenticate? The other constructors are basically the same thing, except for one which takes in a Binding instead of an EndpointConfiguration. Any ideas?

After fighting with this for a day, I found an approach that seems to work, by using the only constructor that does not immediately call ConfigureEndpoint as pointed out in the question. If I create a binding that specifies NTLM, and I pass that binding along with a manually created endpoint, it works:
var binding = new BasicHttpBinding(BasicHttpSecurityMode.TransportCredentialOnly)
{
Security =
{
Transport = new HttpTransportSecurity {ClientCredentialType = HttpClientCredentialType.Ntlm}
}
};
var reportService = new CssbiReportService.ReportExecutionServiceSoapClient(binding,
new EndpointAddress("http://myserver/ReportServer/ReportExecution2005.asmx"));
This is working for me in .NET Core.

Edit: update the code for .NET Core
Unfortunately, I don't have SSRS here to test the code right now.
But, try this code (no error check):
// parameters of report (if any)
ParameterValue[] parameters = {new ParameterValue {Name = "ApontamentoID", Value = "364"}};
// connect to the service
ReportExecutionServiceSoapClient webServiceProxy =
new ReportExecutionServiceSoapClient(
ReportExecutionServiceSoapClient.EndpointConfiguration.ReportExecutionServiceSoap,
"http://report_server_url/ReportExecution2005.asmx?wsdl");
// logon the user
await webServiceProxy.LogonUserAsync("username", "password", null);
// ask for the report
await webServiceProxy.LoadReportAsync("/report_path", null);
await webServiceProxy.SetExecutionParametersAsync(parameters, null);
// you can use RenderStreamRequest too
RenderRequest request = new RenderRequest("pdf", null);
RenderResponse response = await webServiceProxy.RenderAsync(request);
// save to the disk
System.IO.File.WriteAllBytes(#"c:\temp\output.pdf", response.Result);
// logoff the user
await webServiceProxy.LogoffAsync();
// close
await webServiceProxy.CloseAsync();

var binding = new BasicHttpBinding(BasicHttpSecurityMode.TransportCredentialOnly)
{
Security =
{
Transport = new HttpTransportSecurity {
ClientCredentialType = HttpClientCredentialType.Ntlm
}
}
};
yourClient = ReportExecutionServiceSoapClient(rsBinding, rsEndpointAddress) {
ClientCredentials =
{ ...
^^^ This for NTLM.
Also, I was getting read-only errors trying to set some properties on the client after it had been created. In case it helps someone, properties must all be set at client-creation time to avoid this as per "yourClient" above.

I had the same problem, for me the following addition was helpful:
ReportExecutionServiceSoapClient rsClient = new ReportExecutionServiceSoapClient(rsBinding, rsEndpointAddress);
rsClient.ClientCredentials.Windows.AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;

Testing Authentication For Self-Hosted WCF Service in an Integration Test

I am trying to consume a self-hosted WCF application using SSL and a custom authentication validator from within an integration test. So far I am able to self-host the service but I am not able to figure out how to consume it.
Here is the self-hosting code (it is not dependent on Web.Config, as far as I know):
[ClassInitialize]
public static void TestClassInitialize(TestContext testContext)
{
const string serviceAddress = "https://localhost/SelfHostedService";
Uri _svcEndpointUri = new Uri(serviceAddress);
var binding = new WSHttpBinding
{
Security =
{
Mode = SecurityMode.TransportWithMessageCredential,
Message = {ClientCredentialType = MessageCredentialType.UserName}
}
};
ServiceDebugBehavior debugBehavior = new ServiceDebugBehavior
{
IncludeExceptionDetailInFaults = true
};
MyServiceApi _api = new MyServiceApi();
ServiceHost _svcHost = new ServiceHost(_api, _svcEndpointUri);
_svcHost.Description.Behaviors.Remove<ServiceDebugBehavior>();
_svcHost.Description.Behaviors.Add(debugBehavior);
// Ensure that SSL certificate & authentication interceptor get used
ServiceCredentials credentials = new ServiceCredentials();
credentials.UserNameAuthentication.UserNamePasswordValidationMode = UserNamePasswordValidationMode.Custom;
credentials.UserNameAuthentication.CustomUserNamePasswordValidator = new MyCustomAuthenticationValidator();
credentials.ServiceCertificate.SetCertificate(StoreLocation.LocalMachine, StoreName.My, X509FindType.FindBySubjectName, "SubjectName");
_svcHost.Description.Behaviors.Remove<ServiceCredentials>();
_svcHost.Description.Behaviors.Add(credentials);
// Add IUbiquity and mex endpoints
Uri endpointAddress = new Uri(serviceAddress + "/UbiquityApi.svc");
_svcHost.AddServiceEndpoint(typeof (IUbiquityApi), binding, endpointAddress);
// Specify InstanceContextMode, which is required to self-host
var behavior = _svcHost.Description.Behaviors.Find<ServiceBehaviorAttribute>();
behavior.InstanceContextMode = InstanceContextMode.Single;
_svcHost.Open();
}
What I'd like to be able to do looks like this, but I have no idea how I'd go about accomplish this:
[TestMethod]
public void TestAuthentication(){
var api = _svcHost.MagicallyRetrieveServiceInstance();
api.Credentials = new MagicCredentials("my username", "my password");
Assert.AreEqual(3, api.AddNumbers(1,2));
// Also assert that I am authenticated
api.Credentials = new MagicCredentials("my username", "my password");
bool exceptionWasThrown = false;
try {
api.AddNumbers(1,2);
}
catch(NotLoggedInException l){ // or something
exceptionWasThrown = true;
}
Assert.IsTrue(exceptionWasThrown);
}
My ideal solution would allow me to retrieve the service contract from the service host, and allow me to set the credentials used for the service contract. I should only have to supply the credentials once to the service contract, and then I should be able to call methods directly, as if I were communicating over the wire (thus making this an integration test). How should I go about this?

To consume the web service, simply add the service as a service reference, and then use the service reference client.
Done right, this will take care of the bindings needed for authentication, effectively putting the WCF configurations under test.

Accessing list from Sharepoint Webservice with Sharepoint Online 2013

I am trying to access a list from sharepoint via the web services.
I have tried lots of different web reference URLS for my web service.
The list is found at :
example.com/sites/dms/_layouts/15/start.aspx#/Lists/Documents/AllItems.aspx
the Web service URL I am using now is
https://example.com/sites/dms/_vti_bin/lists.asmx
Obviously example.com is not the real URL.
when I run the code
service.GetList("Documents");
I get the error:
List does not exist.
The page you selected contains a list that does not exist. It may have been deleted by another user.
0x82000006
My full code (many things are just for testing purposes):
public void UpdateList()
{
MKLists.Lists service = GetService();
string targetSite = "https://mywebpage.com/sites/dms";
using (ClientContext ctx = ClaimClientContext.GetAuthenticatedContext(targetSite))
{
if (ctx != null)
{
ctx.Load(ctx.Web); // Query for Web
ctx.ExecuteQuery(); // Execute
string test = (ctx.Web.Title);
}
}
CookieCollection authCookie = ClaimClientContext.GetAuthenticatedCookies(targetSite, 925, 525);
service.CookieContainer = new CookieContainer();
service.CookieContainer.Add(authCookie);
XmlNode tester = service.GetList("Documents");
}
private MKLists.Lists GetService()
{
MKLists.Lists myService = new MKLists.Lists();
myService.Credentials = System.Net.CredentialCache.DefaultCredentials;
return myService;
}

change this line:
MKLists.Lists service = GetService();
with
MKLists.Lists service = new MKLists.Lists();
i hope this helps.
Edit
according to your comment in the answer here is the update #Michael
try changing your targetsite url to
string targetSite = "https://mywebpage.com/sites/dms/_vti_bin/Lists.asmx";
hope this time it helps

IT turns out it was to do with the subsites.. and this line solved it:
service.Url = "https://mywebpage.com/sites/dms/_vti_bin/lists.asmx";

I've found some users with the same issue.
They said that these links below solved that issue. Could you try it?
http://blogs.msdn.com/b/sharepointdev/archive/2011/05/12/connecting-to-sharepoint-online-web-services.aspx
http://www.wictorwilen.se/Post/How-to-do-active-authentication-to-Office-365-and-SharePoint-Online.aspx

Google+ API: How can I use RefreshTokens to avoid requesting access every time my app launches?

I'm trying to use the Google+ API to access info for the authenticated user. I've copied some code from one of the samples, which works fine (below), however I'm having trouble making it work in a way I can reuse the token across app-launches.
I tried capturing the "RefreshToken" property and using provider.RefreshToken() (amongst other things) and always get a 400 Bad Request response.
Does anyone know how to make this work, or know where I can find some samples? The Google Code site doesn't seem to cover this :-(
class Program
{
private const string Scope = "https://www.googleapis.com/auth/plus.me";
static void Main(string[] args)
{
var provider = new NativeApplicationClient(GoogleAuthenticationServer.Description);
provider.ClientIdentifier = "BLAH";
provider.ClientSecret = "BLAH";
var auth = new OAuth2Authenticator<NativeApplicationClient>(provider, GetAuthentication);
var plus = new PlusService(auth);
plus.Key = "BLAH";
var me = plus.People.Get("me").Fetch();
Console.WriteLine(me.DisplayName);
}
private static IAuthorizationState GetAuthentication(NativeApplicationClient arg)
{
// Get the auth URL:
IAuthorizationState state = new AuthorizationState(new[] { Scope });
state.Callback = new Uri(NativeApplicationClient.OutOfBandCallbackUrl);
Uri authUri = arg.RequestUserAuthorization(state);
// Request authorization from the user (by opening a browser window):
Process.Start(authUri.ToString());
Console.Write(" Authorization Code: ");
string authCode = Console.ReadLine();
Console.WriteLine();
// Retrieve the access token by using the authorization code:
return arg.ProcessUserAuthorization(authCode, state);
}
}

Here is an example. Make sure you add a string setting called RefreshToken and reference System.Security or find another way to safely store the refresh token.
private static byte[] aditionalEntropy = { 1, 2, 3, 4, 5 };
private static IAuthorizationState GetAuthorization(NativeApplicationClient arg)
{
// Get the auth URL:
IAuthorizationState state = new AuthorizationState(new[] { PlusService.Scopes.PlusMe.GetStringValue() });
state.Callback = new Uri(NativeApplicationClient.OutOfBandCallbackUrl);
string refreshToken = LoadRefreshToken();
if (!String.IsNullOrWhiteSpace(refreshToken))
{
state.RefreshToken = refreshToken;
if (arg.RefreshToken(state))
return state;
}
Uri authUri = arg.RequestUserAuthorization(state);
// Request authorization from the user (by opening a browser window):
Process.Start(authUri.ToString());
Console.Write(" Authorization Code: ");
string authCode = Console.ReadLine();
Console.WriteLine();
// Retrieve the access token by using the authorization code:
var result = arg.ProcessUserAuthorization(authCode, state);
StoreRefreshToken(state);
return result;
}
private static string LoadRefreshToken()
{
return Encoding.Unicode.GetString(ProtectedData.Unprotect(Convert.FromBase64String(Properties.Settings.Default.RefreshToken), aditionalEntropy, DataProtectionScope.CurrentUser));
}
private static void StoreRefreshToken(IAuthorizationState state)
{
Properties.Settings.Default.RefreshToken = Convert.ToBase64String(ProtectedData.Protect(Encoding.Unicode.GetBytes(state.RefreshToken), aditionalEntropy, DataProtectionScope.CurrentUser));
Properties.Settings.Default.Save();
}

The general idea is as follows:
You redirect the user to Google's Authorization Endpoint.
You obtain a short-lived Authorization Code.
You immediately exchange the Authorization Code for a long-lived Access Token using Google's Token Endpoint. The Access Token comes with an expiry date and a Refresh Token.
You make requests to Google's API using the Access Token.
You can reuse the Access Token for as many requests as you like until it expires. Then you can use the Refresh Token to request a new Access Token (which comes with a new expiry date and a new Refresh Token).
See also:
The OAuth 2.0 Authorization Protocol
Google's OAuth 2.0 documentation

I also had problems with getting "offline" authentication to work (i.e. acquiring authentication with a refresh token), and got HTTP-response 400 Bad request with a code similar to the OP's code. However, I got it to work with the line client.ClientCredentialApplicator = ClientCredentialApplicator.PostParameter(this.clientSecret); in the Authenticate-method. This is essential to get a working code -- I think this line forces the clientSecret to be sent as a POST-parameter to the server (instead of as a HTTP Basic Auth-parameter).
This solution assumes that you've already got a client ID, a client secret and a refresh-token. Note that you don't need to enter an access-token in the code. (A short-lived access-code is acquired "under the hood" from the Google server when sending the long-lived refresh-token with the line client.RefreshAuthorization(state);. This access-token is stored as part of the auth-variable, from where it is used to authorize the API-calls "under the hood".)
A code example that works for me with Google API v3 for accessing my Google Calendar:
class SomeClass
{
private string clientID = "XXXXXXXXX.apps.googleusercontent.com";
private string clientSecret = "MY_CLIENT_SECRET";
private string refreshToken = "MY_REFRESH_TOKEN";
private string primaryCal = "MY_GMAIL_ADDRESS";
private void button2_Click_1(object sender, EventArgs e)
{
try
{
NativeApplicationClient client = new NativeApplicationClient(GoogleAuthenticationServer.Description, this.clientID, this.clientSecret);
OAuth2Authenticator<NativeApplicationClient> auth = new OAuth2Authenticator<NativeApplicationClient>(client, Authenticate);
// Authenticated and ready for API calls...
// EITHER Calendar API calls (tested):
CalendarService cal = new CalendarService(auth);
EventsResource.ListRequest listrequest = cal.Events.List(this.primaryCal);
Google.Apis.Calendar.v3.Data.Events events = listrequest.Fetch();
// iterate the events and show them here.
// OR Plus API calls (not tested) - copied from OP's code:
var plus = new PlusService(auth);
plus.Key = "BLAH"; // don't know what this line does.
var me = plus.People.Get("me").Fetch();
Console.WriteLine(me.DisplayName);
// OR some other API calls...
}
catch (Exception ex)
{
Console.WriteLine("Error while communicating with Google servers. Try again(?). The error was:\r\n" + ex.Message + "\r\n\r\nInner exception:\r\n" + ex.InnerException.Message);
}
}
private IAuthorizationState Authenticate(NativeApplicationClient client)
{
IAuthorizationState state = new AuthorizationState(new string[] { }) { RefreshToken = this.refreshToken };
// IMPORTANT - does not work without:
client.ClientCredentialApplicator = ClientCredentialApplicator.PostParameter(this.clientSecret);
client.RefreshAuthorization(state);
return state;
}
}

The OAuth 2.0 spec is not yet finished, and there is a smattering of spec implementations out there across the various clients and services that cause these errors to appear. Mostly likely you're doing everything right, but the DotNetOpenAuth version you're using implements a different draft of OAuth 2.0 than Google is currently implementing. Neither part is "right", since the spec isn't yet finalized, but it makes compatibility something of a nightmare.
You can check that the DotNetOpenAuth version you're using is the latest (in case that helps, which it might), but ultimately you may need to either sit tight until the specs are finalized and everyone implements them correctly, or read the Google docs yourself (which presumably describe their version of OAuth 2.0) and implement one that specifically targets their draft version.

I would recommend looking at the "SampleHelper" project in the Samples solution of the Google .NET Client API:
Samples/SampleHelper/AuthorizationMgr.cs
This file shows both how to use Windows Protected Data to store a Refresh token, and it also shows how to use a Local Loopback Server and different techniques to capture the Access code instead of having the user enter it manually.
One of the samples in the library which use this method of authorization can be found below:
Samples/Tasks.CreateTasks/Program.cs