We Keep Coding

How to get a MSAL access token for SharePoint Online in a federated environment the non-interactive way in a non-interactive .Net console app?

The task as simple as to have a scheduled .NET console app which will download a file from SharePoint Online on a regular basis using AD domain user account.
If I use recommended way
var token = publicApplication.AcquireTokenByIntegratedWindowsAuth(scopes).ExecuteAsync().Result;
I'm getting
UriFormatException: Invalid URI: The hostname could not be parsed.
What does it mean? Which URI, hostname? Should I override something somewhere or add some special parameter?
I've googled thru this stuff a lot, and I have no idea where to look further, any advice will be appreciated.
P.S. I have no permissions to do anything on SharePoint side, I'm not a SP admin. I just have access to specific folder on the site from which I'm downloading the file. And also I have a code which works interactively:
WebRequest.DefaultWebProxy = WebRequest.GetSystemWebProxy();
WebRequest.DefaultWebProxy.Credentials = CredentialCache.DefaultNetworkCredentials;
var scopes = new string[] { "https://tenant.sharepoint.com/.default" };
var options = new PublicClientApplicationOptions()
{
TenantId = "tenant.com",
ClientId = "{872cd9fa-d31f-45e0-9eab-6e460a02d1f1}",//known Visual Studio Id
};
var publicApplication = PublicClientApplicationBuilder.CreateWithApplicationOptions(options).Build();
var token = publicApplication.AcquireTokenInteractive(scopes).WithLoginHint("name.surname#tenant.com").ExecuteAsync().Result;
But it shows a browser window
No questions asked, pop-up disappear, and I get the token which is used further to download a file from SPOnline using /_api/web/GetFileByServerRelativeUrl(' stuff.
So just run the app, see the popup, get the file downloaded. No interaction needed.
But this approach doesn't work if I put this routine really non-interactive:
Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application.

Turns out the non-interactive way is only possible using tenant-side registered application. Implemented using certificate authentication.
But surprisingly the token obtained by ConfidentialClientApplicationBuilder doesn't work the way I wanted/expected (scopes/user impersonation issues). So now we use Graph client approach.
This is the only way which works for me (.NetFramework 4.7.2):
using Azure.Identity;
using Microsoft.Graph;
//...
static async Task GetFile(GraphServiceClient graphClient, string fileName2get)
{
var fileitem = graphClient
.Sites["SiteGuidYouMayGetBy /sites/[your site name]/_api/site/id"]
.Drives["CrazyLongDriveIdYouMayGetByEnumeratingDrivesHere"]
.Root
.ItemWithPath($"/Path To The File starting from Drive Root/{fileName2get}")
.Content
.Request().GetResponseAsync();
var stream = fileitem.GetAwaiter().GetResult();
using (var fileStream = System.IO.File.Create($"C:/Temp/{fileName2get}"))
{
await stream.Content.CopyToAsync(fileStream);
}
}

Workspace Domain-wide Delegation with service account not working for one project, works for others

We have different projects on GCP we use them to access different Google APIs. Most of them for internal use only.
In this particular case, we have 2 projects, both use Service Account and both are allowed on Workspace Domain-wide Delegation on the same scopes. They are almost clones of each other.
I execute a simple request with the same code (Spreadsheet.Get()) with project 1 credentials it works. I execute the same request with project 2 credentials it doesn't work.
Since Workspace Domain-wide Delegation it's activated the spreadsheet its shared to my email and I connect to the API with my email too (works with project 1 so this is not the problem) (impersonating a user)
The only difference it's that one project has OAuth Consent Screen on external (only 100 users cause we use it internally only, anyways..) and the other one it's internal but this has nothing to do with this right?
Where the problem could come from? Do I need to recreate the project that doesn't work?
Here is the error message :
Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested
Edit to answer the comments but this code works depending on the service account we use
Generating the credentials:
internal static ServiceCredential GetApiCredentialsFromJson(string jsonCredentialsPath, string mailToMimic)
{
string jsonCertificate = File.ReadAllText(jsonCredentialsPath);
string privateKey = Regex.Match(jsonCertificate, #"(?<=""private_key"": "")(.*)(?="")").Value.Replace(#"\n", "");
string accountEmail = Regex.Match(jsonCertificate, #"(?<=""client_email"": "")(.*)(?="")").Value;
ServiceAccountCredential.Initializer credentials = new ServiceAccountCredential.Initializer(accountEmail)
{
Scopes = _scopes,
User = mailToMimic
}.FromPrivateKey(privateKey);
return new ServiceAccountCredential(credentials);
}
Using the credentials:
internal GoogleSheetService(ServiceCredential credentials)
{
SheetsService = new SheetsService(new BaseClientService.Initializer()
{
HttpClientInitializer = credentials
});
SheetsService.HttpClient.Timeout = TimeSpan.FromSeconds(100);
}
Client ID is allowed on the Drive, Ads and Spreadsheets scopes on the Workspace console.

The answer was simple, but we had to figure it out by ourselves.
The scopes you add in your app when you initialize the client need to be exactly the same scopes you added in the Google Admin wide-delegation page. Even if your app or part of your app don't need them all.
C# example:
private static readonly string[] _scopes = { DriveService.Scope.Drive, SheetsService.Scope.Spreadsheets, SlidesService.Scope.Presentations };
ServiceAccountCredential.Initializer credentials = new ServiceAccountCredential.Initializer(accountEmail)
{
Scopes = _scopes,
User = mailToMimic
}.FromPrivateKey(privateKey);
return new ServiceAccountCredential(credentials);
Here my app only needs SheetsService.Scope.Spreadsheets but I had to add DriveService.Scope.Drive and SlidesService.Scope.Presentations because the same client its used for other apps that need them.

Google Data API Authorization Redirect URI Mismatch

Background
I am wanting to write a small, personal web app in .NET Core 1.1 to interact with YouTube and make some things easier for me to do and I am following the tutorials/samples in Google's YouTube documentation. Sounds simple enough, right? ;)
Authenticating with Google's APIs seems impossible! I have done the following:
Created an account in the Google Developer Console
Created a new project in the Google Developer Console
Created a Web Application OAuth Client ID and added my Web App debug URI to the list of approved redirect URIs
Saved the json file provided after generating the OAuth Client ID to my system
In my application, my debug server url is set (and when my application launches in debug, it's using the url I set which is http://127.0.0.1:60077).
However, when I attempt to authenticate with Google's APIs, I recieve the following error:
That’s an error.
Error: redirect_uri_mismatch
The redirect URI in the request, http://127.0.0.1:63354/authorize/,
does not match the ones authorized for the OAuth client.
Problem
So now, for the problem. The only thing I can find when searching for a solution for this is people that say
just put the redirect URI in your approved redirect URIs
Unfortunately, the issue is that every single time my code attempts to authenticate with Google's APIs, the redirect URI it is using changes (the port changes even though I set a static port in the project's properties). I cannot seem to find a way to get it to use a static port. Any help or information would be awesome!
NOTE: Please don't say things like "why don't you just do it this other way that doesn't answer your question at all".
The code
client_id.json
{
"web": {
"client_id": "[MY_CLIENT_ID]",
"project_id": "[MY_PROJECT_ID]",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_secret": "[MY_CLIENT_SECRET]",
"redirect_uris": [
"http://127.0.0.1:60077/authorize/"
]
}
}
Method That Is Attempting to Use API
public async Task<IActionResult> Test()
{
string ClientIdPath = #"C:\Path\To\My\client_id.json";
UserCredential credential;
using (var stream = new FileStream(ClientIdPath, FileMode.Open, FileAccess.Read))
{
credential = await GoogleWebAuthorizationBroker.AuthorizeAsync(
GoogleClientSecrets.Load(stream).Secrets,
new[] { YouTubeService.Scope.YoutubeReadonly },
"user",
CancellationToken.None,
new FileDataStore(this.GetType().ToString())
);
}
var youtubeService = new YouTubeService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = this.GetType().ToString()
});
var channelsListRequest = youtubeService.Channels.List("contentDetails");
channelsListRequest.Mine = true;
// Retrieve the contentDetails part of the channel resource for the authenticated user's channel.
var channelsListResponse = await channelsListRequest.ExecuteAsync();
return Ok(channelsListResponse);
}
Project Properties

The Original Answer works, but it is NOT the best way to do this for an ASP.NET Web Application. See the update below for a better way to handle the flow for an ASP.NET Web Application.
Original Answer
So, I figured this out. The issue is that Google thinks of a web app as a JavaScript based web application and NOT a web app with server side processing. Thus, you CANNOT create a Web Application OAuth Client ID in the Google Developer Console for a server based web application.
The solution is to select the type Other when creating an OAuth Client ID in the Google Developer Console. This will have Google treat it as an installed application and NOT a JavaScript application, thus not requiring a redirect URI to handle the callback.
It's somewhat confusing as Google's documentation for .NET tells you to create a Web App OAuth Client ID.
Feb 16, 2018 Updated Better Answer:
I wanted to provide an update to this answer. Though, what I said above works, this is NOT the best way to implement the OAuth workflow for a ASP.NET solution. There is a better way which actually uses a proper OAuth 2.0 flow. Google's documentation is terrible in regards to this (especially for .NET), so I'll provide a simple implementation example here. The sample is using ASP.NET core, but it's easily adapted to the full .NET framework :)
Note: Google does have a Google.Apis.Auth.MVC package to help simplifiy this OAuth 2.0 flow, but unfortunately it's coupled to a specific MVC implementation and does not work for ASP.NET Core or Web API. So, I wouldn't use it. The example I'll be giving will work for ALL ASP.NET applications. This same code flow can be used for any of the Google APIs you've enabled as it's dependent on the scopes you are requesting.
Also, I am assuming you have your application set up in your Google Developer dashboard. That is to say that you have created an application, enabled the necessary YouTube APIs, created a Web Application Client, and set your allowed redirect urls properly.
The flow will work like this:
The user clicks a button (e.g. Add YouTube)
The View calls a method on the Controller to obtain an Authorization URL
On the controller method, we ask Google to give us an Authorization URL based on our client credentials (the ones created in the Google Developer Dashboard) and provide Google with a Redirect URL for our application (this Redirect URL must be in your list of accepted Redirect URLs for your Google Application)
Google gives us back an Authorization URL
We redirect the user to that Authorization URL
User grants our application access
Google gives our application back a special access code using the Redirect URL we provided Google on the request
We use that access code to get the Oauth tokens for the user
We save the Oauth tokens for the user
You need the following NuGet Packages
Google.Apis
Google.Apis.Auth
Google.Apis.Core
Google.apis.YouTube.v3
The Model
public class ExampleModel
{
public bool UserHasYoutubeToken { get; set; }
}
The Controller
public class ExampleController : Controller
{
// I'm assuming you have some sort of service that can read users from and update users to your database
private IUserService userService;
public ExampleController(IUserService userService)
{
this.userService = userService;
}
public async Task<IActionResult> Index()
{
var userId = // Get your user's ID however you get it
// I'm assuming you have some way of knowing if a user has an access token for YouTube or not
var userHasToken = this.userService.UserHasYoutubeToken(userId);
var model = new ExampleModel { UserHasYoutubeToken = userHasToken }
return View(model);
}
// This is a method we'll use to obtain the authorization code flow
private AuthorizationCodeFlow GetGoogleAuthorizationCodeFlow(params string[] scopes)
{
var clientIdPath = #"C:\Path\To\My\client_id.json";
using (var fileStream = new FileStream(clientIdPath, FileMode.Open, FileAccess.Read))
{
var clientSecrets = GoogleClientSecrets.Load(stream).Secrets;
var initializer = new GoogleAuthorizationCodeFlow.Initializer { ClientSecrets = clientSecrets, Scopes = scopes };
var googleAuthorizationCodeFlow = new GoogleAuthorizationCodeFlow(initializer);
return googleAuthorizationCodeFlow;
}
}
// This is a route that your View will call (we'll call it using JQuery)
[HttpPost]
public async Task<string> GetAuthorizationUrl()
{
// First, we need to build a redirect url that Google will use to redirect back to the application after the user grants access
var protocol = Request.IsHttps ? "https" : "http";
var redirectUrl = $"{protocol}://{Request.Host}/{Url.Action(nameof(this.GetYoutubeAuthenticationToken)).TrimStart('/')}";
// Next, let's define the scopes we'll be accessing. We are requesting YouTubeForceSsl so we can manage a user's YouTube account.
var scopes = new[] { YouTubeService.Scope.YoutubeForceSsl };
// Now, let's grab the AuthorizationCodeFlow that will generate a unique authorization URL to redirect our user to
var googleAuthorizationCodeFlow = this.GetGoogleAuthorizationCodeFlow(scopes);
var codeRequestUrl = googleAuthorizationCodeFlow.CreateAuthorizationCodeRequest(redirectUrl);
codeRequestUrl.ResponseType = "code";
// Build the url
var authorizationUrl = codeRequestUrl.Build();
// Give it back to our caller for the redirect
return authorizationUrl;
}
public async Task<IActionResult> GetYoutubeAuthenticationToken([FromQuery] string code)
{
if(string.IsNullOrEmpty(code))
{
/*
This means the user canceled and did not grant us access. In this case, there will be a query parameter
on the request URL called 'error' that will have the error message. You can handle this case however.
Here, we'll just not do anything, but you should write code to handle this case however your application
needs to.
*/
}
// The userId is the ID of the user as it relates to YOUR application (NOT their Youtube Id).
// This is the User ID that you assigned them whenever they signed up or however you uniquely identify people using your application
var userId = // Get your user's ID however you do (whether it's on a claim or you have it stored in session or somewhere else)
// We need to build the same redirect url again. Google uses this for validaiton I think...? Not sure what it's used for
// at this stage, I just know we need it :)
var protocol = Request.IsHttps ? "https" : "http";
var redirectUrl = $"{protocol}://{Request.Host}/{Url.Action(nameof(this.GetYoutubeAuthenticationToken)).TrimStart('/')}";
// Now, let's ask Youtube for our OAuth token that will let us do awesome things for the user
var scopes = new[] { YouTubeService.Scope.YoutubeForceSsl };
var googleAuthorizationCodeFlow = this.GetYoutubeAuthorizationCodeFlow(scopes);
var token = await googleAuthorizationCodeFlow.ExchangeCodeForTokenAsync(userId, code, redirectUrl, CancellationToken.None);
// Now, you need to store this token in rlation to your user. So, however you save your user data, just make sure you
// save the token for your user. This is the token you'll use to build up the UserCredentials needed to act on behalf
// of the user.
var tokenJson = JsonConvert.SerializeObject(token);
await this.userService.SaveUserToken(userId, tokenJson);
// Now that we've got access to the user's YouTube account, let's get back
// to our application :)
return RedirectToAction(nameof(this.Index));
}
}
The View
#using YourApplication.Controllers
#model YourApplication.Models.ExampleModel
<div>
#if(Model.UserHasYoutubeToken)
{
<p>YAY! We have access to your YouTube account!</p>
}
else
{
<button id="addYoutube">Add YouTube</button>
}
</div>
<script>
$(document).ready(function () {
var addYoutubeUrl = '#Url.Action(nameof(ExampleController.GetAuthorizationUrl))';
// When the user clicks the 'Add YouTube' button, we'll call the server
// to get the Authorization URL Google built for us, then redirect the
// user to it.
$('#addYoutube').click(function () {
$.post(addYoutubeUrl, function (result) {
if (result) {
window.location.href = result;
}
});
});
});
</script>

As referred here, you need to specify a fix port for the ASP.NET development server like How to fix a port number in asp.NET development server and add this url with the fix port to the allowed urls. Also as stated in this thread, when your browser redirects the user to Google's oAuth page, you should be passing as a parameter the redirect URI you want Google's server to return to with the token response.

I noticed that there is easy non-programmatic way around.
If you have typical monotlith application built in typical MS convention(so not compatible with 12factor and typical DDD) there is an option to tell your Proxy WWW server to rewrite all requests from HTTP to HTTPS so even if you have set up Web App on http://localhost:5000 and then added in Google API url like: http://your.domain.net/sigin-google, it will work perfectly and it is not that bas because it is much safer to set up main WWW to rewrite all to HTTPS.
It is not very good practice I guess however it makes sense and does the job.

I've struggled with this issue for hours in a .net Core application. What finally fixed it for me was, in the Google developers console, to create and use a credential for "Desktop app" instead of a "Web application".

Yeah!! Using credentials of desktop app instead of web app worked for me fine. It took me more than 2 days to figure out this problem. The main problem is that google auth library dose not adding or supporting http://localhost:8000 as redirect uri for web app creds but credentials of desktop app fixed that issue. Cause its supporting http://___ connection instead of https: connection for redirect uri

Google Coordinate API authentication offline

Trying to authenticate the Google coordinate api. Tried the Service acount authentication usisng service account and posted stack flow with this question. Found this anwer and quite describes my question.
What the problem now is the library used is deprecated. Not able to execute the solution said.
var auth = new OAuth2Authenticator<WebServerClient> (provider, GetAuthorization);
// Create the service.
var service = new CoordinateService(new BaseClientService.Initializer()
{
Authenticator = auth
});
can anybody suggest a way to achieve the above code. I have installed the new version for Google APIs OAuth2 Client Library. But didn't found any similar code.
I am able to do the read the api using the below code snippet
using Google.Apis.Auth.OAuth2;
using Google.Apis.Coordinate.v1;
using Google.Apis.Coordinate.v1.Data;
using Google.Apis.Services;
using (var stream = new FileStream(System.Web.HttpContext.Current.Server.MapPath(#"../client_secret.json"), FileMode.Open, FileAccess.Read))
{
credential = await GoogleWebAuthorizationBroker.AuthorizeAsync(GoogleClientSecrets.Load(stream).Secrets,
new[] { CoordinateService.Scope.Coordinate },
"user", CancellationToken.None);
}
// Create the service.
var service = new CoordinateService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = "test",
});
//builds with time last day 12 am
var locationReq = service.Location.List(teamId, workerMail, (ulong)DateTime.Today.AddDays(-1).ToUniversalTime().Subtract(new DateTime(1970, 1, 1)).TotalSeconds);
var locationResult = locationReq.Execute();
but this method reuires a redirection for the first time. I can't do that in my scenario. so need an offline solution.

Need to authenticate using a browser instance first and can reuse same "refresh token" for all other requests. We can provide a custom folder location to store the "auth token" and the library will use stored token from that folder. Modified source code pasted below:
using (var stream = new FileStream(#"client_secret.json", FileMode.Open, FileAccess.Read))
{
credential = await GoogleWebAuthorizationBroker.AuthorizeAsync(GoogleClientSecrets.Load(stream).Secrets,
new[] { CoordinateService.Scope.Coordinate },
"user", CancellationToken.None,new FileDataStore(folder));
}
MG

It looks like you are using a very old version of the library (pre GA).
I recommend you to download the latest Coordinate API, which is available in NuGet - https://www.nuget.org/packages/Google.Apis.Coordinate.v1/.
Then, follow the get started and the OAuth 2.0 pages. You should find all the right documentation there, if something is missing - let us know. Open an issue in our issue tracker.

c# Exceptions using Oauth2 with Google Drive API (using 4.0 .Net)

I am using a code example at http://www.daimto.com/google-oauth2-csharp/ which would allow me to connect to the google drive api and get authorized using oauth2. I downloaded the .json file that has my client id and client secret and put it in the \bin\debug output folder (and a few other places out of desperation) and when the code progresses to opening that file, it causes an exception which says 'stream handle' threw an exception of type 'System.Object.DisposeException'
The code throws an exception here (at the credential line) when stepping through:
using (var stream = new FileStream("client_secrets.json", FileMode.Open, FileAccess.Read))
{
GoogleWebAuthorizationBroker.Folder = "Tasks.Auth.Store";
credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
GoogleClientSecrets.Load(stream).Secrets,
new[] { DriveService.Scope.Drive,
DriveService.Scope.DriveFile },
"user",
CancellationToken.None,
new FileDataStore("Drive.Auth.Store")).Result;
}
This is probably the 4th or 5th code example I have tried, and none of them seem to work. It is quite disheartening.

I can see that that tutorial is very out of date. I am not sure why you are unable to use stream reader to read the file. But I think that you will find this code much cleaner and easer to understand.
string _client_id = "from apis console";
string _client_secret = "frome apis console";
string[] scopes = new string[] {DriveService.Scope.Drive,
DriveService.Scope.DriveFile };
// here is where we Request the user to give us access,
// or use the Refresh Token that was previously stored in %AppData%
UserCredential credential =
GoogleWebAuthorizationBroker.AuthorizeAsync(new ClientSecrets {ClientId = _client_id,
ClientSecret = _client_secret },
scopes,
Environment.UserName,
CancellationToken.None,
new FileDataStore("Daimto.GooglePlus.Auth.Store")
).Result;
I will update that tutorial ASAP.

You are using an Async method to do your read. This is normally fine, but that means the code is still executing when you get to the end of your using statement. When the end bracket of the using is reached, the filestream is disposed while your async method is still trying to use it.
Try removing your using statement and instead disposing of the stream once your authorization is finished.