We Keep Coding

Youtube v3 API captions downloading

I'm trying to download captions from some videos on Youtube using their nuget package. Here's some code:
var request = _youtube.Search.List("snippet,id");
request.Q = "Bill Gates";
request.MaxResults = 50;
request.Type = "video";
var results = request.Execute();
foreach (var result in results.Items)
{
var captionListRequest = _youtube.Captions.List("id,snippet", result.Id.VideoId);
var captionListResponse = captionListRequest.Execute();
var russianCaptions =
captionListResponse.Items.FirstOrDefault(c => c.Snippet.Language.ToLower() == "ru");
if (russianCaptions != null)
{
var downloadRequest = _youtube.Captions.Download(russianCaptions.Id);
downloadRequest.Tfmt = CaptionsResource.DownloadRequest.TfmtEnum.Srt;
var ms = new MemoryStream();
downloadRequest.Download(ms);
}
}
When the Download method is called I'm getting a weird Newtonsoft.JSON Exception that says:
Newtonsoft.Json.JsonReaderException: 'Unexpected character encountered while parsing value: T. Path '', line 0, position 0.'
at Newtonsoft.Json.JsonTextReader.ParseValue()
I've read some other threads on captions downloading problems and have tried to change my authorization workflow: first I've tried to use just the ApiKey but then also tried OAuth. Here's how it looks now:
var credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
new ClientSecrets
{
ClientId = "CLIENT_ID",
ClientSecret = "CLIENT_SECRET"
},
new[] { YouTubeService.Scope.YoutubeForceSsl },
"user",
CancellationToken.None,
new FileDataStore("Youtube.CaptionsCrawler")).Result;
_youtube = new YouTubeService(new BaseClientService.Initializer
{
ApplicationName = "LKS Captions downloader",
HttpClientInitializer = credential
});
So, is it even possible to do what I'm trying to achieve?
P.S. I was able to dig deep into the youtube nuget package and as I see, the actual message, that I get (that Newtonsoft.JSON is trying to deserialize, huh!) is "The permissions associated with the request are not sufficient to download the caption track. The request might not be properly authorized, or the video order might not have enabled third-party contributions for this caption."
So, do I have to be the video owner to download captions? But if so, how do other programs like Google2SRT work?

Found this post How to get "transcript" in youtube-api v3
You can get them via GET request on: http://video.google.com/timedtext?lang={LANG}&v={VIDEOID}
Example:
http://video.google.com/timedtext?lang=en&v=-osCkzoL53U
Note that they should have subtitles added, will not work if auto-generated.

Facebook Access Token keeps Expiring c#

I have a web application that runs a schedule job which pulls in the Facebook reviews from a page which I manage. Here is a snippet
public void Execute(IJobExecutionContext context)
{
//get api details from the web.config
var pageId = WebConfigurationManager.AppSettings["FacebookPageId"];
var token = WebConfigurationManager.AppSettings["FacebookAPIToken"];
if (!string.IsNullOrEmpty(token))
{
//create a facebook client object
var client = new FacebookClient(token);
//make a call to facebook to retrieve the json data
dynamic graphJson = client.Get(pageId + "?fields=ratings{review_text,reviewer,rating}").ToString();
//deserialize the json returned from facebook
ReviewDeserializeData reviews = JsonConvert.DeserializeObject<ReviewDeserializeData>(graphJson);
//loop through the deserialized data and pass each review to the import class
foreach (var rating in reviews.ratings.data)
{
var fbRating = new FacebookRating
{
RatingReviewerId = long.Parse(rating.reviewer.id),
StarRating = rating.rating,
ReviewerName = rating.reviewer.name,
ReviewText = rating.review_text
};
ImportFacebookRating.ImportTheFacebookRating(fbRating);
}
}
}
This works great until the Page Access Token expires. I have tried following many articles such as this one https://medium.com/#Jenananthan/how-to-create-non-expiry-facebook-page-token-6505c642d0b1#.24vb5pyiv but i have had no luck fixing the token expiring.
Does anyone know how i can achieve this or is there a way to programmatically generate a new token if the existing one has expired? at the moment i have it stored in the web.config as an app setting.
Thanks

I found the answer here and was able to generate a token that 'Never' Expires Long-lasting FB access-token for server to pull FB page info

Get youtube playlist by ID in youtube api .net

I am trying to download the most recent items from a YouTube playlist in a C# .NET program. Right now I have a program that successfully gets the necessary data from my channel's Uploads playlist using "channel.ContentDetails.RelatedPlaylists.Uploads;", which I got from the sample program on the API page. But I can't find any information in the api docs about how to switch that line or lines around it to get a playlist by ID rather than my own uploads.
This is not a duplicate because other examples on this page explain how to find it through an http link, as in "http://gdata.youtube.com/feeds/api/playlists/..." etc. I want to do it directly through the API's methods. The part of my code that downloads the data is included below.
private async Task Run()
{
var youtubeService = new YouTubeService(new BaseClientService.Initializer()
{
ApiKey = "API KEY HERE",
ApplicationName = this.GetType().ToString()
});
//MAYBE I NEED TO CHANGE THIS? SOMETHING LIKE
//'youtubeservice.Playlists.IDUNNOWHAT'
var channelsListRequest = youtubeService.Channels.List("contentDetails");
channelsListRequest.Id = "CHANNEL ID HERE";
// Retrieve the contentDetails part of the channel resource for the authenticated user's channel.
var channelsListResponse = await channelsListRequest.ExecuteAsync();
foreach (var channel in channelsListResponse.Items)
{
//OR MAYBE I NEED TO CHANGE THIS PART?
//LIKE 'channel.ContentDetails.SOMETHING
var uploadsListId = channel.ContentDetails.RelatedPlaylists.Uploads;
var nextPageToken = "";
while (nextPageToken != null)
{
var playlistItemsListRequest = youtubeService.PlaylistItems.List("snippet");
playlistItemsListRequest.PlaylistId = uploadsListId;
playlistItemsListRequest.MaxResults = 50;
playlistItemsListRequest.PageToken = nextPageToken;
// Retrieve the list of videos uploaded to the authenticated user's channel.
var playlistItemsListResponse = await playlistItemsListRequest.ExecuteAsync();
/*
* DO A BUNCH OF STUFF WITH THE YOUTUBE DATA
*/
nextPageToken = playlistItemsListResponse.NextPageToken;
}
}
}

In the documentation, it lists the playlists you can get by using ContentDetails.RelatedPlaylists:
likes
favorites
uploads
watchHistory
watchLater
Therefore, if you want to get the items for a playlist you created you won't be able to do it using ContentDetails.RelatedPlaylists, you'll have to provide the playlist ID. I believe it should work with the code you provided (might need a few tweaks) if you change
playlistItemsListRequest.PlaylistId = uploadsListId;
to use the ID of the playlist whose videos you want to get.

YouTube Data API v3: deleting videos using a service account: Unauthorized client or scope in request

I'm trying to delete one or more videos using a simple C# app (I intend to use a Windows Service later) and I'm getting this error:
Google.Apis.Auth.OAuth2.Responses.TokenResponseException: Error:"unauthorized_client", Description:"Unauthorized client or scope in request.", Uri:""
at Google.Apis.Requests.ClientServiceRequest`1.Execute() in c:\code\google.com\google-api-dotnet-client\default\Tools\Google.Apis.Release\bin\Debug\test\default\Src\GoogleApis\Apis\Requests\ClientServiceRequest.cs:line 93
Uploading videos works perfectly. For both operations, I use the same initialization method:
private static YouTubeService AuthorizeYoutubeService()
{
string serviceAccountEmail = "...#developer.gserviceaccount.com";
string keyFilePath = "Warehouse<...>.p12";
string userAccountEmail = "login#gmail.com";
if (!File.Exists(keyFilePath))
{
System.Windows.Forms.MessageBox.Show("Secret file not found!");
return null;
}
var scope = new string[] { YouTubeService.Scope.Youtube };
var cert = new X509Certificate2(keyFilePath, "notasecret", X509KeyStorageFlags.Exportable);
try
{
ServiceAccountCredential credential = new ServiceAccountCredential
(new ServiceAccountCredential.Initializer(serviceAccountEmail)
{
Scopes = scope,
User = userAccountEmail
}.FromCertificate(cert));
var service = new YouTubeService(new BaseClientService.Initializer
{
HttpClientInitializer = credential,
ApplicationName = "warehouse"
});
return service;
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
return null;
}
}
The difference compared to simply uploading videos, is the defined Scope: YouTubeService.Scope.YoutubeUpload. When I try to delete a video using it, I get an insufficientPermissions (403) error.
So after looking in the documentation I've changed it to YouTubeService.Scope.Youtube.
Here's the code I'm trying to use:
var youtubeService = AuthorizeYoutubeService();
foreach (string id in deleteIds)
{
var videoDeleteRequest = youtubeService.Videos.Delete(id);
var result = videoDeleteRequest.Execute();
}
Where deleteIds is a list of 11 character strings containing IDs of existing videos.
I have YouTube Data API enabled in the developers console.
I've installed the API via NuGet, I don't think there's anything wrong with the packages.
I'm quite new to Google development, and all similar questions were about the calendar API.
I appreciate any help.

What I ended up doing is reseting the list of apps connected to the Google account and setting it up again from scratch. My app was added 2 times for some reason.

Fetching google images using htmlagilitypack

I would like to execute a query on google images to fetch images using htmlagilitypack in c#.
For this I used an xpath request to the image
//*[#id="rg_s"]/div[1]/a/img
But it fails to fetch the image that way. What could be the correct way of doing this?

you can try this too : Here its possible to get the links of images by following
var links = HtmlDocument.DocumentNode.SelectNodes("//a").Where(a => a.InnerHtml.Contains("<img")).Select(b => b.Attributes["href"].Value).ToList();
foreach(var link in links)
{
// you can save the link or do your process here
}

Google keeps found images in div tags with class rg_di. Here is a query to get all links to images:
var links = hdoc.DocumentNode.SelectNodes(#"//div[#class='rg_di']/a")
.Select(a => a.GetAttributeValue("href", ""));

Searching google programmatically outside of their API's is against the TOS. Consider Google Custom Search or Bing Search API, both of which have established JSON and SOAP interfaces.
Both are free for a couple thousand queries per month and comply with the service's TOS.
Edit: Examples of using Bing API with C# below:
const string bingKey = "[your key here]";
var bing = new BingSearchContainer(new Uri("https://api.datamarket.azure.com/Bing/Search/"))
{
Credentials = new NetworkCredential(bingKey, bingKey)
};
var query = bing.Web("Jon Gallant blog", null, null, null, null, null, null, null);
var results = query.Execute();
foreach(var result in results)
{
Console.WriteLine(result.Url);
}
Console.ReadKey();
Google custom search API:
string apiKey = "Your api key";
string cx = "Your custom search engine id";
string query = "Your query";
var svc = new Google.Apis.Customsearch.v1.CustomsearchService(new BaseClientService.Initializer { ApiKey = apiKey });
var listRequest = svc.Cse.List(query);
listRequest.Cx = cx;
var search = listRequest.Fetch();
foreach (var result in search.Items)
{
Response.Output.WriteLine("Title: {0}", result.Title);
Response.Output.WriteLine("Link: {0}", result.Link);
}