Google has a 100 search quota, which is way too low to be of any use. Every time I run a search, the quota increases by 1. Here's the code:
string apiKey = "(MY API KEY HERE)";
string cx = "(MY CUSTOM SEARCH ENGINE CODE HERE)";
var tempi = 0;
var svc = new Google.Apis.Customsearch.v1.CustomsearchService(new BaseClientService.Initializer { ApiKey = apiKey });
string query = "test"
potato = 0;
var listRequest = svc.Cse.List(query);
listRequest.Cx = cx;
var search = listRequest.Execute();
foreach (var result in search.Items.Take(3))
{
if (potato == 0)
{
console.WriteLine("**Title:** " + result.Title + "\n**Link:** " + result.Link);
potato += 1;
}
}
Is there any way of using this without having to use up the query every time? If not, is there any other API that can do something similar to this?
This is part of Google's commercial model. If you want to exceed the quota, then you need to start paying for the service.
You can find the pricing here.
Related
I need to get the results of google searches in order to loop through and parse them. With that aim in view, I followed (as best I could) the tutorial on how to do that here
This is my code, based on the sample/example code in the article referenced above:
private void btnRentFlick_Click(object sender, EventArgs e)
{
OpenBestPageForSearchString("rent amazon movie Will Penny");
}
private void OpenBestPageForSearchString(string searchStr)
{
try
{
const string apiKey = "blaBlaBla"; // "blaBlaBla" stands for my API key
const string searchEngineId = "bla"; // "bla" stands for various things I tried: my client_id
(also called UniqueId), private_key_id (also called KeyId), and project_id. Not having
the correct value may be the problem. If so, how do I get it?
const string query = "rent amazon movie Will Penny";
var customSearchService = new CustomsearchService(new BaseClientService.Initializer { ApiKey
= apiKey });
//CseResource.ListRequest listRequest = customSearchService.Cse.List(query); // This is the
code in the article, but it won't compile - "no overload for "List" takes one argument"
// So how is the value in "query" assigned, then?
CseResource.ListRequest listRequest = customSearchService.Cse.List();
listRequest.Cx = searchEngineId;
List<string> linksReturned = new List<string>();
IList<Result> paging = new List<Result>();
var count = 0; // I don't know what the purpose of the counting is, but I'll leave as-is
until I get it working at least
while (paging != null)
{
listRequest.Start = count * 10 + 1;
paging = listRequest.Execute().Items; // this takes several seconds, then it throws an
exception
if (paging != null)
{
foreach (var item in paging)
{
linksReturned.Add("Title : " + item.Title + Environment.NewLine + "Link : " +
item.Link +
Environment.NewLine + Environment.NewLine);
}
}
count++;
}
MessageBox.Show("Done with google amazon query");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
As the comment at the end of that line says, this line of code:
paging = listRequest.Execute().Items;
...works for several seconds, then throws an exception, namely this:
So what is causing this exception? Is it because the searchEngineId value I assigned is bad? Or is it because the search string (assigned to the query variable) has not been provided to the call?
The info about my Ids is contained in a .json file provided by google, and there is no "searchEngineId" value in it. This is what it does contain:
"type": "service_account", "project_id": "flix4famsasinlocator",
"private_key_id": "[my private key id]", "private_key": "-----BEGIN
PRIVATE KEY-----. . . PRIVATE KEY-----\n", "client_email":
"[bla].gserviceaccount.com", "client_id": "[my client Id]",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url":
"https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url":
"https://www.googleapis.com/robot/v1/metadata/x509/[bla]gserviceaccount.com"
So though the article previously mentioned purported to be, and at first appeared to be, just what the doctor ordered, I have ran into a wall of considerable dimensions. Does anybody know how to scale this wall - perhaps primarily by providing the search string to the CseResource.ListRequest object?
UPDATE
Trying DalmTo's code first, I used this (not showing his GetService() method, which I copied verbatim):
var query = "rent amazon movie Will Penny";
var service = GetService("theRainInSpainFallsMainlyOnTheDirt");
var request = service.Cse.List();
// add option values to the request here.
request.ExactTerms = query;
request.Q = query;
var response = request.ExecuteAsync();
// my contribution:
List<string> linksReturned = new List<string>();
foreach (var item in response.Result.Items)
{
//Console.WriteLine(item.Title);
// next two lines also mine
MessageBox.Show(string.Format("Title: {0}; Link: {1}; ETag: {2}", item.Title, item.Link, item.ETag));
linksReturned.Add(item.Link);
}
...but this exception was thrown while in the foreach loop:
UPDATE 2
Yes, this works (adapted from Trekco's answer):
const string apiKey = "gr8GooglyMoogly";
const string searchEngineId = "theRainInSpainFallsMainOnTheDirt";
const string query = "rent amazon movie Will Penny";
var customSearchService = new CustomsearchService(new BaseClientService.Initializer { ApiKey = apiKey });
CseResource.ListRequest listRequest = customSearchService.Cse.List();
listRequest.Cx = searchEngineId;
listRequest.Q = query;
List<string> linksReturned = new List<string>();
IList<Result> paging = new List<Result>();
var count = 0;
while (paging != null)
{
listRequest.Start = count * 10 + 1;
paging = listRequest.Execute().Items;
if (paging != null)
{
foreach (var item in paging)
{
linksReturned.Add(item.Link);
}
}
count++;
}
The query is not being send to google. To fix your code you need to tell the api what query to use. After listRequest.Cx = searchEngineId; add listRequest.Q = query;
var count = 0;
string apiKey = "THE API KEY";
string searchEngineId = "THE SEARCH ENGIN ID";
string query = "rent amazon movie Will Penny";
var customSearchService = new CustomsearchService(new BaseClientService.Initializer
{
ApiKey = apiKey
});
CseResource.ListRequest listRequest = customSearchService.Cse.List();
listRequest.Cx = searchEngineId;
listRequest.Q = query; // <---- Add this line
List<string> linksReturned = new List<string>();
while (count < 10) // Google limit you to 100 records
{
listRequest.Start = count * 10;
var paging = listRequest.Execute().Items;
foreach (var item in paging)
{
linksReturned.Add("Title : " + item.Title + Environment.NewLine + "Link : " +
item.Link +
Environment.NewLine + Environment.NewLine);
}
count++;
}
In your code you have a comment that you don't know what var count = 0; is for. It is to keep track on how many items you have requested.
If you look at google's documentation you will see that they will only return 100 results max. After that they will give you a error. That error will also be the same generic message: "INVALID_ARGUMENT"
You can review the custom search api requirements here: https://developers.google.com/custom-search/v1/reference/rest/v1/cse/list
The searchEngineId variable is the search Engine id that you generate on the site https://www.google.com/cse/all. The documentation you followed is a bit out of date. you will find the id here:
If you check the doucmntation cse.list I think you will find that the list method has no required fields which means that you need to add the option values in the following manner.
I think ExactTerms may be the one you are looking for. But it could also be Q i think you should read though the option values and decide which one is best for your purpose.
var query = "rent amazon movie Will Penny";
var service = GetService("MYKEY");
var request = service.Cse.List();
// add option values to the request here.
request.ExactTerms = query;
request.Q = query;
var response = request.ExecuteAsync();
foreach (var item in response.Result.Items)
{
Console.WriteLine(item.Title);
}
My get service method
public static CustomsearchService GetService(string apiKey)
{
try
{
if (string.IsNullOrEmpty(apiKey))
throw new ArgumentNullException("api Key");
return new CustomsearchService(new BaseClientService.Initializer()
{
ApiKey = apiKey,
ApplicationName = string.Format("{0} API key example", System.Diagnostics.Process.GetCurrentProcess().ProcessName),
});
}
catch (Exception ex)
{
throw new Exception("Failed to create new Customsearch Service", ex);
}
}
i try to get the livechat Messages vie the c# API from a different channel.
To achieve this i Need the liveboradcast id.
I managed to get the live Video and id via search, but it seems this id isnt the livebroadcast id.
This is my Code so far.
As i said it Returns a Video and the ID, but the Broadcast Response with this id is 0.
Example:
"[GER/HD] Boss Riesenaffe/Megapithecus Hard, oder auch nicht ;) ARK: Survival Evolved (t3CwM9MJSNI)"
Anyone know where i can get the livebroadcast id !?
Stream SStream = new FileStream("client_secrets.json", FileMode.Open);
UserCredential Credentials = await GoogleWebAuthorizationBroker.AuthorizeAsync(GoogleClientSecrets.Load(SStream).Secrets, new[] { YouTubeService.Scope.YoutubeForceSsl }, "user", CancellationToken.None, new FileDataStore(this.GetType().ToString()));
Service = new YouTubeService(new BaseClientService.Initializer
{
HttpClientInitializer = Credentials,
ApplicationName = "name"
});
var searchListRequest = Service.Search.List("snippet");
searchListRequest.EventType = SearchResource.ListRequest.EventTypeEnum.Live;
searchListRequest.Type = "video";
searchListRequest.ChannelId = "thechannelid";
searchListRequest.MaxResults = 50;
var searchListResponse = await searchListRequest.ExecuteAsync();
List<string> videos = new List<string>();
string ID = null;
foreach (var searchResult in searchListResponse.Items)
{
switch (searchResult.Id.Kind)
{
case "youtube#video":
ID = searchResult.Id.VideoId;
videos.Add(String.Format("{0} ({1})", searchResult.Snippet.Title, searchResult.Id.VideoId));
break;
}
}
Console.WriteLine(String.Format("Videos:\n{0}\n", string.Join("\n", videos)));
LiveBroadcastsResource.ListRequest Request = Service.LiveBroadcasts.List("id,snippet,contentDetails,status");
Request.BroadcastType = LiveBroadcastsResource.ListRequest.BroadcastTypeEnum.All;
//Request.BroadcastStatus = LiveBroadcastsResource.ListRequest.BroadcastStatusEnum.Active;
Request.MaxResults = 10;
Request.Id = ID;
Console.WriteLine("ID: " + Request.Id);
//Request.Mine = false;
var BroadCastResponse = Request.Execute();
Console.WriteLine(BroadCastResponse.Items.Count);
foreach (LiveBroadcast c in BroadCastResponse.Items)
{
Console.WriteLine("Title: " + c.Snippet.Title);
}
AFAIK, you can only search broadcasts that the channel you are authenticated has created.
Try using search.list:
Returns a collection of search results that match the query parameters specified in the API request.
As stated in this related SO post, search.list returns video from a particular channel, without being authenticated as that channel/user, if you know that channel's channelId.
HTTPS Request:
HTTP GET https://www.googleapis.com/youtube/v3/search?part=snippet&channelId={channelId}&eventType=live&type=video&key={YOUR_API_KEY}
This particular piece of code for some reason runs much slower on production server (which is XL Azure Cloud Service) than on localhost. The worst thing is that this slowness is not consistent, i.e. it is slow most of the time, but sometimes it works fast. By slow I mean this piece of code takes 4.000ms to run on production (I know this because I'm using Azure AppInsights) , and only around 60ms on localhost. I cannot figure out why this is happening. Here's code (please note that this is only part of bigger method, but I'm sure that this part is slowest, everything else is much faster):
for (var i = 0; i < feedDeserialized.Length; i++)
{
RedisWorkoutItem workout;
bool hasRespected;
string username, fullName, profilePic;
// usersFromRedis is an array of Dictonary <string,string>
var userRedis = usersFromRedis[i];
// workoutsFromRedis is just an array of objects - RedisValue object
var stringWorkout = workoutsFromRedis[i];
// just an array of objects - RedisValue object
var workoutComment = commentsFromRedis[i].HasValue ? commentsFromRedis[i].ToString() : "";
if (userRedis != null)
{
profilePic = userRedis["ProfilePhotoUrl"].HasValue
? userRedis["ProfilePhotoUrl"].ToString()
: "";
fullName = userRedis["FirstName"] + " " + userRedis["LastName"];
username = userRedis["UserName"].HasValue ? userRedis["UserName"].ToString() : "";
}
//code inside this else statement never happens
else
{
var stopWatch2 = new Stopwatch();
stopWatch2.Start();
var user = databaseContext.Users.Find(feedDeserialized[i].UserId);
profilePic = user.ProfilePhotoUrl;
username = user.UserName;
fullName = user.FirstName + " " + user.LastName;
stopWatch2.Stop();
telemetryHelper.TrackEvent(_telemetryClient,
"CreateRedisFeedViewModelAsync: Went to DB for user", stopWatch2.Elapsed);
}
if (stringWorkout.HasValue)
{
workout = JsonConvert.DeserializeObject<RedisWorkoutItem>(stringWorkout);
hasRespected = workout.UsersWhoRespected.Contains(userId);
}
//code inside this else statement never happens
else
{
var stopWatch2 = new Stopwatch();
stopWatch2.Start();
var workoutGuid = Guid.Parse(feedDeserialized[i].WorkoutId);
var workoutFromDb = await databaseContext.Trenings.FindAsync(workoutGuid);
var routine = await databaseContext.AllRoutineses.FindAsync(workoutFromDb.AllRoutinesId);
workout = new RedisWorkoutItem
{
Name = routine.Name,
Id = workoutFromDb.TreningId.ToString(),
Comment = workoutFromDb.UsersCommentOnWorkout,
DateWhenFinished = workoutFromDb.DateTimeWhenTreningCreated,
NumberOfRespects = workoutFromDb.NumberOfLikes,
NumberOfComments = workoutFromDb.NumberOfComments,
UserId = workoutFromDb.UserId,
Length = workoutFromDb.LengthInSeconds,
Points = workoutFromDb.Score
};
workoutComment = workoutFromDb.UsersCommentOnWorkout;
hasRespected = databaseContext.TreningRespects
.FirstOrDefault(r => r.TreningId == workoutGuid && r.UserId == userId) != null;
stopWatch2.Stop();
telemetryHelper.TrackEvent(_telemetryClient,
"CreateRedisFeedViewModelAsync: Went to DB for workout", stopWatch2.Elapsed);
}
string workoutLength;
if (workout.Length >= 3600)
{
var t = TimeSpan.FromSeconds(workout.Length);
workoutLength = $"{t.Hours:D2}:{t.Minutes:D2}:{t.Seconds:D2}";
}
else
{
var t = TimeSpan.FromSeconds(workout.Length);
workoutLength = $"{t.Minutes:D2}:{t.Seconds:D2}";
}
listToReturn.Add(new FeedMobileHelper
{
Id = feedDeserialized[i].Id.ToString(),
UserId = workout.UserId,
WorkoutId = feedDeserialized[i].WorkoutId,
Points = workout.Points.ToString("N0", new NumberFormatInfo
{
NumberGroupSizes = new[] {3},
NumberGroupSeparator = "."
}),
WorkoutName = workout.Name,
WorkoutLength = workoutLength,
NumberOfRespects = workout.NumberOfRespects,
NumberOfComments = workout.NumberOfComments,
WorkoutComment = workoutComment,
HasRespected = hasRespected,
UserImageUrl = profilePic,
UserName = username,
DisplayName = string.IsNullOrWhiteSpace(fullName) ? username : fullName,
TimeStamp = workout.DateWhenFinished,
DateFormatted = workout.DateWhenFinished.FormatDateToHrsDaysWeeksString()
});
}
I cannot understand why this takes 4 seconds to complete on cloud service with 8 cores and 15 GB of RAM, especially because there is nothing fancy in here, no trips to database (as I noted in comments the "else parts" of code are never executed) or disc, everything is done in memory. And most of things in loop are done in constant time - O(1). Please can anyone help me to figure out what's problem here.
And one more thing, the API call which executes this part of code isn't called by 1000s of users at the same time, it is only called by me (I'm sure of this).
P.S.
feedDeserialized.Length is around 60
First thought - are you deploying Debug or Release build to Azure Cloud Service? Release build is optimized and much faster. If you right click on your cloud service and choose Publish... you can select which build to deploy. Also, you might want to enable profiling (under advanced settings as part of the publish wizard) to see the slow parts of your application.
I trying using Google Analytics with C# to get stats information to display in my webiste
Here is my code
public ActionResult Index()
{
string userName = "admin#email.com";
string passWord = "mypass";
string profileId = "ga:xxxxxxxx";
string key = "2d751338cb092ef8da65f716e37a48604386c9sw";
string dataFeedUrl = "https://www.google.com/analytics/feeds/data"+key;
var service = new AnalyticsService("API Project");
service.setUserCredentials(userName, passWord);
var dataQuery = new DataQuery(dataFeedUrl)
{
Ids = profileId,
Metrics = "ga:pageviews",
Sort = "ga:pageviews",
GAStartDate = new DateTime(2010, 3, 1).ToString("yyyy-MM-dd"),
GAEndDate = DateTime.Now.ToString("yyyy-MM-dd")
};
var dataFeed = service.Query(dataQuery);
var totalEntry = dataFeed.Entries[0];
ViewData["Total"] = ((DataEntry)(totalEntry)).Metrics[0].Value;
dataQuery.GAStartDate = DateTime.Now.AddDays(-1).ToString("yyyy-MM-dd");
dataQuery.GAEndDate = DateTime.Now.AddDays(-1).ToString("yyyy-MM-dd");
dataFeed = service.Query(dataQuery);
var yesterdayEntry = dataFeed.Entries[0];
ViewData["Yesterday"] = ((DataEntry)(yesterdayEntry)).Metrics[0].Value;
dataQuery.GAStartDate = DateTime.Now.ToString("yyyy-MM-dd");
dataQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");
dataFeed = service.Query(dataQuery);
var todayEntry = dataFeed.Entries[0];
ViewData["Today"] = ((DataEntry)(todayEntry)).Metrics[0].Value;
return View(dataFeed.Entries);
}
But when i run the code it always said "{"Invalid credentials"}"
Not sure why i facing this error while i checked many time about the key,username,password and profileId
Anyone facing this problem,can help me?
Many thanks
I think that your url is wrong. try in this way (you are missing ?key=).
string dataFeedUrl = "https://www.google.com/analytics/feeds/data?key="+key;
refer this google example where there is this example that should help you
public DataFeedExample()
{
// Configure GA API.
AnalyticsService asv = new AnalyticsService("gaExportAPI_acctSample_v2.0");
// Client Login Authorization.
asv.setUserCredentials(CLIENT_USERNAME, CLIENT_PASS);
// GA Data Feed query uri.
String baseUrl = "https://www.google.com/analytics/feeds/data";
DataQuery query = new DataQuery(baseUrl);
query.Ids = TABLE_ID;
query.Dimensions = "ga:source,ga:medium";
query.Metrics = "ga:visits,ga:bounces";
query.Segment = "gaid::-11";
query.Filters = "ga:medium==referral";
query.Sort = "-ga:visits";
query.NumberToRetrieve = 5;
query.GAStartDate = "2010-03-01";
query.GAEndDate = "2010-03-15";
Uri url = query.Uri;
Console.WriteLine("URL: " + url.ToString());
// Send our request to the Analytics API and wait for the results to
// come back.
feed = asv.Query(query);
}
refer also this guide to configure your project
Also follow this guide to use OAuth 2.0
I am trying to connect to the Google Analytics reporting API to get basic pageview stats. Im trying to follow this tutorial (http://www.arboundy.com/2012/04/getting-started-with-google-analytics-in-c/). I'm having trouble setting the correct bits to get a successful auth as it seems google has changed the APIs a lot lately so the original config doesn't seem to work.
Heres what I currently have:
Service = new AnalyticsService("MyDemoApp");
Service.setUserCredentials("user#gmail.com", "password");
AccountQuery AccountsQuery = new AccountQuery("https://www.googleapis.com/analytics/v3/data/ga"/*Not sure what goes here this gives a 400*/);
AccountFeed AccountsFeed = Service.Query(AccountsQuery); // 400 error here
Any ideas how to connect to this via the V3 api (which appears to be the one I got from NuGet)
this must work for u in c#. (i have tried and worked)
string username = "youremailuser#domain.com";
string pass = "yourpassword";
string gkey = "?key=YourAPIkEY";
string dataFeedUrl = "https://www.google.com/analytics/feeds/data" + gkey;
string accountFeedUrl = "https://www.googleapis.com/analytics/v2.4/management/accounts" + gkey;
AnalyticsService service = new AnalyticsService("WebApp");
service.setUserCredentials(username, pass);
DataQuery query1 = new DataQuery(dataFeedUrl);
query1.Ids = "ga:12345678";
query1.Metrics = "ga:visits";
query1.Sort = "ga:visits";
query1.GAStartDate = new DateTime(2012, 1, 2).ToString("yyyy-MM-dd");
query1.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");
query1.StartIndex = 1;
DataFeed dataFeedVisits = service.Query(query1);
foreach (DataEntry entry in dataFeedVisits.Entries)
{
string st = entry.Title.Text;
string ss = entry.Metrics[0].Value;
visits = ss;
}
for more details Read data from Google data API